Three-dimensional Utility Data Conversion & Utilization
By Adam Chadwick

Introduction
Hardcopy utility records, constructed and maintained over time to reflect the planimetric location of underground facilities, are often augmented with profile information that shows the distance from the ground (typically a road surface) to the underground facilities - in other words, the depth of utility. Utility facility profiles are added to this hardcopy medium to create a plan-and-profile drawing. This profile information is initially captured to increase the accuracy of the underground location information, especially in the case where digging must occur in the vicinity of the facilities. However, it can also be used to help build a network model of the facilities for purposes of analysis. This is especially apparent for pipe-based, fluid-carrying types of utilities where the slope of the pipes, necessary to properly define the model, are indicated in the profile view.

This one-meter-resolution orthophoto is displayed at zero elevation with the pipesbeing shown at their real elevation (above sea level—in this case at approximately 360 meters), another way of comparing the underground pipes with aboveground features.    

     Once a complete set of plan/profile/model records are in place within a utility network, a highly accurate definition of that network is available to support the majority of activities, including planning, design, construction, maintenance, and repair. However, these types of records, built up over time from whatever technology was commonly available, rarely are in a digital format. Nor are they usually housed within the same database. This diversity results in utility facilities being stored and maintained in as many as three different recording systems: one to represent a facility's plan location, one to represent a facility's profile location, and one to represent a facility's network model. Storing and maintaining the various components of utility facilities in different recording systems presents two essential problems.
      First, maintenance of utility facility records is not performed equally across every system. For example, the position of a pipe may be added to the plan-and-profile record, but not to the model itself. This may be due to time constraints, lack of immediate need, operator skills, or process flaws. Second, given the duplication of various data items among recording systems, and in the absence of technology or methodology to ensure a tight link between representations of facilities in the various recording systems, a pipe's diameter as recorded in the plan system might not be the same diameter as was noted in the model system. Inconsistent data under these circumstances is a common problem.
      The components of facility records from various recording systems - plan (xy), profile (z), and model (connectivity) - cannot be easily integrated and analyzed within a single system.
      These problems ultimately lead to a desire to house and maintain utility records as part of the same recording system. This approach eliminates problems associated with disparate systems as described above. In order to do this, an intensive data cleaning process known as "scrubbing" takes place, as well as a data conversion task that translates existing utility records into a system capable of housing and maintaining each of the important components previously mentioned.

Information Analysis and Utilization: Early Requirements
Prior to converting utility facility data to a single, integrated system, one must consider the possibilities available for plan/profile/model data in an integrated digital environment. It should be possible to replace all original source records with the newly converted data. But there is a big difference between simply putting existing records into digital form and then replicating the original source products from that new system, as opposed to creating an entirely new data repository and resultant output products. The former approach, seemingly simpler because it introduces less change into the organization, can actually lead to greater complications when attempting to produce the original products. Conversely, the latter approach ultimately provides for a more complete and capable system in the longer term, even though it may be more difficult to initiate.

Planimetric Map
Planimetric maps in the pipe utility industry must typically serve two purposes, first to record the accurate location of utility facilities, and second to represent the operational network. In a hardcopy environment, having two separate map sets produced at different scales satisfies this requirement. In the digital environment, these two map scales can be produced from a single set of planimetric data by using generalization and scale-based symbology. An operational network map contains less information than an infrastructure location record map (known as the 'as-built' map). Consequently, this product can be generated by dropping out detail from the as-built map and enhancing whatever devices are of interest to an operations staff, for example "normally open" valves.

Profile
In a hardcopy records environment, profile drawings are typically placed either above or below the plan drawing on the same map sheet, hence the term plan-and-profile drawing. This allows the user to match a valve or manhole shown on the profile drawing to the same valve or manhole shown on the planimetric drawing, simply by looking up or down the sheet. To replicate this scenario in a digital environment, fixed map sheets must maintain some method of associating pre-generated profiles with a specific location on a map page. Since this does not allow for the production of plan drawings with associated profiles for any area within a digital utility database, this type of approach would be overly restrictive.
      To avoid this problem, a method must be developed to generate profiles for any area, and for that profile to be placed anywhere on a plan map - or even separately - yet still allow a user to visually associate facilities in the plan drawing with the same facilities in the profile drawing. One method of accomplishing this is to ensure that, during conversion, unique identifiers for valves, manholes, or any similar facility are captured and displayed on the plan drawing. When the profile drawing is generated from the plan information, these unique identifiers are displayed so that facilities on two physically separate outputs - one the plan, the other the profile - can be matched up visually using these selfsame identifiers.
      Another solution that reduces, but does not eliminate, the need for a profile drawing, although it still portrays the necessary elevation information, is to display on the planimetric map the depth of pipe endpoints where devices such as valves, manholes, or catch basins are located. Since inverts and rim elevations (road surfaces) can be recorded in the GIS with the pipes and devices respectively, by utilizing the connectivity of pipes to devices, the pipe depths at those devices can be accurately determined. The difference between the pipe inverts and rim elevations can be displayed as notations on the map at a predefined distance from the pipe endpoints, thereby showing the pipe depth at that location. For in-field applications, this information can be used as an approximate guide for field workers who are digging in the vicinity of underground utilities.

Network Modeling
For the connectivity model of the network to be derivable from infrastructure facility records, it must be possible to directly export the data to the properly selected network modeling system. However, if this sort of high-level systems integration is not available, output routines must be able to format the utility infrastructure data to suit the network modeling system. From a data-conversion perspective, it is important that the information required to adequately create a network model is available in the GIS once it has been loading with the converted utility data. This implies that sufficient three-dimensional infrastructure information is also available. By converting pipe elevations into the GIS and including this information in the infrastructure utility facility data, pipe slopes and true utility facility connectivity can be derived.
      Network modifications should be done on the source infrastructure facility records data in the GIS, and then exported to the network modeling system, instead of making modifications directly in the network modeling system. Also, since data in network modeling systems are updated frequently - as the infrastructure facility data changes - it is vital that network operations engineers are working with the most up-to-date information. For this to happen, the transfer of source infrastructure facility information to the network modeling system must be fast and simple.
      All too often, after struggling to get infrastructure facility information into a network modeling system, and then struggling further to correctly model the network, operators are reluctant to refresh the information because of the amount of work that is involved in re-creating the network representation. The utility network, modeled in the network modeling system, becomes a new source of information as changes to the network occur, as opposed to the actual source of that information, which is the infrastructure facility records data in the GIS.

Data Conversion
The old adage, "Garbage in, garbage out," couldn't be truer when it comes to converting any kind of data. However, it is even more apparent when the data being converted comes from many different sources, and contains complexities not typically encountered. It is for this reason that data conversion of three-dimensional information must be approached in a systematic and controlled manner.

Data Scrubbing
Whatever the decision may be for the most appropriate sources for conversion, it is important that there are as few different data sources as possible. To that end, there should also be a minimal number of anomalies or differences between the various files, maps, or records. In cases where automation has taken place, complete digital information may not be available for all sources. While the operator may be tempted to convert the "new and accurate" digital information and the older hardcopy information, integrating hardcopy and digital information during conversion can be problematic for several reasons. First, there is the difficulty of automatically checking the various aspects of a "record" during scrubbing. Second is the difficulty of combining information from the disparate media during conversion. Third are the different levels of currency and accuracy of the differently managed sources. Then there is the handling of different sources during conversion that often distracts operators from their real data conversion tasks, and the confirmation of data integrity.
      For these reasons, it is prudent to reduce the number and types of sources. Typically this would be hardcopy rather than digital, and most likely would be the infrastructure facility records map. This map usually depicts the greater portion of infrastructure connectivity; however, the profile drawings would have to be included as a conversion source. Plus the amount of potential elevation information available for conversion could actually be too much information to realistically scrub onto the infrastructure maps.
      Since it is likely that the older records are the source of information, it is imperative for experienced people to be employed during the scrubbing process. Long-time employees generally have a much better understanding of existing records than do newcomers. But while the experience of data scrub personnel is important, no one can know everything. Field checking is a must for those cases where sources are in conflict, or where information is missing. If a utility company already has field staff out performing other tasks, it is often useful to make these staff members available to the data scrubbing staff for in-field verification. After all, once the data has been converted and put into production, the field personnel are dependent on its accuracy.

Quality Assurance
Quality assurance, an important part of a data conversion project, gets more complicated once three-dimensional issues are involved. Not only do layering, connectivity, attribution, annotation, and location aspects of the newly converted data have to be checked, but issues related to three-dimensionality require verification as well.
      While views often differ as to whether outside data conversion contractors should be given the same data checking routines as are used by the customer, the finer aspects of network connectivity and three-dimensionality may not be apparent to most data conversion contractors. As a result, these factors may not get checked. The more information and understanding a conversion contractor possesses about three-dimensional connectivity, the better will be the converted data.

Data Maintenance and Integration
Primarily for volume and source content reasons, data is converted one piece at a time. That piece may be area-based, network-zone-based, theme-based, or some other type of data segmentation. Regardless of the approach that is taken, data that has already been received, checked, and accepted from conversion must be matched with neighboring data as that is received, checked, and accepted. Once integrated with existing converted data, data maintenance must be performed as changes occur to the system. In addition to traditional data-capture automation tools, the consequence of three-dimensionality must also be considered. For example, capturing coordinates for pipes on curves (in xy) that are not perfectly flat should be automated so that elevations for each xy coordinate are not calculated manually - a painstaking task. Whenever pipes or devices attach at the same elevation, data-capture tools can be made to assume the connected-to feature's elevation for the feature that is currently being captured.

Utility Operations
Once data has been converted to a single, three-dimensional data source, new possibilities become available. Profile drawings, once a source of information on their own, can be generated either for general use throughout an organization, or for use in the field. Additionally, profiles can be created for specific on-the-fly areas, for example, where profiling of a non-traditional area is required. This could be defined as an area of multiple city blocks for the purpose of street reconstruction and utility replacement design drawings.
      Custom-content profiles can be created so that only certain types of system are profiled, or where additional pipe attributes are shown on the profile. Operational maps can be created at varying scales, with content showing a highly generalized view of the utility system depicting only major devices, or a more neighborhood-type view of the operational system depicting all pipes, devices, and their normal operating status.
      System-modeling software can be used on a more ad hoc basis, for example, to check current system capacity within a specific area and verify the required capacity to add new utility customers. Ad hoc modeling can also be done in specific system areas to track down chronic system problems. These activities are performed on the three-dimensional source data by either directly reading the data from within the network modeling system itself, or through a simple transfer process. No longer is it necessary to permanently store network information within the network modeling system, as source data becomes available through the newly converted and integrated data.

Summary
Given the recent possibilities brought about by advances in GIS capabilities for the storage and manipulation of three-dimensional geographic data, new ways of managing utility information are now available. By carefully considering how these capabilities can be used, conversion of utility information may have to be done differently, and more extensively, to ensure that more usable and complete utility information is available in the GIS. The biggest change brought about is the migration from multiple hardcopy and/or digital sources to a single digital source, one from which traditional map and data products can be derived. Once this concept is adopted, plan, profile, and network model representations of the utility facilities become outputs more easily derived from the single-source GIS utility data set.

About the Author
Adam Chadwick is the GIS manager for the city of Kamloops, British Columbia, Canada. His responsibilities include ongoing implementation of the city's GIS infrastructure. He may be reached via e-mail at [email protected].

Back