| GPS Consumer Series: Batch Processing of GPS Data, Part 2 of 2
By Chuck Gilbert Introduction
Last month we discussed a few of the general things to look for in a batch processor for GPS data. The features described included types of processing automation and the creation of log files that record a history of what processes were performed in a batch run. This month let's look at some details that are more specific to the data processing that is being performed.
The GPS processing functions that are typically run on a daily basis include:
1) data transfer to the PC,
2) differential correction, and
3) export to a GIS interchange format.
The three functions listed above are usually performed in this order and are all quite well suited to batch processing in that they can sometimes take quite a long time to complete and they rarely require any user interaction during the process. Typically, the user can begin a batch process and simply return later to view the results after processing has completed. Data Transfer
Regardless of GPS or GIS manufacturer, virtually all GPS-based data collection systems require that the GPS data be transferred from the field device to a PC or other office based computer. On the surface, this seems to be a very simple, straight-forward task. One might think that there is very little difference between one data download program and another. However, when the download is to be performed in batch mode, there are a few rather subtle features that can turn out to be important. Data backup is perhaps the most important feature to look for in a program that includes data download as a part of a larger batch process.
Data backup refers to the download program being able to create multiple copies of all the data files as they are created. Preferably, the secondary copy should be written to a different directory where the ensuing processes are not likely to touch it. Ideally such backup files will be written in a compressed form that will not only require less storage space but will also help protect the files from corruption.
The reason that a backup copy is so much more important during batch processing is because immediately after download, the data files will be manipulated by a long complex series of processes. The user is not likely to be present during the ensuing processing. If any process fails, and files are damaged or lost; the backup files will prove to be valuable in the extreme.
At another level, it is important that the download process be non-interactive for the user. The data files in the field device should be located and selected automatically. The user should be able to pre-configure whether some are all of the files are to be transferred to the PC, and also the directory to which all files should be transferred. Differential Correction
If the user did not use real-time differential correction, their data will almost certainly require differential post-processing. Differential post-processing can improve the spatial accuracy from about 100m to better than 1 meter.
When a differential correction is performed, the user is generally relied upon to supply a very important piece of information. Recall that a differential correction requires data from a stationary GPS receiver that is positioned at a location of known coordinates. It is the user that must supply these coordinates. In the best case scenario, the correct base coordinates are already encoded in the base file. However, there is a risk that these coordinates may be incorrect. If these coordinates are not accurate then every position in every rover files will be erroneous by the same magnitude as these erroneous base coordinates. Because of this risk, a user will often check these coordinates carefully before a differential correction proceeds. However, when running in batch mode the user may not be available for such as inspection. Therefore, a very desirable feature in a batch processor that does differential correction is the ability for the batch program to inspect the automatically supplied reference coordinates and determine whether or not the coordinates are likely to be accurate.
Such automatic inspection of reference coordinates can be handled in several ways. A good simple solution is to allow the user to pre-define the appropriate reference coordinates. The program can simply use the pre-defined coordinates. A very clever program may even allow the user to store several user supplied reference coordinates and compare the coordinates found in the base file to any set of coordinates in the pre-defined list. When the base file coordinates have been gleaned from the supplied base file, an interesting technique employed by some programs is to compare the coordinates from the base file to those that were found in previously used base files. If the coordinates differ, the user is warned. If they are the same the process is allow to continue. This technique is particularly nice when base data is typically obtained from the same base location everyday.
Finally, differential correction programs typically have a variety of processing options available to the user. These may include such options as whether to output only the successfully corrected data or both corrected and uncorrected position, or whether to reprocess any real-time corrected positions, or whether to apply any advanced processing techniques such as filtering, smoothing, or disregarding certain satellites. When the differential correction program is run in batch mode, the user should still have access to all of the same options as when the program is being run manually. Export
Often the final stage of a batch process is the export of the data from its native GPS format to an appropriate GIS compatible interchange format. To a greater degree than differential correction, export programs can have many configurable options. The way these options are configured can be crucial to ensuring that the export process is accurate and complete. The sidebar lists a few of the options that are available in any mature export program. As with differential correction, when an export program is run in batch mode, the user should still have access to all of the same options as when the program is being run manually. Summary
A well designed batch processor will have two characteristics; it will provide an untouched set of backup data should anything go wrong, and it will offer the full range of processing options that are available when processing manually.
The backup data could prove to be priceless should files become corrupted in the course of processing. Although such file corruption is highly unlikely, automatically created backup files are a very small price to pay for the additional data security.
When GPS processing is performed in batch mode users should not be required to give any processing options that are used when processing manually. The integrity and consistency of the GPS data can depend very much on which options were selected during differential correction and export. Therefore it is crucial that data can be processed in exactly the same manner whether it happens manually or with automation. About the Author:
Chuck Gilbert has over a decade of experience as a GPS user. He has been employed as an applications engineer for Trimble Navigation since 1989. Back |
