A 722Kb PDF of this article as it appeared in the magazine complete with images is available by clicking HERE
As I write this column our company (GeoCue Group) is preparing a new release of our desktop point cloud tool kit, LP360 (it seems as if I have been saying this for months!). One of the new tools included in this release is a measurement system for control and check point assessment for both horizontal and vertical tests. As I discussed in the last column, several new requirements will be levied on lidar data providers by the United States Geological Survey (USGS) for the emerging 3DEP (3D Elevation Program) data collects. One of these is adherence to the new LAS 1.4, Point Data Record Format 6 as was discussed in the last issue’s Random Points column. A second requirement will be testing to the new American Society for Photogrammetry and Remote Sensing (ASPRS) accuracy assessment and reporting standard.
The new ASPRS accuracy specification is called the "ASPRS Positional Accuracy Standards for Digital Geospatial Data." It is currently in the final public review cycle. You can access a copy in the Photogrammetric Applications Division section of the ASPRS web site (www. asprs.org). If you are a provider of lidar data and/or digital orthophotos, I strongly recommend you have a look at the new draft standard since it will replace the current lidar vertical error specification upon final approval.
I like the new reporting standard because it is very straightforward and simple. All basic reporting for accuracy is in terms of Root Mean Square Error (RMSE). New vertical accuracy classes are defined simply in terms of RMSE. An excerpt of the vertical accuracy reporting table is depicted as Table 1. The table lists assessments in terms of Non-Vegetated Vertical Accuracy (NVA) and Vegetated Vertical Accuracy (VVA). The difference accounts for the increased variation in Z that we observe when lidar data impinge upon non-hard surfaces such as tall grasses and under canopy. These standards for VVA will not work for digital surface models derived using correlated imagery (for example, semi-global matching techniques) because there is no foliage penetration; the data simply ride on top of the vegetated surface in an extremely inaccurate way.
Notice in Table 1 that a new way of specifying vertical accuracy classes has been defined. The vertical accuracy class is simply determined from the vertical RMSE. The simplest expression for this is in terms of the RMSE of the data in non-vegetated areas where the class is simply the RMSE expressed in centimeters. Note that except for acres for area, the new specification is metric only. Thus projects performed in feet will have accuracy class determined and reported in centimeters.
As an example, if the RMSE vertical error, NVA, of my project is 14.7 cm then the accuracy class is 14.7. This is quite a nice approach since lidar data are not quantized in the vertical direction. By this I mean we do not speak of the vertical "resolution" of lidar data as we do the horizontal resolution of imagery (e.g. "12 cm pixels").
An example from a test lidar project is shown in Figure 1. The horizontal components of the test are blank since we did not measure horizontal control/ check for this project. Note the vertical statistics to the right in the figure. The RMSE is 0.154 meters with a mean error of 0.002 meters. The new accuracy specification states that the mean error must be less than 25% of the RMSE unless specifically specified otherwise by the procurement specification.
Vertical testing of lidar data is performed by inserting the points to be tested (usually gound) into a Triangulated Irregular Network (TIN) and then projecting a vertical line from the control/ check point to the intersection with the TIN facet. The interpolated elevation at the point of intersection with the TIN is declared the probed elevation. Note that this is a three point linear interpolation. Since vertical testing involves only probing of an elevation surface, it can be computed with no interactive measurements (i.e. fully automatically). We have this testing method built in to LP360.
It is important to note the distinction between Ground Control Points and Check Points. In general, a ground control point (GCP) is used in the mathematical solution of the geomatics problem. For example, in conventional photogrammetric aerotriangulation, GCPs are used to tie the project to a real world system (typically a 2D or 3D geodetic datum). These GCPs are measured independently of the photogrammetric process, typically by a field survey using Global Navigation Satellite System (GNSS) techniques or conventional survey from a priori survey locations. Check points are acquired in the same manner as GCPs, but are not used in the solution. They are withheld and used in validation.
Notice the rule on mean error introduced in the new accuracy specification. Mean error in an adjustment (aerotriangulation for photogrammetry or sensor modeling in lidar corrections) generally indicate a systematic vertical bias in the data. Several common causes of vertical bias include an incorrect antenna height in the GNSS base station or a lever arm error in the data acquisition system (sensor to GNSS antenna parameters). It is a common practice to reduce bias during data geometric correction by simply adding a Z value of opposite sign to the mean error to all of the data points. This is a very bad practice without first performing a root cause analysis of the source of the overall bias. If the error is in the GNSS antenna offset, then merging this shift into the data has just introduced that same error into all of the lidar data! If, on the other hand, the error is due to lever arm offset errors in the sensor, the correction is valid. We think it is good practice to independently validate at lease a subset of the control to ensure that it is truly accurate.
Note that if the GCPs and CPs are correct in terms of the datum (no Z bias) and the data have been shifted to remove global bias then a non-zero mean error in the Check Points is indeed a strong indicator that something has gone wrong, warranting a detailed investigation.
Figure 2 shows a Control/Check Point target that we laid out for a small Unmanned Aerial System (sUAS) test. The image is blurry because I have magnified it beyond the ground sample distance (for the point of this discussion). In this example we painted cross targets with athletic field paint (completely safe to the environment) and measured the target centers with a GNSS survey instrument. This control point is indicated by the red cross. I have measured the location of the ground target with LP360 using the horizontal accuracy assessment tools. The measured center is indicated by the green cross. If there were no error at all, the crosses would be superimposed. The difference in location is the horizontal error, at this location.
The overall horizontal accuracy is shown in Figure 3. Horizontal accuracy classes are now reported in terms of the pixel spacing of the image product being measured (in our case, a digital ortho photo). Note that we automatically pull the pixel spacing out of the image transformation matrix; 0.063 m in our example. The horizontal accuracy class is defined to be the value of n such that (n + 1) multiplied by the pixel size is just above the RMSE. In the example of Figure 3, we have an accuracy class of 0 (the highest in the specification) because the RMSE is below the pixel size of 6.3 cm in both X and Y. Note that the mean error criteria of less than 25% of the RMSE is also met.
One question that does arise when measuring 3D lidar accuracy is where to probe vertical control; at the unadjusted location of the control point or at the adjusted location (i.e. the red cross or the green cross in Figure 2)? The initial consensus of the group defining the new accuracy standards was at the unadjusted check point location. This is based on the idea that the vertical control/check points should be placed in flat terrain areas so a small horizontal shift will not induce a vertical error. I am not sure if we can always meet this placement criteria so we provide an option in LP360 to measure in either location (defaulting to the unadjusted location).
I encourage you to have a look at the new specification, especially if your company has internal code developed for measuring and/or reporting accuracy. The draft standard is very clearly written and has several worked examples in the appendix.
Lewis Graham is the President and CTO of GeoCue Corporation. GeoCue is North America’s largest supplier of LIDAR production and workflow tools and consulting services for airborne and mobile laser scanning.
A 722Kb PDF of this article as it appeared in the magazine complete with images is available by clicking HERE