Figure 1: Keenland data set, circa 2005.
I have been in the business long enough to watch lidar point density go from single-line profilers with perhaps a point every few meters to the incredible density we get now with drone lidar systems such as our own True View 3D Imaging Sensors.
In the beginning, the technical focus of the community was firstly on vertical accuracy and secondly on vegetation penetration. If you think about it, a vertically accurate point on the ground every few meters is a big step up from boots-on-the-ground topo collection. No wonder the Federal Emergency Management Agency (FEMA) was one of the first to mandate lidar for flood-plain mapping: lidar made cyclic updates of Digital Flood Insurance Rating Maps (DFIRM) a reality. Figure 1 shows a lidar project from circa 2005 with a single-flight-line nominal point spacing (NPS) of 1 m (~3 feet). This was a fabulous data set for its time.
Figure 2: Row of HVAC units.
Nowadays we use lidar for direct 3D imaging with a variety of applications as broad as your imagination. However, we really have not moved much beyond the original assessment of lidar data quality.
We now use lidar for myriad tasks beyond constructing a topo map. We routinely collect transmission/distribution wires, compute stockpile volumetrics, model rail lines, monitor construction sites—you get the idea. One of the major concerns is what I call point-cloud “conformance.” Conformance is a measure of how well the point cloud fits the true object space. For example, if we are modeling something lumpy such as a gravel or coal stockpile, do we have some amorphous blob or does the point cloud closely follow the true shape of the object being modeled?
Figure 3: HVAC unit scan by True View 515 – 75m AGL.
Consider a row of HVAC units beside our new GeoCue main building (Figure 2). These units are rectangular with some transparency to the interior via the fan guards. We would expect to see rectangular peaks in the lidar data with a few stray interior points across the mid-section from penetrating rays.
If we examine data from a True View 515 3D Imaging System (3DIS), we see just what we expect (Figure 3). These data were captured during a lidar QC flight at 75 m above ground level (AGL).
Figure 4: System X HVAC row conformance.
We are testing a new, low-cost lidar system based on the Livox Avia (I’ll call this system “X”). This system is showing (among other issues) poor performance in the conformance area. Consider the same scan of the HVAC area using X (that is, 75 m AGL, all-flight-lines scan). As you can see in Figure 4, the conformance of X is really terrible. The rectangular HVAC units are reduced to a series of hay piles! I would be reticent to use this system for anything except perhaps a few very well-known scenarios (clear area; hard, flat surface) owing to this obvious gross inaccuracy in conformance. Now it could be that these errors are being introduced by the proprietary post-processing software, such as a poor data smoother; more investigation is needed.
An interesting note is that the vertical accuracy of X measured on flat, hard surfaces is in the neighborhood of 4.5 cm root mean square error (RMSE)—by the old vertical accuracy standards, a fine lidar sensor. In a feature article scheduled for a Q1-2022 issue of LIDAR Magazine, we will present a detailed overview of this sensor.
The takeaway message here is to look holistically at the performance of lidar, camera and 3D imaging systems. Vertical accuracy, expressed as RMSE against a set of check points, is a very important lidar parameter, but if conformance is as bad as shown in Figure 4, do you really have a useable system? I would say, for most applications, probably not. Today there are no generally accepted tests beyond vertical accuracy for lidar systems. Perhaps it is time we begin collaborating on some assessments that provide a better overall picture of sensor performance.