Lidar typically provides 3D data that facilitates visualization, so that the viewer can interpret geometries and relationships of objects and terrain. Adding color to such data is usually a part of the mapping process that is carried out with photography, either simultaneously or separately to the lidar acquisition. This article explores the prospects for collecting, visualizing and using the color information that is based solely on the lidar instrument—multispectral airborne laser scanning.
Figure 1: The HeliALS-TW multispectral laser scanning system built at FGI is operated on a helicopter and acquires 3D data with reflectances at 532, 905 and 1550 nm. The system can be fitted with passive cameras, such as Specim AFX10 and AFX17, for comparative and complementary hyperspectral data. The tilted scan plane for the frontal scanners permits records from tree stems and other vertical objects; the VQ-840-G operates with a conical scan pattern, instantiating similar capability.
Data to address environmental change
There is a need for accurate and timely data for geospatial analysis of anthropogenic and natural phenomena in our changing environment. The vulnerability of forested and water ecosystems to different distresses is escalating due to the warming climate. Other weather-induced mechanisms that lead to ecosystem decline are of concern. Accelerating urbanization and use of natural resources have an impact on the environment and the livability of human settlements. The headlines in our daily news often report disasters arising from a chain-reaction beginning from a drought, or from wind- or snow-induced damages—these often lead to forest fires, plant diseases and pest infestations, such as spruce bark beetle. Excessive rainfall, flooding and erosion is the other group of forces to affect our environment in many ways with destructive power and long-term implications. The Finnish Geospatial Research Institute (FGI) is partnering with research, industry and public organizations in Finland, Europe and globally to address these challenges with innovations and novel technologies and approaches. These provide efficient tools and means for assessing our environment and studying mechanisms of decline, recovery and revival to help the urgent actions required to mitigate such challenges.
Multispectral laser scanning
Much is being done at the frontiers of passive remote sensing and UAV applications with the help of artificial intelligence and machine learning to interpret the massive environmental data acquired. However, there is a need for precise assessment of the ground truth, improved performance, and also new capabilities that are sought from the development and use of multispectral laser scanning. This was tried some time ago for a commercial airborne application, but has not yet found its place as an operational mapping tool, mostly due to technical issues and subsequent sparsity in the data products for reliable analysis in the early implementations.
Multispectral laser scanning, or lidar, offers the possibility to overcome certain drawbacks in passive multi- or hyperspectral imaging. With an active approach we can circumvent the effects caused by natural variation in the external light source, the Sun. Instead of calibrating and correcting the data for irregularities in irradiance, due to sun angle (seasonal and diurnal variation) and clouds, we can reduce the problem significantly by using the instrument’s own light source—laser. This also reduces the effect of shadowing, since targets that are not illuminated by the beam are also not shown in the data. Consequently, however, the flight pattern needs to be planned so that data gaps are avoided in the target of interest. Active lighting by laser beams also permits color vision in the dark, which is not possible with optical imaging. This feature could have new potential in disaster management and surveillance.
FGI HeliALS-TW triple-wavelength airborne laser scanning system
FGI has built instrumentation for multispectral laser scanning to provide precise ‘pseudocolor’ 3D data from the air to study technical solutions, limitations and feasibility and to demonstrate new possibilities in operational mapping and monitoring applications (Figure 1). The current implementation probes the objects with a triplet of wavelengths, 532, 905 and 1550 nm, using three laser scanners mounted on a single platform. Operational altitudes range from 80 to 400 m above ground. Higher altitudes up to 800m can be achieved in a dual-wavelength setting with 532 and 1550 nm scanners [with inherently sparser point density]. Recorded backscattered echoes are turned into object reflectances (compensated for range and compared to a white perpendicular target at the same distance) that can be used for visualizations and data analysis with computational methods such as semantic segmentation and machine or deep learning.
Figure 2: Fusion of green (532 nm) laser reflectance and point elevation.
Another advantage of the active illumination of the targets is the capability to reconstruct them in 3D. This helps assessing geometric features of the objects of interest, for example, the heights of trees, canopy structure, terrain elevation or artificial objects such as pavements, buildings or cars (Figure 2). It also helps in correcting for irregularities in the reflectivity data stemming from the object geometry, or detecting changes in the area, e.g. fraying of foliage indicated by increased penetration, or of a newly built structure. The penetrability of the laser pulses through vegetation allows for analysis of undergrowth and terrain features under the forest canopy, but also enables signal features to characterize the object based on the pulse scattering data, e.g. spatial echo distribution, pulse deviation and echo type.
Applications
Precise multispectral 3D point-cloud data, usually hundreds or thousands of points per square meter, allows individual tree segmentation, species classification and estimation of timber/stem quality and growth-related parameters to assess, for example, the effects of forest management. In addition to the capture of dominant trees is the capability to detect trees in the subdominant layer as well as the undergrowth better than ever before. Precise data also allows quality-related assessments, for example, stem curve and branch size distribution, or canopy damage (due to wind, snow, ice)—all parameters with an impact on forest resource management.
Figure 3: Reflectance composition of the multispectral 3D laser scanning data of a bark beetle study site collected with the FGI’s HeliALS-TW system shows decayed trees that can be clearly detected from the data by their darker and frayed appearance. The study aims at methods for detecting early infestation and modeling and forecasting the spread, based on the first signs of foliage drying and color change.
In an ongoing research project the group is developing methods to find indicators of the detection of spruce bark beetle infestation in the early stages of swarming of the new generation of pests, in so-called green attack phase. Severely frayed and dead trees are easier to detect and they provide an indicator of the spread of the infestation over multitemporal data to permit analysis of the effect of the forest structure and other spatial variables such as species distribution, forest density and prevailing wind conditions (Figure 3).
Figure 4: Reflectance composition of multispectral lidar point cloud data to study urban mapping and vegetation: 532, 905 and 1550 nm.
Urban environment and autonomous driving
Multispectral laser scanning from the air also provides new capabilities for assessing urban space and vegetation (Figure 4). Better data and analysis tools help manage the urban environment, land use and deployment of resources. Detailed data is explored to help automated mapping and data classification using machine and deep learning techniques to bring order and structure into the point-cloud data. In addition to urban vegetation, the data and computational technologies allow us to pursue new approaches towards maps specialized for autonomous driving use, and also for improved road asset management and urban planning.
Figure 5: A street scene acquired with a two-wavelength MLS system that collects precise street and urban asset data. Reflectance data can contain valuable information for automated detection and processing. This visualization is based on fusion of the point elevation and reflectivity.
The multispectral approach is also being developed and investigated for mobile laser scanning (MLS) on the ground. At present, however, the system uses only 905 and 1550 nm lasers (Figure 5). Through fusion of 3D structure, multispectral reflectance and signal echo deviation information, one can, for example, infer building and road materials and pavement defects using unsupervised and supervised learning techniques (Figure 6). In a forest setting, using a backpack application, there is a new method to separate terrain, tree stem, branches and foliage based on a single-profile scanner data within a deep-learning pipeline providing high reliability and computational speed. These developments provide solid steps towards real-time data analysis, whether the data is acquired by an airborne or ground system.
Figure 6: Training of a deep learning neural network to detect objects and materials from the dual-wavelength MLS data. Image courtesy of Mikael Reichler, FGI, 2023.
Freshwater research
Arctic environments are experiencing drastic changes. Melting of permafrost increases the instability of the ground, increasing erosion and eventually loosening sediments into the arctic rivers. This has a direct impact on the fishery and river habitats in these systems. Another risk is posed by erosion of toxic sediments from mining and industrial activity caused by increased discharges in rivers close to human populations. Moreover, increasing winter rainfall and flooding as a result of oscillating temperatures during winters is expected to emphasize the effects.
Figure 7: Shaded-relief terrain and river-bed model provides a basis for monitoring the evolution of arctic rivers and habitats critical for arctic fisheries and other species. Multispectral laser scanning provides data to monitor the changes in the environment comprehensively. Image courtesy of Harri Kaartinen, FGI, 2022.
Sediment transportation is also related to bed sediment roughness and vegetation. Multispectral laser scanning, along with imagery, is being investigated to develop tools to detect and monitor the geomorphological and biospheric features and indicators to model fluvial and other freshwater environments and processes. Multispectral laser scanning with the 532 nm wavelength generates bathymetry that allows us to reconstruct the complete river channel morphologies (Figure 7). Multispectral composition 3D data of the vegetation, point bars and other landforms provides the capability to monitor changes and evolution over time with repeated aerial or ground surveys (Figure 8).
Figure 8: This cross-section of the multispectral laser scanning data from the Utsjoki river in Finland shows the river banks, surface of the water, rocks breaking the surface and bed topography down to a depth of 3 m.
The effects are investigated and the feasibility of new technologies to provide data for them are explored within the Fresh Water Competence Centre and Digital Waters Flagship (DIWA), where FGI is a partner providing capabilities, expertise and innovations in laser scanning for the wider community to tackle these challenges.
Summary
The Autonomous Mapping and Driving research group in the Finnish Geospatial Research Institute conducts geospatial research by developing integrated sensor systems, data acquisition, and data processing for a wide range of applications. Dense multispectral 3D data permits development of computational methods to analyze and monitor the environment, and build map products for current and future needs. The green laser of the wavelength triplet also allows bathymetric data from water bodies. By exploring possibilities, performance and solutions provided by precise multispectral lidar point clouds, we can meet the needs for improved data and automated analysis for the benefit of society and advance the deployment of novel mapping technologies and computational methods to operational use.