A 2.219Mb PDF of this article as it appeared in the magazine complete with images is available by clicking HERE
In the past twenty or so years, LiDAR technology has evolved significantly. In the beginning of commercial LiDAR, sensors operated at pulse repetition rates as little as 5kHz and only could receive first and last pulses or both. That amount of data at the time was massive and computer requirements were a big concern. The required computers would cost $4000 to $5000 dollars to handle and manipulate the data. There was only one commercial software package and a few proprietary packages that could handle the processing and classification of the LiDAR data.
Currently, there are all types of LiDAR that operate on many different platforms. These systems typically operate at 500kHz to 1 MHz with multiple pulses in the air and the ability to receive multiple returns. These are discrete return systems. There are photon counting, full waveform, bathymetric and some we don’t even know about. Typically, these sensors operate at 532nm to 1550nm on the light spectrum. This doesn’t include the methane detection sensors that operate at 1619nm to 1692nm. Additionally, these sensors are coupled with other remote sensing technologies.
When you put all of these together you end up with a massive amount of LiDAR data and the question of how to store it. Do we store it in private locations? Do we store it on hard drives, disc servers or the Cloud? Do we store it so the public can use it? How can we manipulate it when it is stored? And perhaps, "Is there a Google Earth for LiDAR?"
This article will attempt to answer these questions. To begin a search on the web yielded a lot of results. The LiDAR hosting sites found do a little of everything, some things very well and I am sure within the next couple of years they will continue to evolve, as did the hardware technology itself.
The results of the internet search may not have revealed all the available options, but this is what came up. LiDAR Online, I-cubed DataDoors, OpenTopography, Euclideon Geoserve, QCoherent LiDAR Server and ERDAS Apollo. Also some specific data hosting sites came up for a specific completed LIDAR projects. An example of this would be The Oregon Lidar Consortium. I am sure that several government sites have hosting ability, or use one of the sites listed above. For example, Open Topography hosts all of the State of Indiana data.
Private and public entities often want to host their LiDAR, data but sometimes they have legal and/or security reasons preventing tem from allowing public access to their data. There are legal and political reasons for not making data available, but this article will not discuss that aspect.
LiDAR Online seems to be a site to market LiDAR data over the web using a login and security protection. DataDoors as it relates to LiDAR data seems to be a similar set up. NSF OpenTopography is a site that relies on funding and anyone can post their data on this site. Euclideon’s Geoservse is a rapid and limitless point cloud streaming product line that can access local data or data stored on the Cloud. An evaluation of the software was performed by Jarlath O’Neil-Dunne in Vol.4, No.4 of Lidar News. LiDAR Server is a mass storage and server software product made by QCoherent and owned by its parent company GeoCue. Additionally, ERDAS has a similar software package called Apollo.
It is strongly encouraged to contact each of these sites directly to understand what is available from each. It is important to understand how they will best work for your LiDAR application and intended use of the hosting site. This article is going to concentrate on a couple of the listed sites and software packages above.
OpenTopography was started by a group of individuals lead by Chris Crosby, while he was at the Unviversity of California, San Diego and is available through the San Diego Supercomputer Center at the University of California, San Diego. It is funded by a National Science Foundation (NSF) grant. I first learned about this site when I went on a trip to Egypt a couple of years back with Chris and several other colleagues to do a LiDAR workshop for the Egyptian government which was sponsored by Hamilton College and directed by Professor Barb Tewksbury. Is it the Google Earth of LiDAR? I will let you decide that.
OpenTopography is one of the best publicly funded hosting sites I have found. The primary intent of the site is to serve LiDAR data. The overview video on the site is a very good way to understand what is available on the site. The data on the site can be obtained along with Google Earth hillshade, DEM and point cloud format. The resource tab provides an explanation on how to use the data. A guest on the site can download up to 50 million points per job and a user can download up to 100 million points per job. It is easy to sign up to the site to become a user. The site also provides job tracking.
There are several tabs on the site that explain and provide information to the users. The About tab details the philosophy, the mission of the site and why it exists. The Data tab is the link to available OpenTopography data. The Tools tab is "The OpenTopography Tool registry which provides a community populated clearinghouse of software, utilities and tools oriented towards high-resolution topography data (e.g. collected with LIDAR technology)" as indicated on the tab. The Resource tab explains the data and how to use it. The Community tab provides information on ways to interact with the Open Topography community through social media. There is also a Contribute tab to contribute LiDAR data to the site. Lastly, the Support tab allows the users to get support on the site. This site appears to cover all the requirements of a public LIDAR hosting site.
While OpenTopography is an excellent central repository and data dissemination portal for LiDAR data, it is more of a "Google Map" application for LiDAR which is by no means a bad thing or should retract from the quality of the site. Google Earth provides users the ability to open the application, navigate to any place on the earth, and almost instantly see geospatial data in 2D or 3D. The user does not have to know where to locate data, wait to download files, find an acceptable program to open the downloaded files in, possibly reformat the downloaded data, etc. Google Earth does all of this behind the scenes and the user just accesses streamed data for a geographic region of their choosing. The experience is fluid, easy to use, and widely accepted. A system similar in some ways to this has actually been designed for high speed access to LiDAR and other 3D datasets. Euclideon’s Geoverse products provide fast 3D data access from local disc, network server, or internet accessed cloud storage data and performs well considering the challenges of rendering and streaming unlimited amounts of 3D point cloud data.
Euclideon’s Geoverse is a software product line that is built upon the company’s core technology called Unlimited Detail (UD). UD was originally developed by Euclideon for the gaming industry to remove the limitations of polygon rendering by using ultra dense point clouds to rapidly stream 3D models instead. UD is a high speed 3D data streaming technology that requires low memory usage, minimal data bandwidth requirements, and is completely scalable across data size, extent and point density. Geoverse is Euclideon’s geospatial product line which uses UD. There are two applications in the Geoverse product line–Geoverse Convert and Geoverse Massive Data Manager (MDM). Euclideon also provides all of the functionalities of both applications in their Unlimited Detail Software Development Kit (udSDK). The udSDK provides a lot of potential to access this technology from within other LiDAR software products giving the user the ability to have existing and proven software tools along with this new point cloud rendering disruptive technology. For example, the ability to integrate Geoverse functionality with a product such as Merrick’s MARS software is now a possibility. The LiDAR user could have the full capability to manipulate LiDAR data, generate required products and host all the data on a secure server from within one package.
The Geoverse software has the ability to bring in billions, even trillions of points from multiple project locations. Euclideon has tested the system up to 232 trillion points with no loss of performance or stability. The data must be first converted to Euclideon’s UDS format for high performance 3D point cloud streaming and dissemination. The conversion process supports about a dozen additional input 3D file formats beyond just LAS. It converts data at a very high speed of up to 4 billion points per hour, or approximately 1 million points per second. Once the data is loaded into MDM (in less than 1 second and always viewable at full resolution), the user has to ability to navigate the point cloud in 2D map space, stream underlying Bing or Open Street Maps WMS, navigate in a 3D perspective view, add bookmarks and labels, record flythrough trajectories and export regions of data to LAS format. LAS export from UDS is the only output file format currently supported. UDS data can be accessed locally or via the internet by using secured (password protected) or unsecured URL paths. This is a very good step in the evolution of Google Earth like use of secure LiDAR data hostings.
As noted in Jarlath O’Neil-Dunne’s review article the compression is significant, and has the option to be lossy or lossless, but it seems with available server space not being an issue the real benefit is having access to all the data at one time, plus having the ability to cut out portions of the data for an AOI on demand in LAS format. There might be questions as to what if the data needs to be in another format for say a transportation or engineering project. The simple answer is that since the SDK is available, the LAS can be converted using the sister software that Geoverse is embedded with. At some point there may be other formats that Geoverse will support. There are advantages to having the ability to manipulate the data in the LAS format, such as further classification or refinement with a software such as MARS.
The Geoverse software provides some unique tools and much potential. The ability to bring in other source material and the ability to access WMS data as a reference to the dataset come to mind. This is almost as fast as the data is populated in Geoverse through the UDS format. The View tabs include view by color (typically Camera Colored encoding), intensity, classification, elevation and contour providing additional information about the LiDAR data that may not be realized in other hosting technologies currently. The Classification tab provides the user the ability to turn classes on and off if they exist in the data.
Although all of the LiDAR hosting packages have not been discussed there is a growing demand for this type of solution within the LiDAR profession. There are hosting sites unique to specific LIDAR clients that have been developed internally. There are several software providers that have developed and continue developing tools and public sites available to fill this need. The OpenTopography and Geoverse products are just two of the many options that offer much promise regardless of the users need, application and budget for this type of service.
Jamie Young is Senior Geospatial Technologist for Merrick & Company in Denver, Colorado. He has twenty years of experience in remote sensing including 18 years in LiDAR. He has worked with all types and in all phases of LiDAR.
A 2.219Mb PDF of this article as it appeared in the magazine complete with images is available by clicking HERE