3D Mapping on the Move: Making Fantasy a Reality

The Tricorder from Star Trek and the “pups” from Prometheus are Hollywood’s idea of a futuristic means to sense and capture surroundings instantly—almost magically—wowing onlookers with three-dimensional representations right before their eyes. Now Kaarta1, a pioneering Pittsburgh-based startup, has brought this science-fiction fantasy to life by deploying its robotics-based technology across many applications.

Kaarta captures real-world spaces and instantly produces a high-fidelity 3D map, or “digital twin,” of that environment. The company uses mobile lidar (light detection and ranging) scanners, powered by its proprietary software, to reproduce the surroundings digitally, on the go. Spaces can be natural or man-made, and as varied as office buildings, stadiums, warehouses, factories, bridges, ships, power stations, mines, forests, and archeological sites.

Editor’s note: A PDF of this article as it appeared in the magazine is available HERE.

Kaarta’s breakthrough technology is the purpose-built Kaarta Engine, a novel approach to real-time mapping and localization that intrinsically understands an environment, both capturing what is around it (mapping) and where it is in that environment relative to the map (location). The technology has roots in robotics principles—what Kaarta CEO Kevin Dowling (Figure 2) calls “adding robot smarts to 3D modeling.”

What brings this core technology to life is Kaarta’s range of scanning systems, equipped with an array of lidar, motion, and camera sensors that sense, scan, process, and digitally reproduce the surrounding conditions—all on the go without the need for external signals such as GNSS or prior maps.

Unlike Hollywood fictional tech fantasies, Kaarta solutions are real. They’re in use today across a range of applications in the built world, spanning architecture, engineering, construction & operations (A/E/C/O), industrial and infrastructure planning, and even in helping autonomous mobile robots intelligently navigate their workspaces.

Beginnings

So how did Kaarta begin? “It’s a fun story,” says Dowling, who holds a master’s degree and a Ph.D. in robotics from Carnegie Mellon University (CMU) and was part of the formation of CMU’s famed Robotics Institute and Field Robotics Center. “When I was a student back in the 1980s, a robotics problem was posed: it’s called SLAM, simultaneous localization and mapping. It is where you construct a map of an unknown environment while at the same time keeping track of an agent’s location within it.”

SLAM is a conundrum, and as such, it’s hard to solve. Still, a solution must be found for a robot to operate. People have worked on solutions to SLAM since the 80s, but Dowling didn’t see an effective approach until three decades later when he met up with an old colleague, Sanjiv Singh, and his graduate student, Ji Zhang, at Carnegie Mellon’s Robotics Institute. Zhang was developing a radically different approach to the SLAM problem. Dowling was astounded by Zhang’s ingenuity and could see how his work had far-reaching implications beyond autonomous robotics applications, the primary driver behind most SLAM research.

In 2015, Zhang and Singh founded a company to apply the technology to 3D modeling applications. Months later, Dowling joined as CEO to turn the company’s research into a business, commercialize its technology into products, and build out its sales, marketing, and engineering functions.

One of Dowling’s first changes was the company’s name. “I came up with the name Kaarta to connote cartography, the ancient art and science of mapmaking,” he says. The name—with the extra a added for effect—captures the company’s pioneering take on mapping to produce 3D models in real time. “The name is not about the technique we use, it’s about the problems we solve,” says Dowling.

Kaarta started with minimal capital and maximum sweat equity. At the onset, Dowling didn’t take a salary. While those initial scrappy times are now behind it, Kaarta still runs a lean operation, maintains its culture of putting innovation first, and keeps an “all hands on deck” attitude for the many tasks at hand.

In operation for three years, Kaarta has made impressive strides in continuous technology and product development, market penetration, and financial performance. Though the company is based in the U.S., more than 60 percent of its business is in Asia, Europe, and the Middle East. “We have taken a global approach to this business from day one and have capitalized on those markets that are early and eager technology adopters,” Dowling says.

“Investors,” he says, “are pleased with the company’s progress and realize its considerable market opportunity.” In March 2019, Kaarta raised a Series A round of funding to bring in additional talent and enable an accelerated path to further innovation, partnerships, and growth.

Building the team

Dowling came to Kaarta after 20 years of development and commercialization of industry-first technologies in robotics, semiconductors, LED lighting, wearable electronics, and consumer products. Experience taught him that hiring the right people with the right experience was critical for a nascent company.

He recruited a seasoned leadership team that included two former trusted colleagues: Wade Sheen would head up global sales and business development and Kathy Pattison would lead marketing initiatives. Sheen and Pattison brought plenty of startup experience and a growth mindset. “There are always difficult challenges and obstacles in charting the course of a new business. It’s much better to navigate those waters with a known and proven crew of smart and resilient people,” Dowling says.

Because of the company’s location in Pittsburgh, a hotbed of robotics and AI innovation, Kaarta tapped the CMU brain trust for its engineering talent. Heading Kaarta’s engineering team is Steve Huber, a rocket scientist who has always had a passion for robotics. Steve got his start in robotics in high school through the FIRST Robotics Competition2 program. Now, Kaarta is a sponsor of a regional FIRST competition, hoping to inspire the next generation of roboticists.

Technology and products

A cursory look might leave the impression that Kaarta is a hardware company, but closer inspection shows it really is a software company that packages technology innovation in a physical system. “The algorithms that make up Kaarta Engine are inside our systems, and that is where the magic happens,” Dowling says. “That is what differentiates us. We have artfully solved the SLAM problem without using traditional SLAM techniques.”3

Beyond Kaarta Engine, Kaarta’s software development focuses on the utility, usability, and capabilities of mobile mapping through its on-board software. The software delivers a more efficient workflow through advances such as providing confidence metrics to help users capture a better scan, to delivering new color capabilities, to reducing or eliminating post-scanning processes. The company is leveraging these innovations to build a large, diverse portfolio of patents on its inventions. Its first patent was issued in mid-2018, and many more are in the pipeline.

In 2016 Kaarta introduced Stencil, a modular and customizable 3D mapping and real-time position estimation system delivering capabilities tripod-based scanning systems lack. Stencil scans exterior and large interior spaces without the need for existing infrastructure, external signals, or prior maps. Far more than just a mobile scanner, Stencil processes while capturing, making intelligent decisions to use only good data, and generating a registered point cloud on the fly. Early on, Stencil’s accuracy was put to the test in the Microsoft Indoor Localization Challenge, where it grabbed first place in 2016 and 20174.

The recently launched Stencil 2 (Figure 1) improves upon the original system with a new user interface, wireless tablet display, and the addition of GNSS support to correct for drift over large areas.

In 2017 Kaarta launched Contour (Figure 3), putting the same fundamental capabilities in a sleek hand-held design ideally suited for interiors and tight, complex spaces. Contour is a totally integrated system with a color camera and its powerful Reality Layer which enables one-click cleanup and colorization. Outputs range from point cloud to surface mesh to rich photorealistic model.

Applications for Kaarta’s technology exceed the company’s market focus, but Kaarta works with partners to help them integrate the technology in other solutions through development kits and non-recurring engineering support.

Important to the rapid future advancement of its mobile mapping solutions and other applications through OEM or licensing partnerships is that Kaarta’s software is relatively hardware-agnostic. This flexibility gives Kaarta the agility to incorporate the latest off-the-shelf components: as better performing and new forms of lidar and other sensors come to market, Kaarta is able to evaluate and incorporate them quickly. Dowling summarizes, “The key aspect of our software is knowing how to integrate it with hardware so that the complete system delivers value to the user. The integration of best-in-class software and algorithms with the enabling hardware is key to Kaarta’s success.”

In addition to the above activities, Kaarta offers Kaarta Cloud5, a processing service to generate 3D models of raw Velodyne lidar data. In the near future expect additional services that will let users send captured data directly from a device with one click, store it in the cloud, and within a couple of days turn the captured point clouds into floor plans, CAD, and BIM models.

Limitless applications

Early on, hungry to prove its technical prowess, Kaarta brainstormed ideas to demonstrate its capabilities. “We wanted to map something more complex than a boxy, flat room,” Dowling says. So they went big—indeed, Herculean. They captured a 3D model of the exterior of a massive Lockheed C-130 Hercules military transport aircraft, to model it and determine the feasibility of implementing an autonomous paint-removal system for routine maintenance. Using tripod-mounted terrestrial lidar or drone-mounted lidar to capture the full surface of the aircraft would have been extremely time-consuming and would require many scans. Stationary systems would require many views to capture the complex geometric surfaces of the aircraft, especially near the engines and propellers, all of which would have to be processed and registered in post-processing. With Kaarta, the operator hand carried Stencil, first walking around the aircraft (Figure 4) and then using a lift to gain a vantage point to the top of the plane. Kaarta reveals the model through continuous motion so no surface is missed (Figure 5). “The area was large, curved, and complex, yet we had a registered point cloud in 30 minutes,” Dowling says. “The result is amazing.”

“Scanning complex structures meant there was little limit to what Kaarta’s technology could do, including a Frank Gehry building and old ruins,” Dowling asserts. An unusual project for the City of Pittsburgh again put the technology to the test. City workers discovered immense boilers under a building that burned down. Previously unaware that these boilers existed, the City asked Kaarta to scan the sunken area and pipework to document the site. “We walked through the space and it was dark, ominous, intricate, and detailed,” Dowling says. Kaarta technology quickly and easily captured the entire area (Figure 6). Matthew Jacob from the City of Pittsburgh Department of Public Works adds, “We really put [Kaarta] technology to the test. We took them all over the city. We took them to rooftops, to abandoned buildings, and we were pretty pleased with the results.”

A marine surveyor used Kaarta to scan the existing conditions of an oil tanker that sails the Gulf of Mexico to plan for ballast water treatment retrofits. Rarely in port for more than 48 hours, two decks high, and bearing an intricate maze of complex piping systems, this type of vessel has traditionally been a nightmare to capture. With Kaarta a highly accurate 3D model was quickly built.

On dry land in the woods of rural Maryland, the design, engineering and program management firm CH2M (now part of Jacobs Engineering Group) was testing another application for Kaarta technology. The project would test the rigor of Kaarta’s system and explore its use for mapping applications requiring high precision in GNSS-denied environments to integrate with advanced metal detection and classification systems for locating buried ordnance.

The status quo for positioning with these systems is time-consuming and does not provide positioning accurate enough for consistent results. The area’s rugged terrain and dense vegetation posed a special challenge but the survey (Figure 7) was completed within 30 minutes. All of the previously surveyed points were accurately identified and the difference between Kaarta’s rapidly scanned points and those that were meticulously surveyed through conventional means ranged from just 3.5 cm to 19.8 cm. “Precise positioning is fundamental to our work, and to produce these results with such relative ease and cost efficiency is eye opening,” says Tamir Klaff, principal technologist at CH2M. “We undertake many projects in environments where this technology will undoubtedly be a key problem solver.”

Kaarta set its sights high to test its Reality Layer colorization capabilities by scanning one of Carnegie Mellon’s Beaux Arts buildings. The entrance resembles a cathedral with an eclectic ceiling mural depicting images of great works of architecture. Kaarta captured it in all its glory in 20 minutes. The results in full color (Figure 8) were impressive and some viewers had to be reminded that it was a point cloud and not a photograph.

For facilities inspection company Fasttac, based in Pittsburgh, Pennsylvania, Kaarta’s colorization capabilities served more utilitarian purposes. Fasttac used Kaarta to capture the existing conditions of the boiler rooms of a major medical center (Figures 9 and 10). “Kaarta is a huge win for applications such as facilities maintenance, since it is incredibly easy to capture the layout of spaces that do not have up-to-date drawings, and that’s about 90% of existing buildings,” says Ray Steeb, CEO of Fasttac. “The key is that we can then use the scans right away without having to model.” Steeb was particularly impressed with the colorization of Reality Layer, which makes it even easier to identify any given area visually.

In a return to its roots, Kaarta is also seeing many applications for its technology in “mapping for robots”—aiding autonomous mobile robots programmed to do everything from cleaning floors and bathrooms to taking inventory to delivering packages to your front door. Even robots can benefit from the specialty of other robots.

Endnote

And so it comes full circle. Robot smarts perform miracles inside a system that captures the world outside, bridging the physical world with its 3D digital twin. With conceptual origins in solving a robotics problem posed decades ago, Kaarta is well positioned to answer those two fundamental questions of today: “Where am I?” and “What’s around me?”

Kathy Pattison is CMO of Kaarta. She has spent the better part of her 30-year career in early- to mid-stage startups that have brought transformative technologies to life, and really wouldn’t have had it any other way. Kathy also serves on the board of directors for numerous not-for-profit organizations.

References

Laser–visual–inertial odometry and mapping with high robustness and low drift
onlinelibrary.wiley.com/doi/epdf/10.1002/rob.21809

Visual-lidar Odometry and Mapping: Low-drift, Robust, and Fast
ri.cmu.edu/pub_files/2015/5/ICRA_2015.pdf

LOAM: Lidar Odometry and Mapping in Real-time
ri.cmu.edu/pub_files/2014/7/Ji_LidarMapping_RSS2014_v8.pdf

KITTI benchmark
cvlibs.net/datasets/kitti/eval_odometry.php

2017 Microsoft Indoor Localization Challenge
microsoft.com/en-us/research/event/microsoft-indoor-localization-competition-ipsn-2017/

2016 Microsoft Indoor Localization Challenge
microsoft.com/en-us/research/event/microsoft-indoor-localization-competition-ipsn-2016/#!official-results

---