AV Lidar Needs Software Too

DeepRoute.ai Ltd., which is based in Shenzen, China, with facilities in Beijing and Fremont, California, takes us further into an area that has become key for this magazine – the hardware and software necessary for lidar-equipped autonomous vehicles (AVs) to operate. We were offered a remote Q&A with a DeepRoute executive to provide a perspective on the company and its cutting-edge product, DeepRoute-Sense, the company’s level 4 full-stack self-driving system, which was first introduced in October 2019 and has subsequently been further refined.

DeepRoute offers unique software and hardware solutions for self-driving vehicles. The company works with top automotive OEMs, tier 1 suppliers, and mobility and logistic companies to achieve the commercialization of self-driving vehicles. The level 4 full-stack self-driving system was a CES Innovation Awards 2020 honoree for its unique combination of software and sensing solutions, offering a new driving experience to consumers.

Editor’s note: A PDF of this article as it appeared in the magazine is available HERE.

The system is a combination of hardware and software, which leads LIDAR Magazine more towards the software side than our earlier pieces on automotive lidar. We were pleased to be able to conduct the following Q&A with DeepRoute vice president, Nianqiu Liu (NL; first name pronounced “Nee-an-chi-oo”).

LM: LIDAR Magazine has worked on articles with automotive lidar suppliers such as Cepton Technologies, Intel, Quanergy Systems and Velodyne LiDAR. The magazine originates in the geospatial world, which has benefited enormously because lidar sensors developed from the deep R&D pockets of the automotive world have been successful and economical when integrated on to UAVs or vehicle-based mobile mapping systems. We don’t know so much, however, about DeepRoute, though we understand it was founded as recently as 2019. Please tell us something about the company: when it was founded; its leadership team; its locations and the reason for them; and its funding. We do understand that privately held companies such as DeepRoute have to be discreet about financial data, but even if you can say little about that, perhaps you can tell us how many employees are a part of the DeepRoute team?

NL: Founded in February 2019, DeepRoute has experienced significant growth over the past year and a half and we have over 200 employees as of October 2020 across our three offices. The company has research and development centers in Shenzhen and Beijing. In addition to our R&D centers, we are testing self-driving vehicles in California. DeepRoute raised $50 million in our latest funding round in October 2019 to support production and operations . The financing round was led by Fosun RZ Capital, the Beijing-based venture capital arm of the Chinese conglomerate Fosun International¹ [Guangchang Guo DeepRoute.jpg]. Our proven technology and business model has garnered strong investor interest and has created a healthy pipeline of funds.

LM: DeepRoute is headquartered in Shenzhen, China, but has operations in Beijing and Fremont, California. How do you divide up activities between these centers?

NL: DeepRoute’s global team works cross-functionally tackling projects in their location, driving innovation in each market. We have dedicated teams of engineers, developers and technicians in China and the U.S. office focuses on testing and business development.

LM: Does DeepRoute supply products, services or both?

NL: Both products and services. DeepRoute is a one-stop shop for automotive OEMs, tier 1 suppliers and mobility and logistic companies. With the products and services we provide, our customers can start testing self-driving technology on their vehicles right away. DeepRoute has developed several products, including:

• DeepRoute-Sense: a slim vehicle roof box equipped with eight vehicle-cameras, three lidars, a global navigation satellite system (GNSS), and other sensors, along with their corresponding telecommunication and data synchronization controllers.
• DeepRoute-Tite is the brain of the self-driving system. It processes the data collected by DeepRoute-Sense and transforms the data into driving instructions for vehicles.DeepRoute-Tite is extremely efficient and consumes significantly less power, only about 1/9 of the traditional solutions, mainly because we’ve developed a powerful duo of the inference engine and deep learning model, which minimizes the amount of computation required to output usable data.
• DeepRoute-Syntric: a data-synchronization device that can accurately detect surrounding objects in real-time to deliver a safe, smooth driving experience. DeepRoute-Syntric can also monitor the status of hardware, including sensors, and command the vehicle to stop when sensors don’t work properly, ensuring the safety our passengers and others on the road.
• DeepRoute-Vision: a vehicle-camera with a high dynamic range, allowing optimal detection performance while also avoiding overexposure—even under strong sunlight—using automatic image adjustment and fine LED flicker-resistance.

Our level 4 full-stack self-driving technology includes all of the self-driving technology needed for AV vehicles. Our customers have the option to implement the full-stack solution or individual modules, which include:

• Multiple sensor options
• High density (HD) mapping and localization
• Premium perception
• Planning and control models for a smooth drive at speeds of 50 mph
• Simulation system that can be tested virtually, which is especially valuable for corner cases
• Cloud technology to improve the efficiency of software development
• 5G remote control module

LM: Please tell us in detail about the level 4 full-stack self-driving system. We are very interested indeed, of course, in the sensors that are incorporated into the hardware, especially the lidar ones, but the whole system—hardware plus software—is of great interest to our readers. Our readership is quite knowledgeable on the use of GNSS and IMU together with optical sensors as well as lidar, so information on DeepRoute’s approach to this is especially fascinating. From your website, for example, it appears that the lidar sensors come from Hesai Technology.

NL: We utilize lidar technology from several trusted suppliers, including Hesai Technology, Velodyne Lidar and RoboSense LiDAR. DeepRoute-Sense uses a 64-line lidar, while the lidar on the sides is 16-line. We use eight cameras to collect surrounding color data. For instance, the two cameras on the front of the roof box are used for traffic-light detection. Apart from the sensors mentioned above, there is a GNSS and an IMU. Since we developed our own proprietary ADS synchronization controller, DeepRoute-Syntric, it can be modified to support a wide range of sensor technologies. This allows the vehicle to synchronize information from all different sensors, enabling the perception algorithm to process sensor data with the same standard. The perception module allows the vehicle’s planning and control modules to detect surrounding objects and predict their trajectory. Based on this information, the planning control can manipulate the driving operation and send a command to the vehicle to change its behavior. All of the calculations from the perception, localization planning and control systems are running on the computer platform located in the trunk.

LM: One of the components of your solution is a HD map. Could you please tell us more about this, please? What are its characteristics and who supplies the map data?

NL: The HD map and localization solution offers the precise detail required for driving. These maps have extremely high precision, to the centimeter level, and the self-driving system is like an automated navigation map for human drivers. It also provides detailed information about nearby lanes, traffic lights and speed limits. Our mapping ability can decrease the effect of weak GNSS signals near high buildings in large cities—ensuring that people can navigate safely in urban environments—and we filter out the dynamic information on the road to generate a comprehensive HD map.

LM: Another component is artificial intelligence/machine learning/deep learning. These areas have become critically important throughout the geospatial and automotive worlds. What is their role in your software? In this respect, what is your relationship with NVIDIA?

NL: Deep learning has played a key role in self-driving software development. The software detects the objects surrounding the vehicle and provides information for planning and control modules. Any incorrect detection would cause critical traffic incidents; this is why using multiple sensors is beneficial. The perception algorithm fuses the data before inference, which makes the detection result more reliable. A DeepRoute paper² focused on perception algorithms was accepted by CVPR 2020, the premier annual computer vision event, which is a nice recognition of our advanced technologies. Additionally, DeepRoute’s HVNet technology ranked highly on the KITTI Vision Benchmark Suite; this benchmarking suite is one way to put self-driving technologies to the test and better understand how different solutions fare when identifying different types of objects. DeepRoute is also a member of NVIDIA’s Inception Program, and NVIDIA’s Jetson AGX Xavier is used in our current computing platform.

LM: What is special in the DeepRoute offering that places it ahead of its competitors?

NL: DeepRoute provides safe and reliable solutions to our customers. Our perception algorithm has been ranked highly on the KITTI and Semantic KITTI tests and holds the prestigious recognition of being a CES Innovation Awards honoree. Our highly integrated solutions make it easier for companies to develop and roll out self-driving vehicles and implementations, while also allowing customers to customize solutions for their specific needs.

LM: Your business model is centered on the idea that you are a “tier 1 supplier to OEMs and mobility and logistic companies”. What does this mean? How do you market your products? Do you do direct sales or do you have a distribution network? LIDAR Magazine has learned, through interviews with other leading companies, that relationships with customer organizations are very important, because they can help proselytize and sell your products. Is that true about DeepRoute?

NL: Our business model focuses on the robo-taxis industry, one of the most promising verticals within the autonomous vehicle market. We offer a full-stack level 4 self-driving solution to automotive OEMs, tier 1 companies, and mobility and logistic companies.

We have partnered³ with Dongfeng Motors Corporation Technology Center to co-develop a type of robo-taxi to help people safely navigate around cities. We have also partnered with other large mobility companies in China this year and will be testing several robo-taxis together to prepare for their deployment.

This past May, we partnered with a container terminals company of Cosco Shipping in China to equip its container trucks with DeepRoute’s self-driving solutions to automate operations in the Xiamen Ocean Gate Container Terminal. The self-driving truck helps to streamline the terminal’s loading and unloading process. We value every relationship with our partners and carefully consider the specific requirements for different self-driving use cases. By carefully listening to our customers’ needs for their desired implementations, we’re able to accelerate the deployment process and set our customers up for success as they test out their vehicle fleets⁴.

LM: Do you perceive your market as global, or is it mainly China and U.S.?

NL: DeepRoute is a global company and we have developed business partnerships in Australia, China and the U.S., in addition to other regions.

LM: Last October you were successful in receiving formal permission from the California Department of Motor Vehicles to test autonomous vehicles in the state. Do you feel this shows strong trust in your company and its solution?

NL: We were pleased to see that the California DMV granted us permission to conduct tests in the state, demonstrating their trust in our solutions. As Silicon Valley has become a hotbed for automotive innovation, we see that region and California at large as a major market for our technologies.

LM: As you enjoy success and adoption of your product grows, you must face the challenge of scaling up production. Does your solution scale well? What are your thoughts about mass production?

NL: We are now preparing for mass production and the standardized evaluation of our solutions. Thanks to our leadership team’s vast experience in this industry, we specifically designed our hardware for the manufacturing requirements of mass production. Our software was also designed for deployment at scale.

LM: What are DeepRoute’s plans for the remainder of 2020 and for the further future?

NL: We will continue to develop and refine our solutions and expand our business partnerships. We are working towards commercializing self-driving vehicles with our innovative technology that offers a strong competitive advantage in the industry.

LM: Nianqiu Liu, thank you very much for your time and your detailed answers to our questions. We wish you well with your exciting product and hope to speak to you again as things develop.

---