Intel Makes Lidar Too

As part of our series on lidar players in Silicon Valley, we were keen to talk to Intel Corporation. Our curiosity about the semiconductor giant’s lidar activities was piqued in December 2019, when the company announced its Intel® RealSense™ LiDAR Camera L515. The L515 seemed to us to bear similarities to the lidar sensor in Apple’s new iPad Pro. We were fortunate to be able to put our questions to Sagi Ben Moshe, corporate vice president and general manager of Intel’s Emerging Growth and Incubation (EGI) group, the mission of which is to identity, incubate and scale the next big business for Intel. Here is what he said.

Editor’s note: A PDF of this article as it appeared in the magazine is available here.

LM: Intel is best known as the world’s leading supplier of the microprocessors at the hearts of our desktops, laptops and servers. Why did it decide to get into lidar?

SBM: Over the last 50+ years of innovation, Intel has explored and developed many cutting-edge technologies. We recognized the importance of vision years ago, which led us to establish Intel RealSense. RealSense has more than ten years of expertise creating different types of depth cameras using various sensing technologies such as stereo and coded light. Our depth cameras are used worldwide across industries like robotics, drones, 3D scanning and more. Our RealSense LiDAR Camera L515 is a great addition to our broad portfolio of computer-vision hardware, as lidar rounds out the portfolio of vision technologies, and it offers our customers a full set of solutions depending on their application.

LM: Intel is a huge, well-structured company. Where does lidar fit into this overall structure? Is Intel RealSense a brand name, a subsidiary, a division or a business unit?

SBM: Intel RealSense is a business unit within EGI. Intel RealSense produces stereo, coded light and lidar depth cameras along with an SDK and software libraries to tie it all together. Each of these cameras comprises a number of advances in computer-vision algorithms, custom ASICs and hardware design.

LM: Please tell us about yourself and your history, before and since Intel.

SBM: As well as the role of corporate vice president and general manager of EGI, I also serve as senior vice president of Sensor Technologies at Mobileye®, an Intel company. Intel’s official description is: “Sagi demonstrated his strong entrepreneurial capabilities when he founded his first company at the age of sixteen. He later went on to establish seven more companies. Sagi successfully sold most of his ventures, with four of them going to Fortune 500 companies. Sagi initially joined Intel part-time in 2004, as a member of the design team for the first mobile version of the Intel Core™ 2 processor. In 2009, Sagi left Intel to establish InVision Biometrics Ltd., a startup dedicated to innovative 3D technology. He rejoined Intel in 2012 with Intel’s acquisition of InVision, where he served as CEO, and became general manager of the RealSense™ group. Sagi served as a staff sergeant in Mamram, the Israel Defense Forces’ central computing systems unit. He holds bachelor’s and master’s degrees in computer science, both earned at the Technion—Israel Institute of Technology. He is a Stanford Graduate School of Business alumnus and has been granted multiple patents for his inventions.”

LM: What is the market for the L515? It’s about the size of a tennis ball and weighs 100 g. Is it for indoors only? The target markets include logistics, robotics, and 3D scanning, but are there others that seem to you to be particularly exciting? What do you think about the price point of $349? Related to the question of markets for the L515, could you please explain your sales strategy, i.e. do you mix direct sales with OEMs, resellers etc.?

SBM: I’ll answer the multiple parts of the question with the following bullets:

• The first product in the new LiDAR Depth family, L515 is for indoor applications. It features unparalleled depth resolution and accuracy that make it perfect for use cases such as health and retail, logistics, robotics and measurement. This new addition to the Intel RealSense family works seamlessly with the same Intel RealSense SDK 2.0 as other RealSense devices, allowing users to develop once and deploy on any current or future Intel RealSense depth device.
• L515 produces a very accurate and high-resolution depth map and there are many market segments that can take advantage of such information. Robot navigation, object detection/recognition, logistics, and object scanning are a few. We are seeing immediate, strong interest, in particular from automated pick and place robots using the camera for object detection and logistics companies using the camera for package dimensioning and inventory counting.
• L515 is the first product in our new LiDAR family of depth cameras. We are working on enhancements for the next generation to expand the family to enable outdoor uses cases.
• Intel RealSense cameras, including the L515, are available to purchase directly from our website (store.intelrealsense.com). This is the easiest way for customers to buy a camera for evaluation. At intelrealsense.com, you can find white papers, code examples, and video tutorials. Customers can also purchase L515 through any distributor that has a relationship with Intel—that’s a large number of distributors worldwide. It won’t be long before OEMs are selling products with the L515 built in.

LM: You’ve mentioned the software that will help developers work with the L515—the Intel RealSense SDK 2.0, which is open source. Would you like to comment further?

SBM: Intel RealSense SDK 2.0 is a mature, open source SDK that customers have been using with the Intel RealSense D400 stereo camera product line. The SDK also supports L515 at launch, which means that users have access to a full-featured SDK from the beginning and any development work done previously with D400 cameras can easily be ported to the L515. The SDK is designed with preset configurations and simple APIs so developers can get started quickly. It provides programmability of the camera for customers who need a high degree of control for their particular application. There are several SDK wrappers available for developers used to working in a certain environment, such as Unity, Matlab, and ROS. An open source viewer tool is also available, allowing anyone to get started with the camera immediately even if they have no software background. L515 has a USB 3 type C connection and supports Windows, Linux and Android operating systems.

LM: The accuracy of the L515 is range-related, but is of the order of 1 cm, which is impressive. The L515 has a range of 9 meters, which is interesting, but is probably too little for many automotive applications and certainly too little for UAV applications: UAV-borne lidar sensors typically have ranges between 100 and 200 m. Do you foresee the range being increased as time passes?

SBM: I think the following covers your questions:

• The L515 is not targeted for automotive applications or UAVs. The underlying technology is capable of longer ranges and working outdoors, beyond what is supported on this first lidar product. The goal was to build a highly accurate and high-resolution depth camera that was small, lightweight, and power-efficient, making lidar technology practical for markets beyond automotive and bringing the advantage of lidar to industries where it was not considered in the past.
• L515 enables exciting new applications with its internal vision processor, motion blur artifact reduction and short photon-to-depth latency. It features an onboard vision processor, which enables a processing rate of 23 million depth points per second, without requiring host depth-processing. This makes it perfect for “compute lite” edge devices. Motion blur artifact reduction is achieved by an exposure time of less than 100 nanoseconds, enabling use cases such as gesture recognition and hand tracking. Additionally, the 4-millisecond photon-to-depth latency allows real time applications such as autonomous indoor navigation.

LM: Your website information on L515 includes the language, “The L515 is a revolutionary solid-state lidar depth camera which uses a proprietary MEMS mirror-scanning technology, enabling better laser power efficiency compared to other time-of-flight technologies. With less than 3.5W power consumption for depth streaming, the L515 is the world’s most power-efficient high-resolution lidar camera.” How does this differ from the technology used by automotive lidar suppliers on which LIDAR Magazine has reported, i.e. what is special about the technologies Intel uses in its lidar sensors?

SBM: L515 is capable of creating a very dense and accurate depth point cloud compared to other lidar solutions focused on the automotive market. It is a very light device with a very small form factor. L515 can scan a person accurately enough for fitness assessments or custom tailoring. The technology that Intel has developed produces a very dense depth map using comparatively low power. Our technology utilizes a MEMS mirror for scanning and a very low input referred noise (IRN) receiver, enabling it to work in a very low SNR, which in turn allows the L515 to be a power-efficient, high-accuracy lidar system at a very high resolution of up to 23 million depth points per second. We believe that this is very disruptive technology that will enable many new markets.

LM: The L515 includes a camera and an IMU. It does not include a GNSS receiver, presumably because it is designed for indoor use. Is there an intention, therefore, to link the position and pose of each L515 data capture with algorithms such as SLAM, so that all the data can be merged into one dataset, for example for the inside of a building?

SBM: We see the IMU used by customers in many ways like scanning applications and SLAM. Having the IMU data makes “localization” much easier for SLAM, autonomous navigation for robotics, AR/VR applications and many other navigation use cases. Also, knowing how the camera moved as frames were captured allows software to orient and stitch the frames together more efficiently. Scanning applications may include scanning the inside of a building for a virtual tour or scanning an historical object for digital preservation.

LM: In addition to the L515, Intel RealSense offers a series of cameras that measure depth stereoscopically, the D400 series (and the earlier D300 series), all with prices below $250. These are offered with AI, including the Intel Neural Compute Stick 2. There is also the T265 Tracking Camera. Although these are not lidar products, they are complementary, and LIDAR Magazine is interested. Could you say a little about this product line, please?

SBM: RealSense products are not tied to any single technology—we focus on how to help our customers solve their problems, so they can bring value to their customers. We think that each depth camera technology has a lot of benefits and it really depends on the application’s use case. We deliver the best solutions suited to meet a range of use cases. The D400 family provides quality depth at incredible value, and works both indoors and outdoors. The four camera options for various use cases can be used in the same field of view without interference. The stereo line complements the lidar line, offering a choice for our customers. (You can find all our offerings at www.intelrealsense.com.)

LM: You’ve worked for Mobileye, an Israeli company that was founded in 1999 and acquired by Intel in 2017. Mobileye is a tier 2 automotive supplier, very active in the advanced driver-assistance systems (ADAS) space. Almost everyone agrees that lidar is an essential component of ADAS and AV systems, so I guess Mobileye has been working on lidar for a while. Could you please say more about this? I’ve read up a little on Mobileye and it seems to do some ADAS systems with imagery only, as well as others using lidar, radar and other sensors, so the two can be complementary. Please tell us about Mobileye and whether there is any relationship to the RealSense business area.

SBM: Mobileye is the global leader in the development of vision technology for ADAS. Mobileye’s system-on-chip (SoC)—the EyeQ® family¹—provides the processing power to support a comprehensive suite of ADAS functions based on a single camera sensor. We have significant experience within EGI of various sensing technologies including lidar and radar. We are always exploring new opportunities, including automotive sensing technologies, which have a lot of technical challenges to be solved and several interesting use cases.

LM: By exploring RealSense and Mobileye today, we hope that we have gained some insight into Intel’s lidar activities. It’s a huge company, however, so are there any other lidar activities that we’ve missed?

SBM: Intel is committed to the development of industry-leading lidar and 3D computer-vision technology. The first applications of lidar technology have been developed within the RealSense team, but the R&D can be applied to applications across Intel as we move into new market segments and uses cases.

LM: Sagi, thank you very much indeed for answering all these questions from LIDAR Magazine. Is there any chance of a brief tour of the lidar part of your factory at some time in the future, or are the lidar sensors not made in Santa Clara?

SBM: Our lidar sensors and cameras are not built in Santa Clara and, given the current covid-19 pandemic, we are limiting access to all Intel facilities for safety purposes. However, once things are safer, we will be happy to show you more details on the assembly process in person.

---