Autonomous Vehicles Operational Thanks to Lidar

When Managing Editor Stewart Walker first visited Cepton Technologies to investigate their efforts to build on their automotive lidar experience with a sensor designed for UAVs1, Cepton recommended that he visit its customer, May Mobility, which was successfully using its sensors on autonomous vehicles for transporting office workers in Detroit, Michigan. He visited May Mobility’s headquarters in Ann Arbor, Michigan and its operations center in Detroit in July 2018. Here is what he discovered.

Ann Arbor headquarters
The purpose of my visit was to interview May Mobility’s CTO, Dr. Steve Vozar. When I arrived at their Ann Arbor office, all the principals were in Detroit, submerged in preparation for a visit by investors—a state of affairs not so different from Silicon Valley! I hit gold, therefore, when the interview was postponed till three days later in Detroit and instead I was hosted in Ann Arbor by Head of Product Zafar Razzacki, who gave me a tour of the production area, accompanied by insightful commentary. Like many May Mobility employees, Zafar is a graduate of the University of Michigan (U-M). He came to May Mobility by way of Google and General Motors. He said that the company is expanding rapidly—the software development “bullpen” area where we started the tour had seen a doubling of its population in a few weeks. He agreed with me that automotive lidar will influence the development of airborne owing to the enormously larger market size. He thinks we will see a convergence of these technologies. They will inform each other, resulting in a convergence of different applications, and learn from each other. The software developers and robotics engineers in the bullpen work in the areas of autonomy, perception and control software. They include U-M alumni and experienced personnel from the industry, e.g. Bosch and Argo. May Mobility’s vehicles provide a service to transport employees of Quicken Loans and Bedrock in downtown Detroit. So far, Bedrock is happy with May Mobility’s performance and excited about the future. The company shares its name, of course, with the home of the Flintstones. The state of Michigan licenses May Mobility to drive autonomously and the city of Detroit has given its permission to mount infrastructure on light posts along the route. Zafar emphasized the complementary purposes of the units in their sensor suite, to give a fuller picture of the world.

As we moved through the employees’ break area, I noticed a touch of the idiosyncracy one often finds in start-ups—a four-seater electric vehicle, in use as a conference room for small meetings. We proceeded into the space where vehicle engineering happens, packed with machining equipment—May Mobility has the capability to create prototype molds for some of the vehicle parts. These are sent to manufacturing partners, e.g. plastics suppliers. I saw the custom-made housing for side-mounted sensors and a cowl for the sensors in the front housing, which includes two Cepton HR80Ws. This shop is where they do fabrication work, electrical assembly, and prototyping.

Is there a human driver? A fleet attendant sits behind a custom-built “T-bar”, which he can grab if necessary, but for the most part the vehicles are autonomous and learn as they go. The attendant conforms with safety requirements and makes passengers more comfortable, building trust, i.e. users take a comfortable step from a human-driven bus or taxi. The vehicle is a six-seater, so there are five seats for passengers. It’s pleasant to the eye, inside and out. Zafar confirmed, “People stop them on the road and say it’s cute.” We went to the rear of the vehicle. The battery packs are massive, to give a four-hour service model. The vehicle has surprisingly heavy construction, appearing to be rigid and stable. It’s not a big industrial, diesel bus, yet it’s a proper full-sized vehicle, not a glorified golf-cart or science experiment. The spaciousness surprises people.

Before leaving, I asked Zafar about May Mobility’s ambitions regarding small vehicles for use in warehouses. They are, “Actively exploring what our vehicle portfolio should look like”. They have lots of flexibility for different application types and anticipate demand for larger form-factors. He was sure that there is immense potential for more business and making all aspects of the experience better—sensors, software, vehicle.

Detroit operations center
On arrival at the Detroit office, I was welcomed by co-founder and CEO, Edwin Olson, who has three degrees from MIT and is a part-time faculty member at U-M, where his main responsibility is to run the APRIL Robotics Laboratory (Autonomy, Perception, Robotics, Interfaces, and Learning; May Mobility was named after the month that follows April). Co-founder and COO Alisyn Malek was also there, but my interview was with co-founder and CTO Steve Vozar. Edwin was on Steve’s PhD committee at U-M. Indeed, Steve and Alisyn were undergraduates at U-M at the same time but didn’t know each other. The Detroit office is an operations center, a sort of “mission control” with staff looking at the route on a big wall-screen with detailed mapping, moving icons etc. There are four smaller screens providing additional information.

Steve agreed with Zafar that automotive lidar will benefit the airborne side, eventually bringing prices down, but there will be competition for sensors, so it may be hard for manufacturers to prioritize features for airborne use—smaller buyers may get lost in the fray.

How and when did the company begin? Ed, Alisyn and Steve, with background in “big auto”, such as Ford, General Motors and Toyota, and academic life, were looking at the 10-15-20-year roadmap for AVs. They mused on whether there was a better way: was there anything useful that could be done in the meantime before self-owned, fully autonomous passenger cars were ready for prime time? To reduce risk to acceptable levels, they decided to “slice and dice today’s technology to serve certain markets,” such as “community-scale fleet transportation.” These markets embraced requirements such as lower speeds and complex environments, but sensors were available that could see far enough; furthermore, stopping distances were shorter at lower speeds. They felt that freeway operation of AVs was hampered by sensor performance but designing for speeds of 20-25 mph was feasible, so they could “get customers in car, turn sensors on, be ready for ‘aha moments.’” Top speed doesn’t matter on short routes. The maximum speed on the Detroit route is 25 mph, with most operations at 15-18 mph, which feels normal in a dense urban setting. Steve emphasized that May Mobility aspires not to fundamental research but to take applications to the next level as available technologies improve.

The documents forming May Mobility were signed in January 2017, funding was sourced in late spring and the new company moved into its Ann Arbor offices in May. We talked about Cepton’s new offices and Steve had been there on their first day while Cepton was moving in. May Mobility has expanded rapidly and now has almost 60 employees, more than 40 full-time and the remainder, contractors. The Detroit office officially opened with a media/VIP “ride along” on 26 June 2018 and went operational the next day, though they had been preparing for months. They don’t do development in Detroit—it’s purely a site operations center. Staff can see the infrastructure of the environment they run in, the status of roadside sensing units (RSUs), locations of vehicles and so on.

Is May Mobility a spinoff from U-M? Many staff, including core and original employees, came from U-M. The company licensed some technology from U-M in exchange for equity. U-M has been very supportive: May Mobility has an excellent relationship with the university and uses its Mcity Test Facility, built jointly with Michigan Department of Transportation, the first purpose-built proving ground for testing connected and automated vehicles and technologies in simulated urban and suburban driving environments. The local ecosystem is very advantageous. There are other start-ups, e.g Duo Security is the first unicorn, and strong support organizations such as Ann Arbor Spark, Michigan Economic Development Corporation and Planet M. He added that U-M is a great talent pipeline—May Mobility is still hiring college graduates.

How did May Mobility land Bedrock as its first customer? Business development is part of Alisyn’s brief. After an introduction by one of May Mobility’s investors, Detroit Venture Partners, Alisyn contacted Bedrock’s VP Parking and Mobility. Bedrock is the real estate management arm of Dan Gilbert’s Rock Ventures empire, with hundreds of properties including parking garages. It offers an amenity, connecting people working for its tenants with transportation to parking garages. May Mobility’s contract is for moving people, mainly employees of Quicken Loans, a mortgage lender, from Detroit’s Bricktown parking garage to Cadillac Square, a clockwise circuit over a mile-long. This process illustrates the value of investors helping out with connections, not just funding.

I asked Steve about the Society of Automotive Engineers’ five levels of autonomous vehicles. May Mobility’s solution is classified as level 4, which means no human interaction, though it is permitted in exceptional circumstances, but May Mobility is full autonomy in pre-mapped, geofenced areas.

The vehicle
The vehicle platform is a Polaris GEM e6 electric six-seater. May Mobility buys stock GEMs and performs a deep-dive integration, including the configuration of the seating and doors, sensor suite, compute stack, vehicle control unit, drive-by-wire system, custom sensor cowls, custom user interface and custom glass roof, slightly tinted to protect against intense sunlight, which creates a pleasant user experience, especially in Detroit with its beautiful large buildings. I sat in a May Shuttle, as the final product is called, and was attracted by the beautiful 49-inch curved monitor serving as a dashboard. I observed the custom T-bar for fleet attendants to supervise the vehicle. The middle row of seats is flipped around, so that they are rear-facing, like a London taxi. The standard e6 has three doors on each side, of which two are replaced by one long one, making it very easy to get in and out. This configuration is based on feedback from the pilot in Detroit.

In the trunk there is a bay of huge lithium-ion batteries, for which there are charging stations in the operations center—each vehicle comes in for recharge at the end of its shift and May Mobility manages this as part of the service. Five vehicles are deployed for each shift, typically three for the morning and evening peaks, two at other times. Thus May Mobility is not just a software company, but has a sense of design, deploys a team designated to the platform, and can use sensors fast and in an integrated way. “Passengers must trust to get in with family, grandma too.”

I asked about one of May Mobility’s suppliers, Magna International, which is a massive automotive supplier with 172,000 employees, yet not known to me—“sneaky big” in Steve’s words. Magna is May Mobility’s up-fitter or manufacturer. May Mobility designs and does the engineering and prototyping of the vehicle in Ann Arbor, then Magna produces it at scale. This is where the molds that I had seen come into play.

How did the relationship with Cepton begin? May Mobility has known Cepton’s senior director, business development, Wei Wei, for a long time. Steve’s team saw demos of the HR80W and knew the technology was special. The two companies share similar values in terms of getting product into customers’ hands fast. May Mobility was impressed with Cepton’s quick reactions to feedback and the result was a strong working relationship. Steve was blunt: Cepton provided “the best customer service I’ve ever encountered. They’re doing a great job.” They received their first unit from Cepton in the summer of 2017 on affordable terms. The HR80W point cloud immediately captured the engineers’ imaginations, since its point density is much higher than those from lidars with rotating mirrors or prisms—Steve described it as very similar to cross-hatching with vertical lines, so it complements the beam pattern of the spinning lidar, as well as offering satisfactory range, accuracy and precision. No beams are wasted. The two HR80Ws have 60° degree FOV and are mounted at 45° to each other, so they are performing the long-range sensing in the entire direction of travel, including stop signs, cross traffic etc. May Mobility is excited about Cepton’s new Vista sensor and the companies are working together to integrate it in the vehicle. The mounting and electronics are similar to those required by the HR80W, but the Vista offers higher performance and is half the size of the HR80W.

Indeed, there is a gallimaufry of sensors on the vehicle, giving redundancy round the full 360°: in addition to the Cepton sensors at the front, there is a Velodyne VLP-16 Puck Hi-Res on the roof, and multiple imaging, flash lidar and radar sensors on all four sides. May Mobility prefers not to reveal the total number of sensors on the vehicle, but it must be 25 or more. The measurements are transformed not into point clouds, but information covering an area in front of the sensors, to know whether there is something there, rather like a multi-beam rangefinder. Part of May Mobility’s sensor strategy embraces different modalities of sensing, with orthogonal strengths and weaknesses. Steve explained that the Puck on the roof is part of a typical AV configuration and helps with localization as well as object detection.

We turned to the other sensors, starting with the GNSS/IMU. May Mobility deploys a fiber optic gyro but does not use integrated GNSS/IMU, preferring to separate out all components. The gyro is sourced from KVH Industries. There is a GNSS receiver, but May Mobility does not rely on it for localization: “lidar is our heavy hitter for localization. We don’t rely on GPS for localization, but instead we rely on the sensors on the vehicle … we use lidar, radar, cameras.”

The RSUs are special to May Mobility and are feasible for deployments at a community scale. May Mobility uses RSUs as additional sensors—redundancy and sensor fusion are two of their core competencies. Also, RSUs are a different way of sensing the surroundings, because they are not on the vehicle itself—they provide “eyes on the road” when the vehicle isn’t there.

The hardware is impressive, but making it all work is even more so. May Mobility’s approach to sensor fusion and autonomy is key: the “secret sauce” is its MayStack software, which is extensively reported in the academic literature2. As part of the MayStack, a key algorithm for planning vehicle behavior is called Multi-Policy Decision-Making (MPDM). Developed at the APRIL lab and licensed through the U-M Technology Transfer program, MPDM is the real differentiator for building May Mobility’s intelligent system. It gives a 360-degree view of objects on and around the road, along routes where the team has created detailed maps of lane edges, traffic signals, road signage and speed limits, and taken into account pedestrian, cyclist and auto traffic. By using a robust suite of on-vehicle and environmental sensors and powerful software, the vehicle is able to traverse complex urban scenarios on its own, without human intervention. The current approach to building autonomous systems relies on training with known scenarios, including edge cases which are difficult to define and collect. With MPDM, May Mobility’s systems are able to model and predict future behaviors of agents in any scenario because it understands the situation instead of having to recognize it and recall a specific coded solution.

I was curious because the product that May Mobility provides to Bedrock is a shuttle service, so have they had to write more software for calling the shuttle, optimizing timing to minimize dwell time and so on? Steve described the current offering as a “fixed route circulator” and emphasized that May Mobility is committed to adding features as customers demand. In order to go to market and start the revenue stream, May Mobility could not implement everything, but began with a really solid vehicle and didn’t make assumptions about all customer requirements in advance. The team developed software “from the ground up”, but also had to design a shuttle service. They had to spin up the Detroit operations center, simultaneously adding a team of staff and technicians—this is part of the product. They like to hire people in the local communities where they operate, making the company “sticky”, with a physical location for interested people to come to talk. I wondered, therefore, whether May Mobility’s solution, which relies heavily on using local companies and setting up a control center, will scale. Steve admitted that any time they go into a new location, they require a physical presence. Vehicles have to be charged and able to offload data, but that’s a way of getting a foothold in the community.

I gave Steve the example of San Diego airport, where large buses are driven along a special road just inside the airfield perimeter, to and from the car rental facility, while roads surrounding Lindbergh Field are cluttered with minibuses serving parking lots and hotels. Zafar had said that if May Mobility can build business models to cope with those flows, it can reduce passenger dwell time waiting for a vehicle. He thought that this could be tackled, but May Mobility has larger aspirations, both shuttle-type applications and a more robust service model to replace all vehicles in a particular urban setting, point-to-point on demand like Uber. The route in Detroit serves a real need, replacing a 30-person diesel shuttle bus with two or three AVs, giving the same throughput with lower latency, i.e. better service. I pointed out, in the airport context, that the vehicle has little or no space for luggage (the trunk is full of batteries) and Steve agreed that this is a challenge to come. I suggested twin vehicles, the second one carrying luggage for the passengers in the first, so Steve gracefully said he would consider this!

May Mobility’s current focus though is smooth running in Detroit and several more customers are currently being engaged. Cities are a compelling target market, but there are great advantages to working on a private campus. They want to cover a wide spectrum of different markets. What are May Mobility’s plans for the future? The first target market is enterprise and municipal customers in more cities across the US. The company has strong sales and customer success teams engaged on the quest to add more customers.

I asked whether there are any other sensors May Mobility would like, for example, for weather. Steve was confident that the current sensor suite can meet Detroit’s challenging weather, but he and the team stay on the look-out for new sensors with novel physics or fresh applications. They are always evaluating new types of sensor and like to try things out.

We walked to the parking garage of the operations center and saw “where the vehicles sleep at night.” The bays had special electrical wiring for charging and fiber optic cable for data download, so that’s another part of the investment. The vehicle downloads everything acquired since the last download, from the solid-state drive on board to the company’s own servers. The database is then updated. It’s how the system learns what the vehicles are seeing and enables new ideas to be validated. All the mapping is their own, albeit in a small area. They map only what they need—it’s part of their pragmatic approach. The ideal site for May Mobility is less than ten square miles with routes of one to three miles within a clearly defined, geofenced area.

The fleet attendant builds customer trust and gives peace of mind—Steve drew the analogy of early elevators. He firmly believes May Mobility will get there, just like elevators!

1 Walker, A.S., 2018. Cepton Technologies: the Silicon Valley approach to lidar sensor development, LIDAR Magazine, 8(2): 34-44, March/April.

Editor’s Note (September 2020): We followed up with Cepton mid-2020 for an update. The complete article is here.