A 2.225Mb PDF of this article as it appeared in the magazine complete with images is available by clicking HERE
You can’t cross a modern film set without tripping over any one of a dozen pieces of data capture equipment, including the ubiquitous lidar scanner. I interviewed Ron Bedard of Industrial Pixel, a Canadian-based company with incorporated offices in the USA, about their scanning work in the motion picture and television industries.
What’s your background, Ron, and how did you get into the lidar business?
I was a commercial helicopter pilot for 17 years, as well as an avid photographer. During my aviation career, I became certified as an aircraft accident investigator–I studied at Kirtland Air Force Base in New Mexico. I also got certified as a professional photographer, and following that as a forensic photographer. At my aircraft accident investigation company, we utilised scanning technology to document debris fields.
How did you make the leap then into motion pictures?
The transition wasn’t quite that abrupt. Local businesses started to find out that I had scanners, and we began to get calls like, "Hey, we make automotive parts, and we have this old 1967 piston head, and we want to start machining them. Can you scan this one part and reverse engineer it for us?"
So how does lidar work?
Lidar means light detection and ranging. It works by putting out a pulse, or photon, of light. The light hits whatever it hits–whether it’s an atmospheric phenomenon or a physical surface–and bounces back to the sensor, which then records the amount of time that it’s taken for that photon to travel. Most terrestrialbased lidar systems are predicated on the sensor being in a single location. If you’re moving that sensor, you have to attribute where that sensor is in three-dimensional space so you can compensate the XYZ values of each measurement point. That’s commonly used in airborne lidar systems.
What kind of data does the scanner output?
Every software suite does it a little differently, but they all start with a point cloud. We do offer a modelling service, but primarily what we end up providing our clients is an OBJ–a polygonal mesh created from the point cloud–as well as the raw point cloud file.
How do you manage the data?
Our scanner captures over 900,000 points per second. And a large movie set may require over 100 scans. That generates a massive amount of data–too much for a lot of people to work with. So we provide our clients with the individual point clouds from each of the scans, as well as a merged point cloud that has been resurfaced into a polygonal mesh. So, instead of making the entire model super-high resolution, we create a nice, clean scene. Then, if they want some part at higher resolution, they let us know and we create it from the original raw point cloud. If they have the point cloud themselves, they just highlight a certain area and work from that.
Is lidar affected by the weather?
Rain can create more noise, because anything that affects the quality of the light will affect the quality of the scan data. And wet surfaces have a layer of reflectivity on top. Then there’s the effects of the weather on the technology itself. Our modern system has a laser beam that comes out of the sensor and hits a spinning mirror, bouncing the light off at 90. So if you get a raindrop on that mirror, that can certainly affect where the photons are travelling to.
Is cyberscanning an actor the same as lidar?
No, it’s completely different. You have to think of lidar as a sledgehammer–the point cloud generated is not of a high enough resolution to be able to capture all those subtle details of the human face. So when it comes to scanning people, there are other technologies out there, such as structured white light scanning or photogrammetry, which are better suited to the task.
Have actors become used to the process of being scanned?
For the most part, I think they are. I think there’s still some caution. It’s not that the technology is new–it’s more about the ability to re-create somebody digitally. There are some people who have cautions about that, because they’re never sure how their likeness might be used in the future.
Is there such a thing as a typical day on a movie set?
No. Every day is a new day, with new challenges, new scenes, new sets, new people. If we’re doing lidar, nine times out of ten we’re there when nobody else is there. If we’re trying to create our digital double of the set with people running around, that creates noisier data and possible scan registration issues. So we do a lot of night work, when they’ve finished filming. A lot of times, we’ll actually be in there scanning while they’re breaking the set down! There’s such a rapid turnaround now as far as data collection is concerned. You’ve just got to get in and get out.
If we’re on location, scanning an outdoor scene downtown for example, usually the night-time is best anyway, for a couple of reasons. First, you’re going to get a lot less interference from people and traffic. Second, if there are lots of skyscrapers with glass facades, you can get a lot of noise in the scanning data as the sun is reflecting off the buildings.
Is it hard to keep up with scanning technology as it develops?
Oh, absolutely. We’re dogs chasing our tails. With today’s rapid advancements, if you can get three years out of a technology, maybe four, you’re lucky.
Is there any future technology you’ve got your eye on?
I think photogrammetry is really making a comeback. That really has to do with the resolution of the sensors that are now available. Now that you’re talking high numbers of megapixels, you’re able to get much finer detail than you were in days past. As these high-density sensors come down in price, and get incorporated into things like smartphones, I think we’ll see 3D photography–in combination with a sonar- or a laser-based system to get the scale–really getting market-hard.
What does the future hold for lidar?
I think flash lidar will become much more prevalent. Instead of a single pulse of light, flash lidar sends out thousands of photons at once. They use flash lidar on spacecraft for docking, on fighter jets for aerial refuelling, and you’re starting to see low-cost flash lidar systems being incorporated into bumpers on vehicles for collision avoidance.
What are the benefits of flash lidar for the film business?
When you’re trying to do motion tracking, instead of putting balls on people and using infra-red sensors, you can use flash lidar instead. You can create an environment with flash lidar firing at 24 frames per second, and capture anyone who walks within that environment. That’s something I know we’re going to see a lot more of in the future.
What’s the weirdest thing you’ve ever had to scan?
Everything’s weird. Animals are tricky because each one is different, and you never know how they’re going to react to the light source. Another challenging question we were asked was, "Can you scan a boat that’s floating out in the open sea?" We built a custom rig so that the scanner was constantly moving with the boat, and we hung it out over the edge of the boat and scanned the whole hull.
Lidar providers are among the many unsung heroes of movies. Do you ever crave the limelight?
No. In the end, our job is to provide solutions for our customers. For us, that’s the reward. When they’re happy, we’re happy.
Editor’s Note: A longer version of this article was originally published on the Cinefex blog. All content copyright 2015 Cinefex LLC.
Graham Edwards writes weekly blog articles about cinema visual effects for Cinefex magazine. He is also the author of a number of science fiction, fantasy and crime novels.
A 2.225Mb PDF of this article as it appeared in the magazine complete with images is available by clicking HERE