In the realm of filmmaking and television, 3D laser scanning is increasingly utilized to bring physically impossible worlds and captivating stories to life. Visual effects (VFX) specialists use a technique called Matchmove to generate effects, seamlessly blending live-action footage and realistic computer-generated imagery (CGI). Laser scanning enhances this process, creating evermore immersive worlds and visuals for audiences worldwide.
The Leica RTC360 3D laser scanner gathers high-quality survey data on set.
One pioneering company deploying the power of laser scanning for VFX is South Korea’s 75mm Studio Co., Ltd1. (75mm). Expert in Matchmove and on-set data captures for the film, television, and gaming industries, 75mm uses 3D laser scanning to conduct digital surveys of sets. These serve as a foundation for applying visual effects to live-action sequences. Using the Leica RTC3602 3D laser scanner, 75mm can swiftly and efficiently gather high-quality survey data, even in the most challenging of circumstances – such as on the set of the 2021 Netflix international hit series, Squid Game.
However, 75mm’s incorporation of 3D laser scanning solutions goes beyond its success with Matchmove. Through lidar technology, 75mm has optimized and expanded its use of point-cloud data, leading to new service offerings for video game development, historical and cultural site documentation, digital twin creation and more.
Converging reality and virtuality: 3D laser scanning takes VFX to new heights
Since deploying 3D laser scanning technology in 2019, 75mm’s digital background assets and set surveys have been used for numerous productions. From the Netflix series Squid Game and Kingdom, to the Korean tvN drama, Jirisan, 3D scanning has left an indelible mark on the entertainment landscape. It has also contributed to commercials such as the Samsung Galaxy Z Flip3 music video, where the application of lidar-acquired meshes and textures allowed 75mm to place footage of the K-pop phenomenon BTS into a digital model of a technicolor gymnasium.
75mm interpolating a 3D scan of historical surroundings with live action footage to seamlessly create a convincing battle scene in Netflix series Kingdom.
“There are various ways to utilize lidar scans in VFX,” explains Cholho Shin, CEO of 75mm, “for example, to fill up an empty auditorium or stadium with digital characters or to create a scene with a dangerous car chase in a tunnel. We consider the safety of the actors by filming the scene on set and adding the tunnel scan data after.”
From early pre-production through to post-processing, 75mm leverages 3D laser scanning across the entire film production pipeline. During pre-production, the firm captures and delivers survey data of sets to enhance pre-visual directing and scene preparation. On set, it provides survey data to production and set teams, along with computer graphics supervision.
“We use survey data with high-resolution imagery to add background assets to virtual sets,” describes Shin. “Additionally, scan data from set locations serves as a great help for live filming.”
Through the use of Matchmove in post-production, 75mm delivers a base layout for computer graphics. “We process Matchmove work by utilizing high quality on-set data of the scene and attaching it to the 3D tracked scene,” explains Shin.
75mm employs captured survey data from sets to precisely match the geometry of live-action frames and digital environments. This enables VFX specialists to merge live-action footage filmed with 3D cameras into virtual environments to build computer-generated imagery into scenes, while ensuring action continuity and deepening realism.
Seamless integration and enhanced output with lidar scanning solutions
At the heart of 75mm’s Matchmove work lies a powerful duo of laser scanner and software, the Leica RTC360 and Leica Cyclone Register 3603. The RTC360 is equipped with lidar, a visual inertial system (VIS) and high-dynamic range (HDR) imaging, so 75mm can scan two million points per second and capture 360-degree HDR images to generate optimal point-cloud results, ready for registration. This allows it to successfully align and georeference point clouds, speeding up post-processing for a much more efficient field-to-finish workflow.
“We normally don’t have enough time on real sets for acquisitions, and we have to consider the condition of the filming location,” explains Shin. “VIS processing creates 3D point clouds generated from HDR images and this feature allows the scanning process at filming locations to be done quickly with great precision and simple usability.”
Lidar in hit series
In two recent Netflix series releases, 75mm used set survey scan data to build the foundations for visually stunning scenes.
In the Golden Globe nominated survival drama Squid Game, 75mm performed several 3D laser scans of sets for high-precision Matchmove work. One set featured a maze-like cascade of multicolour stairways with complex patterns and textures. The high-accuracy data required to capture the set effectively was possible only with lidar scanning. The number of points captured from the intricate set was massive, and 75mm optimized the vast mesh so the film team could use the survey data easily.
75mm’s work with 3D scanned set data across production stages enables the creation of a convincing car chase scene through a tunnel.
For a scene of life-or-death tug-of-war, they also helped capture set data by scanning a large set with raised platforms. After the actors filmed the scene against a blue screen background with each “team” standing on opposing platforms, 75mm applied its expert Matchmove work and visual effects to complete the perception that the game’s losers fell from great heights to their demise.
75mm also scanned sets for Netflix’s Kingdom, a period horror thriller set in 16th century Korea. They captured locations in the Mungyeong Saejae Provincial Park in central South Korea, a scenic mountain pass area with well-preserved historical buildings and an open set for filming. Later, live-action footage was merged with the set survey data to create a seamlessly integrated battle scene.
Laser scanning for the future of filming
3D laser scanning at historical sites like this and other set locations has not only provided 75mm with survey data for its Matchmove work but also helped it to form plans and focus research for where reality-capture technology will take its business in the future.
A historical building in South Korea reconstructed from a 3D scan for use in the Netflix series Kingdom.
“Currently, many VFX companies in Korea are planning additional business content through collaboration with our technology and solutions. We are also making efforts to distribute our data to the Korean Federation of Film Archives and the Korean Film Council,” says Shin.
75mm’s plans for 3D laser scanning extend far beyond the VFX industry itself. “We are researching how we can use the data in different ways,” says Shin. “For example, lidar data can also be used to document historical locations and cultural heritage sites for digital exhibitions, and to create digital assets, like digital twins, for future industries and the metaverse. We are also focusing on optimising the data size and texture while maintaining quality for game engines and augmented reality applications.”
A 3D digital replication of an empty sports stadium used to create a version full of spectators.
Lidar promises to be an integral part not only of the future of VFX, but across a vast array of industries for decades to come. With solutions like those used by 75mm, VFX specialists can continually expand the imaginative bounds of the worlds and stories they create to reach new blockbusting heights.
Megan Hansen is a copy writer and SEO specialist at Leica Geosystems, part of Hexagon. Megan writes about the innovation of 3D scanning solutions, sensors and software and the stories of their application across the world.