A 1.657Mb PDF of this article as it appeared in the magazine complete with images is available by clicking HERE
3D scanning and modeling firm with roots in the surveying business is presenting building information models (BIM) in an entirely new way using augmented reality (AR) technology. This innovative adaptation of AR enables design engineers and maintenance crews equipped with standard tablet computers at the job site to `see’ hidden building features, such as mechanical, electrical and structural assets, displayed onscreen in their as-built locations.
The patent-pending technique, referred to as "Augmented Industrial" by its developers at F3 & Associates in Benicia, Calif., relies primarily on high-definition surveying with 3D laser scanners to capture the infrastructure data for inclusion in the BIM. Assets such as pipes, conduits and foundations are otherwise concealed behind dry wall and under pavement or obscured by a maze of other equipment in the as-built environment.
F3’s new take on visualizing BIM layers has already changed the way some petro-chemical plant operators approach both the design and implementation of upgrades in their facilities. And several hospitals have incorporated the AR technique into their asset management solutions to streamline maintenance and renovation activities.
To view the facility assets that have been scanned and modeled by the F3 team, the user simply activates the tablet camera and points the device at an area of interest–a hospital room or plant unit, for instance. If the camera detects a QR code embedded by F3 somewhere in the real-life scene, the tablet will use the code to retrieve BIM data relevant to that location. The Augmented Industrial application then overlays the model layers on the live view, perfectly aligned to the orientation and scale of the scene, even if the user moves around.
"All the guessing is taken out of [the project] on the job walk," said F3 Principal Sean Finn. "The number-one benefit to the client is safety…because they can see hazards around the project before it begins."
Augmented reality improves overall efficiency too, according to Finn. The advantages of viewing the as-built BIM onscreen in the office prior to designing a plant renovation or hospital expansion are well documented. But Finn has seen design plans changed onsite when engineers and designers walk the project area with the AR application, viewing the big picture showing how the various assets interconnect within the context of the entire facility.
"Showing them a 3D model overlaid on the live scene in the tablet makes a light bulb go on," said Finn. "It keeps them from going too far down a road they don’t want to go down."
Modeling Reality
Finn and two partners, Gene and Fred Feickert, started F3 in 2004 as a land surveying company. Business quickly grew to include providing survey control for 3D scanning projects at large industrial facilities, such as petro-chemical plants. The scanning appealed to F3 as a new revenue stream, and the firm bought a Leica ScanStation C10 high-definition laser scanner. Demand for scanning and 3D modeling services was brisk, especially among industrial plants and medical facilities, and F3 bought a second C10 to handle the work and often leases a third for peak demand.
Learn more about the Leica ScanStation C10 here, or visit www. leica-geosystems.us.
The decision to buy Leica Geosystems scanners was based on the firm’s positive experiences with Leica Geosystems surveying equipment, which F3 used extensively in that portion of its business. The C10 is an all-in-one pulse scanner with a reputation for achieving highaccuracy scans even at long distances. This was a critical capability for modeling large industrial plants where restricted access make long-range scan capture a necessity.
"The primary appeal of the C10 was that it works great in the survey workflow," said Finn. "That’s a unique capability, and it makes for very fast scanning with the dual-axis compensator. We collect our data and get out of the client’s way."
He explained that accurately tying 3D scans into survey coordinate systems has been a crucial selling proposition in Augmented Industrial and other services the firm offers. For many scanners, targets must be established throughout the area to be scanned. But with the C10, F3 technicians simply set up the unit on a surveyed control point, site several other control points from it, and then proceed to scan the rest of the facility. The entire point cloud is tied to the survey control.
"From the point clouds, we can create 3D BIMs accurate to within one-eighth of an inch," said Finn.
For the first year or so in the scanning business, the firm offered deliverables similar to most of its competitors. F3 usually delivered just the point cloud to its clients, but the firm quickly realized there was additional business in also performing the processing and modeling to generate a survey-grade 3D BIM for the client.
After submitting several BIMs in digital and printed paper formats, however, F3 wanted to develop a new method of presenting the models that would give the client a different visualization experience and differentiate the firm from its competitors.
"That’s when we came up with the concept of using augmented reality," said Finn.
Augmenting Reality
The largest customers for the Augmented Industrial application thus far have been petro-chemical plants and hospitals, but F3 has also deployed the solution for other industrial facilities and an amusement park. In all these cases, the most popular use of the AR technology is visualizing concealed as-built assets so that upgrades or renovations can be designed and constructed.
At petro-chemical plants, the exact dimensions and locations of all existing pipes, conduits and valves have to be modeled so that designs of new mechanical, electrical and structural components can accommodate what is already there. In some situations, new conduit must be threaded in the void spaces between existing equipment, while in other cases a proposed pipe has to fit precisely at a tie-in point with an older pipe.
Modeling the above-ground sections of a plant usually involves placing the C10 scanners at multiple points in and around the facility until its exposed assets have been scanned from all possible angles. Much more difficult is capturing details of underground pipelines, supports and foundation structures, which may involve scanning from below in basements and crawl spaces. If no subterranean access is available, F3 works with a pot-holing contractor to carefully dig a hole to accommodate the laser or uses ground penetrating radar (GPR) from above when excavation is impossible.
"When we use GPR to locate assets, we mark their locations on the surface with bright paint," said Finn. "The paint markings are picked up in the scans, so the buried features are tied into the same coordinates as the above-ground ones."
Scanning in hospitals requires a slightly different approach. Because almost all mechanical, electrical and plumbing assets are behind the walls or above ceiling tiles for sanitary reasons, F3 scans newly constructed rooms prior to dry walling when the features are still visible.
Once the scanning is completed, F3 filters the scans and integrates them into a cohesive point cloud in the Leica Cyclone software developed for use with the C10 devices. From there, the firm extracts and models features to the extent requested by the client. Some are more interested in structural assets while others need all the mechanical, electrical and plumbing modeled in their correct location, orientation and scales. Features are color-coded in the BIM according to client specifications.
"We use a variety of software for the modeling, usually based on the details to be extracted and file formats to be delivered," said Finn.
After the 3D BIMs have been generated, F3 delivers digital files to the client and prepares the AR application. The files are either loaded onto an FTP site or stored locally on tablet computers or smart phones. Storing the data on the device is currently preferred because accessing an FTP site requires a Wi-Fi or cellular signal which can be difficult at remote plant locations or inside large buildings.
Next, QR codes are created to serve as triggers for accessing the BIM, or part of it, relating to a specific location at the plant site. The QR code is merely a unique patterns printed on a sign or etched into the concrete at a plant site. In hospital rooms, F3 typically uses a framed picture hanging on the wall as the code. F3 then creates a relationship between the BIM and QR code through traditional surveying methods and coordinate geometry, a patent-pending process.
"We place coordinates in the model for the location of the QR code," said Finn. "That’s how we associate the BIM and the real-world coordinate system."
As long as the QR code is in the scene when the tablet is pointed at it, the AR application will display the BIM features in their correct orientation. For example, the user can walk in a circle around a QR code on the sidewalk to view the buried assets under the pavement from all angles. The AR perspective changes with the device’s location and position.
As-Built and Proposed Realities
While there are many applications for the Augmented Industrial application, it is used most often in one of three ways. The first, already noted, is by the architects and engineers before they begin designing a plant retrofit or upgrade. Walking the site in advance and looking at the as-built layers of concealed assets gives them a better understanding of how all the equipment and components in a complex industrial site connect with each other. This impacts how they approach the design process.
After the plan has been drafted and included as a layer in the BIM, the designers often return to the site to view precisely how the proposed new features will interact with the real-life scene. Often the design is tweaked at that time, and F3 has witnessed one proposed petro-chemical retrofit plan that was scrapped after the designer visited the site with the AR and realized the concept just wasn’t feasible.
The third and most common use is by the work crews that actually carry out the construction or maintenance jobs. At a plant retrofit, for example, the pipe fitters get a much better idea of precisely where they are supposed to tie a new line into an existing one if they can see it in 3D at the actual job site. And hospital maintenance personnel stand much less chance of drilling through a concealed water pipe if they can see behind a wall before they mount a new piece of equipment on it.
"They shine their tablets at the [QR code] painting on the wall, and the wall just disappears," said Finn.
The Future of AR
F3 believes the most significant value they bring to the client is a new way of looking at–and interacting with–the 3D BIM. In addition to developing the Augmented Industrial application, the firm has also created innovative methods of using 3D printers to build plastic models of the BIM, another patent-pending workflow. Regardless of whether the deliverable is a digital file, AR application or plastic 3D model, F3 performs the laser scanning portion of each project in exactly the same way.
"This application of augmented reality couldn’t be done without the laser scanner," said Finn, "because the point cloud captures so much extremely accurate data in a very short period of time."
The future of AR technology is in asset management systems, Finn says. He predicts that soon every plant, hospital of other large facility will have its as-built 3D BIM stored along with other critical infrastructure data on tablets or remotely accessible FTP sites. The BIM will be the first and last files viewed when a facility upgrade is planned or routine maintenance is performed.
"Large facilities are constantly changing and are never finished," said Finn.
Kevin Corbley is a business consultant working in the geospatial profession. He can be reached at www.corbleycommunications. com. For more information about F3, visit www.f3-inc.com. To learn more about laser scanners for augmented industrial applications, visit www.leica-geosystems.us.
A 1.657Mb PDF of this article as it appeared in the magazine complete with images is available by clicking HERE