A 1.871Mb PDF of this article as it appeared in the magazine complete with images is available by clicking HERE
Civil engineers observe and diagnose the visual changes of civil infrastructures in a manner similar to how medical doctors diagnose patients. As the human body ages, small changes gradually influence each other. Over time, small interacting changes accumulate and a young person becomes older. For example, the skin of a person contains aging pigments hardly noticeable at the beginning, but which become more and more obvious as they combine with other "wear-and-tear" pigments found in the liver, kidney, heart muscle, retina, adrenals, nerve cells, and ganglion cells. Similarly, subtle and correlated changes gradually occur and spread in aging civil infrastructures, such as bridges and water tanks. Small changes in structures can often accumulate into catastrophic collapse through unwanted interactions between these accretions. The responsibility of civil engineers is to recognize those correlated changes and take actions early before small changes grow into tragedies.
Change analysis seems to be a simple idea, but many changes of civil infrastructures are subtle and hardly recognizable by human eyes at the early stage of structural decaying. When changes become visible to human, maintenance is already late and dangerous. Correlations among subtle changes are even more difficult to identify when certain changes are still tiny. Civil engineers, therefore, have employed various technologies to augment their eyes in detecting and correlating changes. 3D LiDAR (Light Detection and Ranging), with its capability of capturing very detailed geometries and visual information of structures, is attracting the interest of inspectors.
For example, Figure 1 visualizes a number of correlated changes accumulated in the life cycle of a water tank in Phoenix. Red points indicate large deviations between the point cloud and the as-built geometry. Blue and green points indicate smaller deviations. Detailed analyses revealed that the subtle enlargement of the upper part of the cylindrical body of the tank has strong correlation with the settlement of the central column, as well as the torsional deformation of the rafters. The connections between these components cause interactions between the changes of rafters, central column, and the cylindrical body.
However, the example shown above indicates that effective use of 3D LiDAR in the practice of structure diagnosis is still an art and not yet a science. The interpretation of the data relies heavily on the experiences of engineers and their knowledge of structural behaviors. Even using the same LiDAR data and similar software tools, knowledge and experiences of engineers can seriously influence the reliability of the structural diagnosis.
The SWARM Lab (Sensing, Workflow, Algorithm, Recognition, and Modeling of the Construction Systems Laboratory), led by Dr. Pingbo Tang at Arizona State University, is trying to automate the LiDAR data analysis for supporting comprehensive and reliable change diagnosis of civil infrastructures. The lab recently received a National Science Foundation (NSF) CAREER award to achieve this goal. Dr. Tang expects that automatic data-driven change analysis will improve the sensitivity of change detection in LiDAR data, reduce errorprone processes of change correlation analysis, and ultimately improve the reliability of structural condition assessment. In addition, more reliable change analysis has the potential of predicting the decaying processes of structures so that maintenance agencies can allocate limited maintenance resources in a more proactive manner.
Preliminary findings of this project reveal that correlations among changes can show the loading paths of structures, and anomalous loading paths can guide engineers in identifying structural defects. For example, the rafters that sustain more loads would transfer more forces from the settling central column to enlarge the upper part of the cylindrical body of the tank. As a result, more deviations from the as-built radius would reveal rafters that are under more pressure and thus need more detailed stability analysis. Figure 2 highlights the loading paths of the structures by identifying rafters that cause large radius growth of the tank. Engineers can use these detected loading paths to update a numerical simulation model of the tank for predicting the most likely modes of structural failure.
This loading path analysis concept guides the SWARM lab researchers in developing algorithms that can achieve automatic identification of loading paths based on 3D point clouds collected at different times. These algorithms are addressing a number of technical challenges that impede automatic change analysis based on LiDAR data. For example, existing change analysis algorithms cannot effectively separate local changes from global changes. Global changes can be rotation and dislocation of objects, while local changes are deformations of individual objects. Local changes, such as bending of rafters, are critical for determining internal forces of components.
Figure 3 shows such an issue: when the rafter has both global displacement and torsional changes, previous algorithms cannot correctly associate data points with the corresponding parts of the rafter in the as-built model. Unreliable data-model association causes the algorithms to compare irrelevant data and model parts, and as a result produce unreliable change analysis results. When global changes occur, such issues make it impossible to resolve the mixture of displacement and torsions for deriving the torsional forces inside the rafter. Another example of the challenges is that engineers need extensive experience to be able to classify the deviation patterns and derive the underlying loading paths. In the water tank case, engineers need to inspect the deviation patterns of structural components and manually classify their changes (e.g., torsion, bending, etc.) for recognizing loading paths.
Therefore, the SWARM lab is developing algorithms to address these challenges in order to achieve reliable and automatic detection of correlated changes and underlying loading paths. The algorithm for reliable global and local change analysis uses spatial relationship among objects to ensure reliable data-model association and correct separation of local changes from global changes. Figure 4 shows a relational graph that represents objects as nodes, and their spatial relationships as edges. The algorithm derives relational graphs from both as-built model and the 3D point cloud, and compare shapes and spatial contexts (relationships with neighboring objects and environments) extracted from LiDAR data and as-built models to associate data points with correct model objects. Even when the rafters have vertical displacements, the algorithm will still be able to associate data points with the corresponding rafters because they form similar shapes and have similar spatial contexts: "a line between the top of the central column and the top of the cylindrical wall." Such method can achieve reliable global and local change detection.
The algorithm for automatic loading path recognition statistically analyzes the changes along connected structural components and identifies connected components that have changes reflecting the kinematic behavior of the structure. Specifically, the algorithm automatically follows the connections to identify changes that are along the same directions (e.g., similar translations and rotations along the way) for discovering loading paths. As the loading paths might change due to defect developments within structures, civil engineers can compare loading paths derived at different times in order to identify defects in structures.
Combined with time series data and high-speed camera imageries, more correlated changes can be visible for understanding structures. Specifically, the SWARM Lab is exploring algorithms that correlate the time series of forces collected at multiple locations of the structures, and the vibration modes captured in the videos collected by high-speed cameras. Internal forces can verify the loading paths detected in LIDAR data. Correlated vibrations of building components can also reveal the loading paths and defective components, because the stiffness and vibration modes of structures having different loading paths will vary significantly.
The SWARM lab researchers are currently collecting data across the world in order to understand the fundamental science of visual and spatial changes in civil infrastructures in the real world. The SWARM Lab is looking for your participation and would love to integrate our efforts with yours in collaborative discoveries and engineering renovation.
Pingbo Tang, Ph.D., is an assistant professor in the School of Sustainable Engineering and the Built Environment at Arizona State University. Dr. Tang’s research explores the remote sensing (e.g., LiDAR) and information modeling technology (e.g., Building Information Modeling) to support spatial analyses needed for effective management of construction sites, constructed facilities, and civil infrastructure systems.
Vamsi Sai Kalasapudi, is a graduate research assistant in the School of Sustainable Engineering and the Built Environment at Arizona State University. Vamsi’s research focuses on utilizing LiDAR technology for risk monitoring of civil infrastructure (bridges, water tanks etc.) and using spatiotemporal correlated deformation patterns for predicting structural collapse.
A 1.871Mb PDF of this article as it appeared in the magazine complete with images is available by clicking HERE