The experimental method, whose founder is considered to be Galileo Galilei (1564–1642), is the basis of all applied sciences and engineering. Its fundamental principle is that science is based on experience, the starting point to formulate scientific laws and the criterion to verify their validity. Inevitably, the experimental method requires the definition of procedures for performing tests as well as technologies and instruments to acquire and eventually post-process the experimental outcomes. If attention is focused on civil engineering, experimental testing involves different levels: materials, structural components, connections and sub-assemblies, small-scale structures, full-scale structures, and infrastructures.
Loading tests on structural materials are common practice to evaluate their mechanical characteristics and performances, e.g., strength and deformation capacity under monotonic or cyclic conditions. Many of such material tests follow specific protocols detailed by building codes when used for the qualification of construction materials. Laboratory testing of structural components, connections, sub-assemblies, small structures such as downscaled prototypes, or full-scale building splices are, for example, essential to validate or calibrate prediction models and support the development of innovative solutions. Such structural tests are commonly performed up to different levels of damage or even failure; many testing options are possible depending on the structural aspects being investigated, e.g., from quasi-static monotonic and cyclic loading tests to high-energy dynamic tests such as those performed on shake tables to simulate destructive seismic events.
The situation changes when the size of the structure increases and experimental testing is required to study the behavior of large structures and infrastructures. In this case, high-energy loading is generally not feasible (because of technical and economic limits) or simple not acceptable (because of the subsequent damages that would be determined). Static loading tests are generally limited in space, e.g., loading of a partition of a floor level in a large multi-story frame building and loading of a given span of a long multi-span bridge. The information gained in this way is indeed useful, but generally does not provide a big picture of the structural behavior. On the other hand, low-energy dynamic tests allow studying the time-varying response of structures, leading to the identification of their stiffness, mass, and damping properties. The dynamic input in low-energy tests can be either directly assigned (such as, for example, through electromechanical shakers) or be an unmeasured operational/ambient noise (such as, for example, wind- or traffic-induced vibrations). In the former case, reference is made to the methods of experimental modal analysis (EMA), e.g., [1
]; in the latter case, the procedures of operational modal analysis (OMA) are used, e.g., [2
]. In particular, it is OMA that in the past decades has attracted much interest in civil engineering and became widely used thanks to its most appealing feature, i.e., large structures can be analyzed under their normal operational conditions, typically without service interruptions, with simple instrumentations [2
]. Data collected from vibration monitoring permits many structural engineering applications, ranging from model updating and system identification, e.g., [4
]; specific activities such as fine calibration of tuned mass dampers, e.g., [14
]; up to their wide use in condition monitoring and damage detection, e.g., [19
The process of acquiring data from experimental tests is inevitably influenced by the available technologies with their characteristics and limitations. The consolidated approach in data acquisition in civil engineering is based on contact point sensors (could be displacement, strain, velocity, or acceleration sensors) whose measurements are transferred via wired connections to the data acquisition hardware. For example, high-sensitivity accelerometers and high quality shielded cables are commonly adopted for data acquisitions in OMA [2
]. Alternative technologies exist to avoid the use of connection cables (given the difficulties and efforts in cabling large structures) with the use of wireless sensors, e.g., [26
]. Other alternatives provide distributed monitoring possibilities with optical fiber sensors [31
] to overcome the pointwise nature of traditional approaches. However, non-contact monitoring technologies probably attracted most of the recent interest of the engineering community, e.g., [36
]. Various approaches were explored in civil engineering for vibration testing; that is, laser Doppler vibrometry, e.g., [37
]; microwave radar interferometry, e.g., [41
]; infrasound, e.g., [46
]; global positioning system (GPS) sensing, e.g., [48
]; satellite remote sensing, e.g., [52
]; theodolites and total stations, e.g., [58
]; optical methods based on the moiré effect, e.g., [60
]; and optical vision-based methods using digital image correlation (DIC), e.g., [65
]. Indeed, the possibility to eliminate physical installations of sensors is very attractive, especially for structures that might not be easily or safely accessible, yet requiring rapid assessment of their conditions, for example, following extreme events such as strong earthquakes, explosions, or floods.
Among contactless technologies, attention is here focused on vision-based methods, given that a large number of applications involving objects ranging from the tiniest dimension to large constructions showed great potentialities even with low-cost consumer-grade instrumentations [65
]. The idea is simple in its principle: images of the object to be monitored are acquired and subsequently analyzed to extract motion information, thus obtaining displacement time histories eventually used to compute strains, velocities, and accelerations. One video camera is used for detecting in-plane movements; two video cameras in stereo mode enable three-dimensional tracking. If the movement of the entire object to be monitored can be acquired with sufficient spatial resolution within the available combinations of cameras and lenses, then a full-field representation of the mechanical response is possible. This is typically the case of small specimens tested in the laboratory. Otherwise, the selected portions of the object to be monitored are acquired with a number of synchronized cameras. This is the case of field monitoring of large structures and infrastructures. The movement is evaluated following the position, frame after frame, of installed targets or prepared speckle patterns, material texture, building edges and corners, or visible structural details such as bolts in steel structures [65
The early history of image-based measurements belongs to the field of photogrammetry—see, for example, Sutton et al. [65
] for a wide review of the initial developments. The first laboratory applications in experimental structural mechanics to evaluate displacements and strains date back to the mid 1980s [74
]. Since then, significant progress has been made thanks to the advancements in computer vision algorithms, e.g., [65
], that found their way into commercial software, such as, for example, in a toolbox for programming [84
]; in dedicated software, e.g., [85
]; and in complete vision-based monitoring systems derived from earlier research studies at the University of South Carolina [86
] and at the University of Bristol [87
If attention is focused on the use of vision-based monitoring for low-energy/low-amplitude vibrations, as is the case of most operational conditions in civil engineering, many challenges have to be faced, e.g., [67
]. Even if the frame rate of the camera (acquired images per second) is high enough to avoid aliasing, effective algorithms (backed by adequate computational resources) as well as camera/lens performances/quality are crucial to accurately track small structural motions.
Regarding video processing, the term optical flow, e.g., [65
], is used to identify general computer vision techniques associating pixels in a reference image with corresponding pixels in another image of the same scene. Among available algorithms, the most popular in structural engineering applications [71
] appears to be correlation-based template matching and feature point matching.
Template matching searches for an area in a new frame most closely resembling the reference (or template) predefined as a rectangular subset in the initial frame. The advantage of template matching is the minimal user intervention, limited to specifying the template region in the reference frame. The limitations of template matching are related to its sensitivity to changes in lighting and background conditions, in addition to tracking problems with very deformable structures.
Feature point matching is an efficient approach based on key-point detection and matching. Key-points are defined in computer vision as points that are stable, distinctive, and invariant to image transformation, e.g., building corners, peculiar building details, and bolts in steel structures. Instead of the raw image intensities as in template matching, a feature descriptor, i.e., a complex representation based on the shape and appearance of a small window around the key-point, is used for matching. In this way, feature matching is less sensitive to illumination change, shape change, and scale variation. However, feature point matching requires the target regions to have rich textures.
Particularly promising and interesting for monitoring low-energy/low-amplitude vibrations are recent methodologies for motion magnifications [88
], able to reveal subtle changes in the images, in the original Lagrangian formulation (tracking a specific feature in a video in time and space), as well as in the Eulerian formulation where a pixel with a fixed coordinate is selected and its value is monitored in time. In fact, the amplification of information relevant to the motion betters the signal-to-noise ratio in vision-based monitoring, which might be critical in many operational conditions in civil engineering structures and infrastructures.
Recent articles analyzed the state-of-the-art in dynamic monitoring of structures using vision-based methodologies with applications focused on structural health monitoring [68
]. These contributions, published by some of the leading experts in the subject, are a great source of information on a multitude of aspects, such as performance and improvements in motion tracking algorithms, comparisons of the performance of various consumer- and industrial-grade cameras, and the influence of environmental conditions on acquired measurements. The abundancy of information is inevitable, given the large number of research contributions published in the last decade, as detailed in the third paragraph of this review article. All such information could be overwhelming to structural engineers and stakeholders with no background on vision-based monitoring, although possibly familiar with consolidated monitoring methodologies. Accordingly, the objective of this review article is to provide an introductory discussion of the latest state-of-the-art of vision-based dynamic monitoring of structures and infrastructures through an overview of the results achieved in field monitoring of vibrations for full-scale case studies. In this way, engineers and stakeholders interested in the possibilities of contactless monitoring of structures and infrastructure could have an overview of up-to-date achievements of vision-based techniques to support a first evaluation of the feasibility and convenience for future monitoring tasks.
The overview of the vision-based field applications presented in this review article led to the following remarks involving four main aspects: camera installation, hardware, software, and hybrid contact–contactless solutions.
Regarding the camera installation, the possibility to place the video camera in a good vantage point, both stable and allowing views of the structural displacements with few perspective distortions, appears to be the most important aspect in the considered applications. If this is the case, good results can be achieved even with low-cost video cameras and simple video processing algorithms. This condition inevitably sets the inherent limits of video-based monitoring: only points that are clearly visible from the video camera can be monitored; major difficulties are expected in locating good vantage points in urban environments, for example, when monitoring tall buildings in crowded downtown areas.
Regarding the hardware, it is essential to choose the appropriate camera lens so that the obtained field of view is suitable for testing. In fact, the sensitivity is controlled by the scale factor (the ratio between the physical displacement and the pixels in the recorded image); a lower scale factor results in higher resolution of the measure and in lower noise. Accordingly, narrower fields of view (zooming in the lens) provide better resolution of the monitored structure, hence decreasing the scale factor. On the other hand, a wider field of view (zooming out the lens) provides less resolution of the monitored structure and reduces the quality of the displacement measures, even if more monitoring points can be identified and tracked with the same camera. Other ways to reduce the scale factor might be the use of higher camera resolutions, i.e., more pixels for the same displacement. However, the increment in the size of the digital image would be demanding in terms of video footage storage and post-processing; the latter point might compromise the possibility of real-time processing. Nevertheless, the achievable resolution in video-based monitoring is a quantity that, of course, does not make sense if not compared to the magnitude of the structural response. In the examined case studies, there were situations with limited resolution in absolute terms that, however, led to satisfactory results, as the monitored structure had important displacements when excited, e.g., lively footbridges under heavy pedestrian traffic, bridges under train passages, and masonry structures close to subways.
Another aspect involving the hardware discussed in the field applications considered in the presented overview is the image sampling rate. The maximum frame rate of most conventional video cameras is in the range of 30 to 60 frames per second; such speeds are indicated as sufficient for most civil engineering structures. Industrial video cameras are available with much higher speeds; however, such speeds do not find application for field monitoring in civil engineering, mostly owing to low frequency contents of structures and infrastructures, as well as the fact that, the higher the frame rate, the more difficult it is to achieve real-time processing.
Regarding the software, there are many possibilities in image processing given the number of algorithms available in the technical literature. Template matching and feature matching appear to be the most common approaches. However, recent motion magnification algorithms were tested for field applications and provided very interesting results, even with stiff and massive structures.
The final remark is made on the fact that some field applications used vision-based monitoring together with conventional contact sensors. In most cases, such combined use was for comparisons or validation purposes of the vision-based monitoring. However, some studies highlighted significant benefits in combining the results obtained from two such different technologies by means of appropriate data fusion methods. In this way, it is possible to successfully combine the benefits of each technology in a hybrid contact–contactless monitoring system.