Hyunjun Kim

Surface damage is an important indicator for evaluating the safety and durability of bridges. As manual visual inspections are often time-consuming and risky, alternative approaches are essential for effective assessments. Computer vision-based methods have shown the potential for improved inspection utilizing a set of images acquired during a bridge inspection. However, because the position and orientation of a camera are different even for the same damage in routine inspections, tracking the progression of damage over time is challenging in practice. This study presents an integrated framework for the long-term monitoring of structural damage based on multi-view images captured from routine inspections of bridges. In terms of qualitative assessment, a time series of images are grouped for individual damages based on a similarity index. Afterward, a group of pixels for structural damage in each image are mapped onto a common coordinate system, tracking the progression of the same damage over time. In the case of quantitative assessment, a scale conversion factor is calculated using the global navigation satellite system data, in which a plane fitting algorithm is employed to calculate the damage area in physical units. The proposed method is validated using an in-service bridge over a period of 120 days, enabling the long-term assessment of crack, spalling, and water leakage with a maximum error of 4.61%.

Lip reading aims to predict speech based on lip movements alone. As it focuses on visual information to model the speech, its performance is inherently sensitive to personal lip appearances and movements. This makes the lip reading models show degraded performance when they are applied to unseen speakers due to the mismatch between training and testing conditions. Speaker adaptation technique aims to reduce this mismatch between train and test speakers, thus guiding a trained model to focus on modeling the speech content without being intervened by the speaker variations. In contrast to the efforts made in audio-based speech recognition for decades, the speaker adaptation methods have not well been studied in lip reading. In this paper, to remedy the performance degradation of lip reading model on unseen speakers, we propose a speaker-adaptive lip reading method, namely user-dependent padding. The user-dependent padding is a speaker-specific input that can participate in the visual feature extraction stage of a pre-trained lip reading model. Therefore, the lip appearances and movements information of different speakers can be considered during the visual feature encoding, adaptively for individual speakers. Moreover, the proposed method does not need 1) any additional layers, 2) to modify the learned weights of the pre-trained model, and 3) the speaker label of train data used during pre-train. It can directly adapt to unseen speakers by learning the user-dependent padding only, in a supervised or unsupervised manner. Finally, to alleviate the speaker information insufficiency in public lip reading databases, we label the speaker of a well-known audio-visual database, LRW, and design an unseen-speaker lip reading scenario named LRW-ID.