Kangping Du

Data-driven soft sensors are extensively applied to predicting quality indicators. Recent studies have revealed that deep learning models can provide promising means for efficient predictive modeling. Nevertheless, most of them suffer from the intricate characteristics of spatiotemporal coupling relations, which hammers their widespread application in process industries. To handle those characteristics, this paper presents a spatiotemporal masked pre-training (SMPT) framework for soft sensing, which eases the masked modeling and guides the feature representation learning. Specifically, a graph-based memory unit with dynamic graph construction is devised to capture the connections among various process variables, accounting for the temporal and spatial correlations simultaneously. A multiple masked time series-based pre-training strategy is adopted to recover the randomly generated series to the original series via the weighted aggregation of similar neighbors, which assembles the ruined but complementary spatiotemporal information. In addition, a contrastive learning-based network training approach is designed for the effective high-level feature learning and series-wise similarity learning. Finally, SMPT is fine-tuned to obtain an output-feature mapping for predicting quality indicators. Compared with nine state-of-the-art (SOTA) methods, the proposed SMPT empirically presents superior results on two industrial datasets, yielding relative reductions in mean absolute error (MAE) of 18.0% and 9.0% over the second-best methods on the two datasets, respectively.

As modern industrial processes increase in integration and scale, there exist intricate dynamic time variability and nonlinearity within process data. Deep-learning-based nonlinear dynamic models, such as Transformer, are frequently applied to soft sensor modeling for industrial time sequences due to their powerful ability in learning temporal feature representations. Nevertheless, process data gathered from industrial plants are usually sampled at irregular intervals, posing a challenge for most mainstream dynamic models to handle the resulting temporally changeable relations in process sequence data. Thus, this article proposes a time-aware rotary Transformer (TART) for soft sensor modeling of irregular sampled time series in industrial processes, which adaptively and efficiently model the temporally changeable dynamics among series data. Specifically, a sampling interval embedding layer is devised to simultaneously encode the process variables, positional information, and sampling intervals, facilitating the efficient extraction of temporal dynamic features. Accordingly, a nonincreasing function-based time-aware rotary attention mechanism is proposed to deal with the temporally changeable sampling intervals via assigning proper weights to the corresponding feature representations. Theoretical analysis shows that the proposed time-aware rotary attention mechanism can adaptively assign temporal similarities according to the irregular sampling intervals. The TART-based soft sensor is applied to an actual-run industrial sugar crystallization process to predict the supersaturation and purity of mother liquor. In comparison with recent predictive modeling methods, the proposed TART achieves state-of-the-art performance, demonstrating its feasibility and efficacy for soft sensing of practical industrial processes.