Jun Chen

Flow visualization through motion estimation using time-sequenced images plays a significant role in analyzing and understanding complex flow phenomena, and it is widely used in meteorology, oceanography, medicine, astronomy, experimental fluid mechanics, etc. However, it is difficult for current motion estimators to adapt to illumination changes, remove instable perturbation, and capture diverse motion patterns. In this paper, a novel flow visualization tool is developed to address these issues by employing a structure-enhanced motion estimator composed of a data term and a regularization term. Specifically, a statistical correlation descriptor is designed for the data term to improve the accuracy of motion estimation by enhancing both illumination robustness and matching discrimination. Inspired by the strong distinguishability of a structure-texture distribution in a local window, a structure-enhanced regularizer that considers the physical mechanism of fluid diffusion is introduced to capture different motion patterns, enhance prominent flow structures, and remove unnecessary ripples or textures caused by instable perturbation or noise. The experimental results demonstrate that our approach significantly outperforms current motion estimators in handling illumination changes and predicting complex fluid flows, and it also achieves state-of-the-art evaluation results on the public fluid flow datasets. Furthermore, the designed flow visualization tool successfully captures diverse motion patterns in Jupiter’s White Ovals, which is crucial for understanding the physical mechanisms behind their formation and sustenance.

This paper presents two new state-of-the-art Spatial Rain Field Interpolation Convolutional Neural Networks (SRFICNNs), referred to as LD (Learned Deviation) and LI (Learned Interpolation) models, for predicting the point rain rate at finer spatial scales. The main contribution is the successful introduction of the prior-art deep learning technique into high resolution rainfall rate prediction with significant improvement in accuracy. This is very important for the effective implementation of fade mitigation techniques for both terrestrial and satellite networks. The comparison of the models’ performances with ground truth (radar measurements) shows that the proposed models give an excellent mean square error (MSE) and Structural SIMilarity (SSIM) in rainfall fields reconstruction if the network depth falls in the range of 15~25 weight layers. The final model uses 20 layers for high resolution point rain rates prediction. Further study shows that the LD model offers a faster convergence and yields a more accurate rain rate prediction. In particular, this paper compares the rain rate exceedance distribution and Log-Normality property from the model estimates with values calculated from measured data. Results show that the LD model gives a highly accurate estimates of these two indices with corresponding Root Mean Square (RMS) error of 5.1709 × 10 ^-4 and 0.0013, respectively.

Motion estimation for complex fluid flows via their image sequences is a challenging issue in computer vision. It plays a significant role in scientific research and engineering applications related to meteorology, oceanography, and fluid mechanics. In this paper, we introduce a novel convolutional neural network (CNN)-based motion estimator for complex fluid flows using multiscale cost volume. It uses correlation coefficients as the matching costs, which can improve the accuracy of motion estimation by enhancing the discrimination of the feature matching and overcoming the feature distortions caused by the changes of fluid shapes and illuminations. Specifically, it first generates sparse seeds by a feature extraction network. A correlation pyramid is then constructed for all pairs of sparse seeds, and the predicted matches are iteratively updated through a recurrent neural network, which lookups a multi-scale cost volume from a correlation pyramid via a multi-scale search scheme. Then it uses the searched multi-scale cost volume, the current matches, and the context features as the input features to correlate the predicted matches. Since the multi-scale cost volume contains motion information for both large and small displacements, it can recover small-scale motion structures. However, the predicted matches are sparse, so the final flow field is computed by performing a CNN-based interpolation for these sparse matches. The experimental results show that our method significantly outperforms the current motion estimators in capturing different motion patterns in complex fluid flows, especially in recovering some small-scale vortices. It also achieves state-of-the-art evaluation results on the public fluid datasets and successfully captures the storms in Jupiter’s White Ovals from the remote sensing images.

In this paper, we proposed a novel motion model with Helmholtz decomposition for complex fluid flows in a filtering-based optical flow framework, where the optimization of the regularization term is treated as a filtering process, and different motion patterns can be captured by finding appropriate filter kernels based on the designed filters. In this framework, we introduce a novel optical flow method with a joint spatial filter, which is based on the Helmholtz decomposition theorem that assumes a local motion field is composed of a curl field and a divergence field. By adjusting the scale of the weights in the filter kernels and combining a curl filter with a divergence filter in a certain ratio, it can simulate different motion patterns. In addition, if the correlation between the horizontal and vertical components of the optical flow field in the filter kernel is eliminated, it will be transformed into a linear motion model. Based on this linear motion model, we also develop a novel optical flow method with an adaptive guided filter. By finding an adaptive filter kernel driven by both the input image and the guided motion field for the designed filter, it can successfully capture different motion patterns, and yield an edge-preserving smoothing optical flow field. Most importantly, the proposed optical flow model provides a new way to design the regularizers for capturing different motion patterns in complex fluid flow. In particular, the designed optical flow method with an adaptive guided filter significantly outperforms the current state-of-the-art optical flow methods in predicting complex fluid flows.

Efficient optical flow estimation with high accuracy is a challenging problem in computer vision. In this paper, we present a simple but efficient segmentation-based PatchMatch framework to address this issue. Specifically, it firstly generates sparse seeds without losing important motion information by oversegmentation, and then yields sparse matches by adopting a coarse-to-fine PatchMatch with sparse seeds. Such a scheme enhances the robustness of global regularization and yields better matching results compared with the existing NNF techniques while leading to a significant speed-up due to the sparsity of these seeds. Simultaneously, we introduce an extended nonlocal propagation and adaptive random search to address the basic limitation of traditional coarse-to-fine framework in handing motion details that often vanish at coarser levels. Finally, we obtain dense matches at the finest level through an efficient sparse-todense matching according to the cues of oversegmentation. While performing an efficient approximation for oversegmentation, the proposed algorithm runs significantly fast and are robust to large displacements while preserving important motion details. It also achieves good performance on the challenging MPI-Sintel and Kitti flow 2015 datasets.

We present a novel optical flow estimation framework that reinterprets most popular algorithms (e.g., the Horn- Schunck model, PDE-based models, TV-based models, and nonlocal-based models) from an iterative filtering perspective. Different regularizers in the related optical flow estimation algorithms can be achieved by adopting different filtering operations. The key merit of the proposed framework is that a variety of filtering models and corresponding optimization strategies, which are used in image processing, can be directly utilized in the regularization term, leading to a convenient way of designing appropriate optical flow estimation algorithms. Under the proposed filtering framework, we demonstrate easily designing optical flow estimation algorithms with respect to translation consistency, rotation consistency, and divergence consistency constraints by adopting different spatial filters, respectively. We also derive a novel optical flow estimation algorithm with 3D filtering based model for the regularization term, which makes use of nonlocal self-similarity and sparse character of optical flow field. Benefit from the advantages of patch-based nonlocal sparse regularizer, the derived algorithm can remove outliers while preserving sharp flow edges and important motion details. At present, the proposed algorithm achieves state-of-the-art results on the challenging MPI-Sintel dataset and shows good performance on the Kitti flow 2015 and Middlebury datasets.

Fast and accurate optical flow estimation is a challenging problem in computer vision. In this paper, we present a novel model to solve the optical flow problem by combining the strengths of the Split Bregman method with the advantages of an efficient variational framework. It allows us to employ different regularization tensors for the Split Bregman regularizer to preserve motion discontinuities. Simultaneously, a novel nonlocal Split Bregman method with adaptive support weights is also developed for the smoothness regularization to preserve motion details, repel outliers, enhance contrast, and reduce motion blurring and staircase effect. The proposed algorithm shows significant improvements in preserving sharp flow edges and important motion details. Most of all, it requires only a few iterations to achieve fast convergence and runs faster than the classic TV-L1 flow method. Our method significantly outperforms the current state-of-the-art methods on the challenging MPI-Sintel dataset and shows good performance on the Kitti flow 2015 and Middlebury datasets.