Jing Huang

Multimodal remote sensing images (MRSIs) have extensive nonlinear radiation differences, geometric distortions and noise corruption, which bring challenges for registration. Existing feature matching methods usually use gradient or phase congruency (PC) to extract image features. However, gradient and PC are sensitive to strong nonlinear radiation distortions and noises when dealing with MRSIs. To solve this problem, we propose a novel efficient feature detector called adaptive spectrum congruency (ASC). The ASC is a data-driven anti-noise edge detector which adopts an adaptive threshold for noise compensation. Compared with gradient and PC, ASC is more robust to significant radiometric distortions and noise in MRSIs. Based on ASC, we develop a feature matching method for MRSIs registration. First, we propose a novel corner detection function by combining ASC and the Sobel operator to improve the repeatability of feature points. Then, we use the local histogram of ASC to construct the feature descriptor (LHASC) to describe the attributes of the feature points. LHASC is built on the ASC structural map, which can improve the discriminability and robustness of the structural descriptor. We perform extensive experiments on a variety of MRSIs to demonstrate the noise resistance and rotation invariance of our registration method. Compared with the classic and state-of-the-art methods, our method improves the average number of correct matches by at least 1.56 times, the average ratio of corrected number by 15.62%, the average success rate by 58.21% and with an average root mean square error of 1.47 pixels.

The joint use of hyperspectral image (HSI) and Light Detection And Ranging (LiDAR) data has been widely applied for land cover classification because it can comprehensively represent the urban structures and land material properties. However, existing methods fail to combine the different image information effectively, which limits the semantic relevance of different data sources. To solve this problem, in this paper, an Attention-guided Fusion and Classification framework based on Convolutional Neural Network (AFC-CNN) is proposed to classify the land cover based on the joint use of HSI and LiDAR data. In the feature extraction module, AFC-CNN employs the three dimensional convolutional neural network (3D-CNN) combined with a multi-scale structure to extract the spatial-spectral features of HSI, and uses a 2D-CNN to extract the spatial features from LiDAR data. Simultaneously, the spectral attention mechanism is adopted to assign weights to the spectral channels, and the cross attention mechanism is introduced to impart significant spatial weights from LiDAR to HSI, which enhance the interaction between HSI and LiDAR data and leverage the fusion information. Then two feature branches are concatenated and transferred to the feature fusion module for higher-level feature extraction and fusion. In the fusion module, AFC-CNN adopts the depth separable convolution connected through the residual structures to obtain the advanced features, which can help reduce computational complexity and improve the fitting ability of the model. Finally, the fused features are sent into the linear classification module for final classification. Experimental results on three datasets, i.e., Houston, MUUFL and Trento datasets show that the proposed AFC-CNN framework achieves better classification accuracy compared with the state-of-the-art algorithms. The overall accuracy of AFC-CNN on Houston, MUUFL and Trento datasets are 94.2%, 95.3% and 99.5%, respectively.

Among the existing registration methods, most feature descriptors are designed with image intensity, gradient information and phase congruency (PC). However, both intensity and gradient are sensitive to image illumination changes, complex intensity difference, noise, etc . Despite the fact that PC is invariant to image illumination and contrast, it does not perform well when images are corrupted with noise and nonlinear radiation distortions. In this paper, we propose a novel feature called spectrum congruency (SC), which is robust to noises and variations of image illumination and intensity. SC focuses on exploiting the correlation of the multiscale patches based on their local energy and measures the congruency of the energy distribution in a data-driven transform domain. To demonstrate the superiority of SC, we apply it to multimodal image registration. We construct a histogram-based feature descriptor based on SC, termed as HOSC. Then the HOSC descriptor is integrated with two similarity metrics for multimodal remote sensing image registration. Extensive experimental results on both real and noisy image pairs show that the proposed method presents superior registration accuracy and excellent performance in resisting the nonlinear distortion and noise.

Cerebral perfusion computed tomography (CPCT) provides functional information about capillary-level hemody-namic of brain parenchyma, which is an important tool for evaluating acute stroke, vasospasm and other neurovascular disorders. Meanwhile, the side effect of accumulated radiation, relating to genetic or cancerous diseases due to repeated dynamic scans at the same site, has caused great public concerns. One interesting general approach to counter the effect of overdose of radiation is to develop an iterative image restoration algorithm. In this paper, we focus on presenting a tensor-based restoration approach via fully adopting the intrinsic tensor sparsity (ITS) characteristics of CPCT sequential images. For simplicity, the present approach is termed as “CP-ITS”. More specifically, the gains of ITS measure are to adopt the nonlocal self-similarity across space and global correlation along frames, therefore ITS measure can encode both sparsity insights delivered by the most Tucker and CANDECOMP/PAREFAC (CP) low-rank decomposition for a general tensor. Then the ITS measure is introduced into the CPCT images restoration model through characterizing the tensors formed by nonlocal similar patches with in CPCT images. To minimize the associative objective function, we proposed an efficient alternating direction method of multipliers (ADMM) based algorithm. Extensive evaluations on the digital phantom data demonstrate the performance of the present CP-ITS approach for low-dose CPCT imaging.

We proposed a fast and fully automatic mosaic method for high-resolution satellite images. First, the overlapped rectangle is computed according to geographical locations of the reference and mosaic images and feature points on both the reference and mosaic images are extracted by a scale-invariant feature transform (SIFT) algorithm only from the overlapped region. Then, the RANSAC method is used to match feature points of both images. Finally, the two images are fused into a seamlessly panoramic image by the simple linear weighted fusion method or other method. The proposed method is implemented in C++ language based on OpenCV and GDAL, and tested by Worldview-2 multispectral images with a spatial resolution of 2 meters. Results show that the proposed method can detect feature points efficiently and mosaic images automatically.

Smart homes for the aging population have recently started attracting the attention of the research community. One of the problems of interest is this of monitoring the activities of daily living (ADLs) of the elderly, in order to help identify critical problems, aiming to improve their protection and general well-being. In this paper, we report on our initial attempts to recognize such activities, based on input from networks of far-field microphones distributed inside the home. We propose two approaches to the problem: The first models the entire activity, which typically covers long time spans, with a single statistical model, for example a hidden Markov model (HMM), a Gaussian mixture model (GMM), or GMM super-vectors in conjunction with support vector machines (SVMs). The second is a two-step approach: It first performs acoustic event detection (AED) to locate distinctive events, characteristic of the ADLs, and it is subsequently followed by a post-processing stage that employs activity-specific language models (LMs) to classify the output sequences of detected events into ADLs. Experiments are reported on a corpus containing a small number of acted ADLs, collected as part of the Netcarity Integrated Project inside a two-room smart home. Our results show that SVM GMM supervector modeling improves six-class ADL classification accuracy to 76%, compared to 56% achieved by the GMMs, while also outperforming HMMs by 8% absolute. Preliminary results from LM scoring of acoustic event sequences are comparable to those from GMMs on a three-class ADL classification task.