Jianwei Yang

The widely adopted Jacobi-Fourier moment (JFM) is limited by its inability to effectively capture spatial information. Although fractional-order JFM (FOJFM) introduces spatial information through a fractional-order parameter, the control of spatial information remains inadequate. This limitation stems from the insufficient control over zeros distribution associated with the used moment’s radial kernel. To address this issue, we generalize both JFM and FOJFM into a transformed JFM. A transformed function with four parameters is designed, and adjustable JFM (AJFM) is proposed. Two parameters correlate to increasing velocities on the left and right parts of the transformed functions, enabling zeros quantities of radial kernel fall in the left and right parts of the interval. The other two parameters segment the transformed function, adjusting regions where different quantities of zeros fall in. This refined control over the radial kernel’s zero distribution enhances the versatility of feature extraction by the AJFM, governed by the introduced parameters. Experimental results demonstrate that AJFM, with properly chosen parameters, can emphasize specific regions within an image more effectively.

By introducing parameters with local information, several types of orthogonal moments have recently been developed for the extraction of local features in an image. But with the existing orthogonal moments, local features cannot be well-controlled with these parameters. The reason lies in that zeros distribution of these moments’ basis function cannot be well-adjusted by the introduced parameters. To overcome this obstacle, a new framework, transformed orthogonal moment (TOM), is set up. Most existing continuous orthogonal moments such as Zernike moments, fractional-order orthogonal moments (FOOMs), etc. are all special cases of TOM. To control the basis function’s zeros distribution, a novel local constructor is designed, and local orthogonal moment (LOM) is proposed. Zeros distribution of LOM’s basis function can be adjusted with parameters introduced by the designed local constructor. Consequently, locations, where local features extracted from by LOM, are more accurate than those by FOOMs. In comparison with Krawtchouk moments and Hahn moments etc., the range, where local features are extracted from by LOM, is order insensitive. Experimental results demonstrate that LOM can be utilized to extract local features in an image.

Visual recognition is recently learned via either supervised learning on human-annotated image-label data or language-image contrastive learning with webly-crawled image-text pairs. While supervised learning may result in a more discriminative representation, language-image pretraining shows unprecedented zero-shot recognition capability, largely due to the different properties of data sources and learning objectives. In this work, we introduce a new formulation by combining the two data sources into a common image-text-label space. In this space, we propose a new learning paradigm, called Unified Contrastive Learning (UniCL) with a single learning objective to seamlessly prompt the synergy of two data types. Extensive experiments show that our UniCL is an effective way of learning semantically rich yet discriminative representations, universally for image recognition in zero-shot, linear-probe, fully finetuning and transfer learning scenarios. Particularly, it attains gains up to 9.2% and 14.5% in average on zero-shot recognition benchmarks over the language-image contrastive learning and supervised learning methods, respectively. In linear probe setting, it also boosts the performance over the two methods by 7.3% and 3.4%, respectively. Our study also indicates that UniCL stand-alone is a good learner on pure image-label data, rivaling the supervised learning methods across three image classification datasets and two types of vision backbones, ResNet and Swin Transformer. Code is available at https://github.com/microsoft/UniCL.

Due to their simplicity and flexibility, the unsupervised statistical models such as Gaussian mixture model (GMM) are powerful tools to address the brain magnetic resonance (MR) images segmentation problems. However, the GMM is based only on the intensity information, which makes it sensitive to noise. Lots of GMM-based segmentation algorithms have the low segmentation accuracy because of the influence caused by noise and intensity inhomogeneity. To further improve the segmentation accuracy, a robust spatial information-theoretic GMM algorithm is proposed in this paper to simultaneously estimate the intensity inhomogeneity and segment brain MR images. First, a novel spatial factor containing the non-local spatial information is incorporated to reduce the impact of noise. The proposed spatial factor not only takes into account the local neighboring information, but also considers the spatial structure information of pixels. Thus, our algorithm can retain more image details while reducing the influence of noise. Second, the mutual information (MI) maximization method is used to identify and eliminate outliers. Finally, to overcome the impact of intensity inhomogeneity, we use a linear combination of a set of orthogonal polynomials to approximate the bias field. The objective function is integrated with the bias field estimation model to segment the images and estimate the bias field simultaneously. The experimental results on both synthetic and clinical brain MR images show that the proposed algorithm can overcome the influence of noise and intensity inhomogeneity, and achieve more accurate segmentation results.

Fourier-Mellin transform (FMT) has been widely used for the extraction of rotation- and scale-invariant features. However, affine transform is a more reasonable approximation model for real viewpoint change. Due to shearing, the integral along the angular direction in the calculation of FMT cannot be used to extract the inherent features of an image undergoing affine transform. To eliminate the effect of shearing, whitening transform should be conducted on the integral along the radial direction. FMT can hardly be modified by conventional whitening-based methods with low computational cost due to additional processes. In this paper, two factors are constructed and embedded into FMT. Quasi Fourier-Mellin transform (QFMT) is proposed. The embedding of these factors is equivalent to whitening transform and can eliminate the effect of shearing in the affine transform. In particular, QFMT can also be calculated by integrating along the radial direction followed by integrating along the angular direction, as in FMT. Based on QFMT, the quasi Fourier-Mellin descriptor (QFMD) is constructed for the extraction of affine invariant features. Some experiments have also been conducted to test the performance of the proposed method.