Weifeng Ou

Vein-based biometric technology offers secure identity authentication due to the concealed nature of blood vessels. Despite the promising performance of deep learning-based biometric vein recognition, the scarcity of vein data hinders the discriminative power of deep features, thus affecting overall performance. To tackle this problem, this paper presents a generative self-supervised contrastive learning (GSCL) scheme, designed from a data-centric viewpoint to fully mine the potential prior knowledge from limited vein data for improving feature representations. GSCL first utilizes a style-based generator to model vein image distribution and then generate numerous vein image samples. These generated vein images are then leveraged to pretrain the feature extraction network via self-supervised contrastive learning. Subsequently, the network undergoes further fine-tuning using the original training data in a supervised manner. This systematic combination of generative and discriminative modeling allows the network to comprehensively excavate the semantic prior knowledge inherent in vein data, ultimately improving the quality of feature representations. In addition, we investigate a multi-template enrollment method for improving practical verification accuracy. Extensive experiments conducted on public finger vein and palm vein databases, as well as a newly collected finger vein video database, demonstrate the effectiveness of GSCL in improving representation quality.

Real-time semantic segmentation has received considerable attention due to growing demands in many practical applications, such as autonomous vehicles, robotics, etc. Existing real-time segmentation approaches often utilize feature fusion to improve segmentation accuracy. However, they fail to fully consider the feature information at different resolutions and the receptive fields of the networks are relatively limited, thereby compromising the performance. To tackle this problem, we propose a light Cascaded Selective Resolution Network (CSRNet) to improve the performance of real-time segmentation through multiple context information embedding and enhanced feature aggregation. The proposed network builds a three-stage segmentation system, which integrates feature information from low resolution to high resolution and achieves feature refinement progressively. CSRNet contains two critical modules: the Shorted Pyramid Fusion Module (SPFM) and the Selective Resolution Module (SRM). The SPFM is a computationally efficient module to incorporate the global context information and significantly enlarge the receptive field at each stage. The SRM is designed to fuse multi-resolution feature maps with various receptive fields, which assigns soft channel attentions across the feature maps and helps to remedy the problem caused by multi-scale objects. Comprehensive experiments on two well-known datasets demonstrate that the proposed CSRNet effectively improves the performance for real-time segmentation.

The appearances of children are inherited from their parents, which makes it feasible to predict them. Predicting realistic childrens faces may help settle many social problems, such as age-invariant face recognition, kinship verification, and missing child identification. It can be regarded as an image-to-image translation task. Existing approaches usually assume domain information in the image-to-image translation can be interpreted by style, i.e., the separation of image content and style. However, such separation is improper for the child face prediction, because the facial contours between children and parents are not the same. To address this issue, we propose a new disentangled learning strategy for childrens face prediction. We assume that childrens faces are determined by genetic factors (compact family features, e.g., face contour), external factors (facial attributes irrelevant to prediction, such as moustaches and glasses), and variety factors (individual properties for each child). On this basis, we formulate predictions as a mapping from parents' genetic factors to children's genetic factors, and disentangle them from external and variety factors. In order to obtain accurate genetic factors and perform the mapping, we propose a ChildPredictor framework. It transfers human faces to genetic factors by encoders and back by generators. Then, it learns the relationship between the genetic factors of parents and children through a mapping function. To ensure the generated faces are realistic, we collect a large Family Face Database to train ChildPredictor and evaluate it on the FF-Database validation set. Experimental results demonstrate that ChildPredictor is superior to other well-known image-to-image translation methods in predicting realistic and diverse child faces. Implementation codes can be found at https://github.com/zhaoyuzhi/ChildPredictor.

We propose a hybrid recurrent Video Colorization with Hybrid Generative Adversarial Network (VCGAN), an improved approach to video colorization using end-to-end learning. The VCGAN addresses two prevalent issues in the video colorization domain: Temporal consistency and unification of colorization network and refinement network into a single architecture. To enhance colorization quality and spatiotemporal consistency, the mainstream of generator in VCGAN is assisted by two additional networks, i.e., global feature extractor and placeholder feature extractor, respectively. The global feature extractor encodes the global semantics of grayscale input to enhance colorization quality, whereas the placeholder feature extractor acts as a feedback connection to encode the semantics of the previous colorized frame in order to maintain spatiotemporal consistency. If changing the input for placeholder feature extractor as grayscale input, the hybrid VCGAN also has the potential to perform image colorization. To improve the consistency of far frames, we propose a dense long-term loss that smooths the temporal disparity of every two remote frames. Trained with colorization and temporal losses jointly, VCGAN strikes a good balance between color vividness and video continuity. Experimental results demonstrate that VCGAN produces higher-quality and temporally more consistent colorful videos than existing approaches.

Biometric vein recognition using deep learning has shown promising performance recently. However, due to the lack of vein image training data, the performance is greatly limited. Although the use of traditional data augmentation can alleviate this problem to some extent, it can only expand the intra-class samples and is fundamentally limited to the intra-class space, resulting in limited performance improvement. In this paper, we tackle this deep learning data shortage problem by proposing a GAN-based framework that can generate arbitrary-pattern vein images, which can augment the training data with new vein classes. The proposed generation framework consists of three progressive synthetic steps, namely, generation of random vein patterns in the binary space, refinement of the binary vein patterns, and rendering them into grayscale vein images. Moreover, we use the synthetic dataset to pretrain the feature embedding network via unsupervised contrastive learning, which allows learning data augmentation-invariant and instance-separating representations. After that, we further fine-tune the embedding network on the real training data in a supervised manner. Our results show that highfidelity and diverse vein image samples can be generated to alleviate the problem of data shortage and improve the learning of features representations for biometric vein verification.

Most of the deep quantization methods adopt unsupervised approaches, and the quantization process usually occurs in the Euclidean space on top of the deep feature and its approximate value. When this approach is applied to the retrieval tasks, since the internal product space of the retrieval process is different from the Euclidean space of quantization, minimizing the quantization error (QE) does not necessarily lead to a good performance on the maximum inner product search (MIPS). To solve these problems, we treat Softmax classification as vector quantization (VQ) with angular decision boundaries and propose angular deep supervised VQ (ADSVQ) for image retrieval. Our approach can simultaneously learn the discriminative feature representation and the updatable codebook, both lying on a hypersphere. To reduce the QE between centroids and deep features, two regularization terms are proposed as supervision signals to encourage the intra-class compactness and inter-class balance, respectively. ADSVQ explicitly reformulates the asymmetric distance computation in MIPS to transform the image retrieval process into a two-stage classification process. Moreover, we discuss the extension of multiple-label cases from the perspective of quantization with binary classification. Extensive experiments demonstrate that the proposed ADSVQ has excellent performance on four well-known image data sets when compared with the state-of-the-art hashing methods.

In recent years, the angle-based softmax losses have significantly improved the performance of face recognition whereas these loss functions are all based on cosine logit. A potential weakness is that the nonlinearity of the cosine function may undesirably saturate the angular optimization between the features and the corresponding weight vectors, thereby preventing the network from fully learning to maximize the angular discriminability of features. As a result, the generalization of learned features may be compromised. To tackle this issue, we propose a Linear-Cosine Softmax Loss (LinCos-Softmax) to more effectively learn angle-discriminative facial features. The main characteristic of the loss function we propose is the use of an approximated linear logit. Compared with the conventional cosine logit, it has a stronger linear relationship with the angle on enhancing angular discrimination through Taylor expansion. We also propose an automatic scale parameter selection scheme, which can conveniently provide an appropriate scale for different logits without the need for exhaustive parameter search to improve performance. In addition, we propose a marginenhanced Linear-Cosine Softmax Loss (m-LinCos-Softmax) to further enlarge inter-class distances and reduce intra-class variations. Experimental results on several face recognition benchmarks (LFW, AgeDB- 30, CFP-FP, MegaFace Challenge 1) demonstrate the effectiveness of the proposed method and its superiority to existing angular softmax loss variants.