SPShiliang Pu

Shiliang Pu

*This is an algorithmically generated author record based on publicly available information.

Shiliang Pu has a total of 81 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    NeRF-Gaze: A Head-Eye Redirection Parametric Model for Gaze Estimation

    by Pengwei Yin, Jiawu Dai, Jingjing Wang, Di Xie, Shiliang Pu

    Gaze estimation is the fundamental basis for many visual tasks. Yet, the high cost of acquiring gaze datasets with 3D annotations hinders the optimization and application of gaze estimation models. In this work, we propose a novel Head-Eye redirection parametric model based on Neural Radiance Field, which allows dense gaze data generation with view consistency and accurate gaze direction. Moreover, our head-eye redirection parametric model can decouple the face and eyes for separate neural rendering, so it can achieve the purpose of separately controlling the attributes of the face, identity, illumination, and eye gaze direction. Thus diverse 3D-aware gaze datasets could be obtained by manipulating the latent code belonging to different face attributions in an unsupervised manner. Extensive experiments on several benchmarks demonstrate the effectiveness of our method in domain generalization and domain adaptation for gaze estimation tasks.

  • Open Access

    A Strong Baseline for Semi-Supervised Incremental Few-Shot Learning

    by Linlan Zhao, Dashan Guo, Yunlu Xu, Liang Qiao, Zhanzhan Cheng, Shiliang Pu, Yi Niu, Xiangzhong Fang

    Few-shot learning (FSL) aims to learn models that generalize to novel classes with limited training samples. Recent works advance FSL towards a scenario where unlabeled examples are also available and propose semi-supervised FSL methods. Another line of methods also cares about the performance of base classes in addition to the novel ones and thus establishes the incremental FSL scenario. In this paper, we generalize the above two under a more realistic yet complex setting, named by Semi-Supervised Incremental Few-Shot Learning (S2 I-FSL). To tackle the task, we propose a novel paradigm containing two parts: (1) a well-designed meta-training algorithm for mitigating ambiguity between base and novel classes caused by unreliable pseudo labels and (2) a model adaptation mechanism to learn discriminative features for novel classes while preserving base knowledge using few labeled and all the unlabeled data. Extensive experiments on standard FSL, semi-supervised FSL, incremental FSL, and the firstly built S2 I-FSL benchmarks demonstrate the effectiveness of our proposed method.

  • Open Access

    SPIN: Structure-Preserving Inner Offset Network for Scene Text Recognition

    by Chengwei Zhang, Yunlu Xu, Zhanzhan Cheng, Shiliang Pu, Yi Niu, Fei Wu, Futai Zou

    Arbitrary text appearance poses a great challenge in scene text recognition tasks. Existing works mostly handle with the problem in consideration of the shape distortion, including perspective distortions, line curvature or other style variations. Therefore, methods based on spatial transformers are extensively studied. However, chromatic difficulties in complex scenes have not been paid much attention on. In this work, we introduce a new learnable geometric-unrelated module, the Structure-Preserving Inner Offset Network (SPIN), which allows the color manipulation of source data within the network. This differentiable module can be inserted before any recognition architecture to ease the downstream tasks, giving neural networks the ability to actively transform input intensity rather than the existing spatial rectification. It can also serve as a complementary module to known spatial transformations and work in both independent and collaborative ways with them. Extensive experiments show that the use of SPIN results in a significant improvement on multiple text recognition benchmarks compared to the state-of-the-arts.

  • NetRL: Task-aware Network Denoising via Deep Reinforcement Learning

    by Jiarong Xu, Yang Yang, Shiliang Pu, Yao Fu, Jun Feng, Weihao Jiang, Jiangang Lu, Chunping Wang

    Network data in real-world is error-prone, which results in inaccurate results when performing network analysis or modeling such as node classification and link prediction on these flawed networks. In this paper, we target at reconstructing a reliable network from a flawed network, named as network enhancement. Specifically, network enhancement aims to both detect the noisy links which are observed in the network but should not exist in the real world, and predict the missing links that indeed exist in the real world yet being unobserved in the network. Different from existing works that calculate a unified score to measure the above two kinds of links, we propose E-Net, an end-to-end graph neural network model, to leverage the mutual influence of the two tasks to achieve both the goals more effectively. Because on one hand, detecting noisy links can benefit the performance of predicting missing links; and on the other hand, predicting missing links can provide indirect supervision for detecting noisy links when the labels of the noisy links are unavailable. The experimental results on several datasets show that the proposed model obtains significant improvement for predicting missing links and detecting noisy links.


  • Correlation Graph Convolutional Network for Pedestrian Attribute Recognition

    by Haonan Fan, Hai-Miao Hu, Shuailing Liu, Weiqing Lu, Shiliang Pu

    The pedestrian attribute recognition aims at generating the structured description of pedestrian, which plays an important role in surveillance. However, it is difficult to achieve accurate recognition results due to diverse illumination, partial body occlusion and limited resolutions. Therefore, this paper proposes a comprehensive relationship framework for comprehensively describing and utilizing relations among attributes, describing different type of relations in the same dimension, and implementing complex transfers of relations in a GCN manner. This framework is named Correlation Graph Convolutional Network (CGCN). Based on the proposed framework, the feature vectors are built to associate attributes with image features and generate different relation matrices through self-attention among different feature vectors, describing different attribute relations. Then, we conduct multi-layer transfer of attribute relations by means of graph convolution, realizing complex utilization of attribute relations. In addition, the relations among attributes are fully exploited and two types of relations, namely the explicit and implicit relations, are proposed to be integrate into the proposed comprehensive relationship framework. The experimental results on RAP and PETA demonstrate that the recognition performance of the proposed CGCN can obviously outperform the state-of-the-arts, and moreover, the CGCN can achieve a better synergy with different relations.


  • Spectrum Characteristics Preserved Visible and Near-Infrared Image Fusion Algorithm

    by Zhuo Li, Hai-Miao Hu, Wei Zhang, Shiliang Pu, Bo Li

    The visible and near-infrared images fusion aims at utilizing their spectrum characteristics to enhance visibility. However, the current visible and near-infrared fusion algorithms cannot well preserve spectrum characteristics, which results in color distortion and halo artifacts. Therefore, this paper proposes a new visible and near infrared images fusion algorithm by fully considering their different reflection and scattering characteristics. According to image degradation model, the reflection weight model and the transmission weight model are established, respectively. The reflection weight model is established by calculating the difference between the visible (red, green, and blue) spectra and the near-infrared spectrum, while maintaining the correlation of the visible spectra. The proposed reflection weight model can preserve the original reflection characteristic of objects in natural scenes. On the other hand, the transmission weight model is explicitly proposed by calculating the gradient ratio of the visible spectra to the near-infrared spectrum. The proposed transmission weight model intends to make full use of the strong transmission performance of the near-infrared spectrum, which can complement the details loss of the visible spectra caused by light scattering. Moreover, the fused image based on two models is further enhanced according to the reflection characteristics of near-infrared spectrum in case of the non-uniform illumination. The experimental results demonstrate that the proposed algorithm can not only well preserve spectrum characteristics, but also avoid color distortion while maintaining the naturalness, which outperforms the state-of-the-art.


  • Open Access

    Neural Inheritance Relation Guided One-Shot Layer Assignment Search

    by Rang Meng, Weijie Chen, Di Xie, Yuan Zhang, Shiliang Pu

    Layer assignment is seldom picked out as an independent research topic in neural architecture search. In this paper, for the first time, we systematically investigate the impact of different layer assignments to the network performance by building an architecture dataset of layer assignment on CIFAR-100. Through analyzing this dataset, we discover a neural inheritance relation among the networks with different layer assignments, that is, the optimal layer assignments for deeper networks always inherit from those for shallow networks. Inspired by this neural inheritance relation, we propose an efficient one-shot layer assignment search approach via inherited sampling. Specifically, the optimal layer assignment searched in the shallow network can be provided as a strong sampling priori to train and search the deeper ones in supernet, which extremely reduces the network search space. Comprehensive experiments carried out on CIFAR-100 illustrate the efficiency of our proposed method. Our search results are strongly consistent with the optimal ones directly selected from the architecture dataset. To further confirm the generalization of our proposed method, we also conduct experiments on Tiny-ImageNet and ImageNet. Our searched results are remarkably superior to the handcrafted ones under the unchanged computational budgets. The neural inheritance relation discovered in this paper can provide insights to the universal neural architecture search.

  • Learning embeddings of a heterogeneous behavior network for potential behavior prediction

    by Yue-Yang Wang, Wei-Hao Jiang, Shi-Liang Pu, Yue-Ting Zhuang

    Potential behavior prediction involves understanding the latent human behavior of specific groups, and can assist organizations in making strategic decisio


  • NetVision: On-Demand Video Processing in Wireless Networks

    by Zongqing Lu, Kevin Chan, Rahul Urgaonkar, Shiliang Pu, Thomas La Porta

    The vast adoption of mobile devices with cameras has greatly contributed to the proliferation of the creation and distribution of videos. For a variety of purposes, valuable information may be extracted from these videos. While the computational capability of mobile devices has greatly improved recently, video processing is still a demanding task for mobile devices. We design an on-demand video processing system, NetVision, that performs distributed video processing using deep learning across a wireless network of mobile and edge devices to answer queries while minimizing the query response time. However, the problem of minimal query response time for processing videos stored across a network is a strongly NP-hard problem. To deal with this, we design a greedy algorithm with bounded performance. To further deal with the dynamics of the transmission rate between mobile and edge devices, we design an adaptive algorithm. We built NetVision and deployed it on a small testbed. Based on the measurements of the testbed and by extensive simulations, we show that the greedy algorithm is close to the optimum and the adaptive algorithm performs better with more dynamic transmission rates. We then perform experiments on the small testbed to examine the realized system performance in both stationary networks and mobile networks.