Shuyan Liu has a total of 16 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    An End-to-End Framework Based on Vision-Language Fusion for Remote Sensing Cross-Modal Text-Image Retrieval

    by Liu He, Shuyan Liu, Ran An, Yudong Zhuo, Jian Tao

    Remote sensing cross-modal text-image retrieval (RSCTIR) has recently attracted extensive attention due to its advantages of fast extraction of remote sensing image information and flexible human–computer interaction. Traditional RSCTIR methods mainly focus on improving the performance of uni-modal feature extraction separately, and most rely on pre-trained object detectors to obtain better local feature representation, which not only lack multi-modal interaction information, but also cause the training gap between the pre-trained object detector and the retrieval task. In this paper, we propose an end-to-end RSCTIR framework based on vision-language fusion (EnVLF) consisting of two uni-modal (vision and language) encoders and a muti-modal encoder which can be optimized by multitask training. Specifically, to achieve an end-to-end training process, we introduce a vision transformer module for image local features instead of a pre-trained object detector. By semantic alignment of visual and text features, the vision transformer module achieves the same performance as pre-trained object detectors for image local features. In addition, the trained multi-modal encoder can improve the top-one and top-five ranking performances after retrieval processing. Experiments on common RSICD and RSITMD datasets demonstrate that our EnVLF can obtain state-of-the-art retrieval performance.


  • Genre-Conditioned Long-Term 3D Dance Generation Driven by Music

    by Yuhang Huang, Junjie Zhang, Shuyan Liu, Qian Bao, Dan Zeng, Zhineng Chen, Wu Liu

    Dancing to music is an artistic behavior of humans, however, letting machines generate dances from music is still challenging. Most existing works have been made progress in tackling the problem of motion prediction conditioned by music, yet they rarely consider the importance of the musical genre. In this paper, we focus on generating long-term 3D dance from music with a specific genre. Specifically, we construct a pure transformer-based architecture to correlate motion features and music features. To utilize the genre information, we propose to embed the genre categories into the transformer decoder so that it can guide every frame. Moreover, different from previous inference schemes, we introduce the motion queries to output the dance sequence in parallel that significantly improves the efficiency. Extensive experiments on AIST++[1] dataset show that our model outperforms state-of-the-art methods with a much faster inference speed.


  • Comparison of Variant Principal Component Analysis Using New RNN-based Framework for Stock Prediction

    by Shuyan Liu, Yue Chen

    Due to the high complexity and dimensionality of stock price data, high volatility and uncertainty cause an over-fitting problem in predicting stock price. In this paper, we compare different combinations of variant Principal Component Analysis (PCA) methods and Recurrent Neural Network (RNN) based frameworks that can be applied to extract high-level features from a rich set of initial variables for producing 2-year ahead forecasts of the daily stock price of IBM and Wells Fargo. The objective of our comparative study is to find which combination performs best in forecasting different stock price data. The experimental results show that the best combination for IBM is 2-Directional 2-Dimensional PCA (2d2d-PCA) with Long Short-Term Memory (LSTM), which achieves accuracy of 91.77% based on R-Square (R 2 ) and for Wells Fargo is 2d2d-PCA with Gated Recurrent Unit (GRU), which achieves accuracy of 92.47%. Our comparative study indicates that the best combination of dimension reduction method and deep learning model is unfixed for predicting different stock price data. In addition, it is confirmed that GRU is an efficient model in predicting stock price; and 2d2d-PCA performs as well as PCA with respect to other dimension reduction methods with less error.


  • EffiDiag: an Efficient Framework for Breast Cancer Diagnosis in Multi-Gigapixel Whole Slide Images

    by Shuyan Liu, Junda Ren, Zhineng Chen, Kai Hu, Fen Xiao, Xuanya Li, Xieping Gao

    Breast cancer diagnosis in multi-gigapixel whole slide images (WSIs) is an important task that highly relevant to cancer grading and prognosis. In recent years, many computer-aided diagnosis methods were proposed and achieved promising performance. However, they mostly suffer from heavy computational burden that becomes a significant barrier to clinical practice. Efficient solutions are urgently demanded but still less studied. In this paper, we propose a novel framework named EffiDiag for a fast and lightweight breast cancer diagnosis. To this end, a loss-modified U-net is developed at first to enable a fast suspected cancer Region Of Interest (ROI) localization. Therefore the subsequent patch-based classification, which commonly executes at the finest magnification hundreds of thousands times per WSI for cancer identification, could be carried out on these ROIs only rather than the whole WSI for speedup. Meanwhile, a super-efficient convolutional neural network (CNN) is devised to optimize the classification speed and resource consumption per classification. Experiments on the Camelyonl6 benchmark demonstrate, by integrating the two contributions into a well-established approach, 47x inference acceleration is obtained with limited accuracy drop, yet with much less resource consumption even compared to popular lightweight networks.