Yang Shen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Yang Shen has a total of 35 co-authors. Here are the top co-authors by number of shared publications:
- Xiu-Shen Wei
School of Computer Science and Engineering, Southeast University, Nanjing, Jiangsu, China, School of Intelligence Science and Engineering, Southeast University, Nanjing, Jiangsu, China, and Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Southeast University, Nanjing, Jiangsu, China
6 shared publications - Jian Jin
Nanjing University of Science and Technology
3 shared publications - Jian Yang
Nanjing University of Science and Technology, Nanjing, China
3 shared publications - Zhenyong Fu
Nanjing University of Science and Technology
3 shared publications - Xuhao Sun
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
2 shared publications - Yazhou Yao
Nanjing University of Science and Technology, Nanjing, China
2 shared publications
Latest Publications
- 23 June 2025
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 133 (10) , 6729-6767
An Empirical Study on Training Paradigms for Deep Supervised Hashing
- 23 June 2025
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 133 (10) , 6729-6767
An Empirical Study on Training Paradigms for Deep Supervised Hashing
- 27 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 4670-4683
Prune and Merge: Efficient Token Compression for Vision Transformer With Spatial Information Preserved
Token compression is essential for reducing the computational and memory requirements of transformer models, enabling their deployment in resource-constrained environments. In this work, we propose an efficient and hardware-compatible token compression method called Prune and Merge. Our approach integrates token pruning and merging operations within transformer models to achieve layer-wise token compression. By introducing trainable merge and reconstruct matrices and utilizing shortcut connections, we efficiently merge tokens while preserving important information and enabling the restoration of pruned tokens. Additionally, we introduce a novel gradient-weighted attention scoring mechanism that computes token importance scores during the training phase, eliminating the need for separate computations during inference and enhancing compression efficiency. We also leverage gradient information to capture the global impact of tokens and automatically identify optimal compression structures. Extensive experiments on the ImageNet-1k and ADE20K datasets validate the effectiveness of our approach, achieving significant speed-ups with minimal accuracy degradation compared to state-of-the-art methods. For instance, on DeiT-Small, we achieve a 1.64× speed-up with only a 0.2% drop in accuracy on ImageNet-1k. Moreover, by compressing segmenter models and comparing with existing methods, we demonstrate the superior performance of our approach in terms of efficiency and effectiveness. Code and models have been made available at https://github.com/NUST-Machine-Intelligence-Laboratory/prune_and_merge.
- 27 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 4670-4683
Prune and Merge: Efficient Token Compression for Vision Transformer With Spatial Information Preserved
Token compression is essential for reducing the computational and memory requirements of transformer models, enabling their deployment in resource-constrained environments. In this work, we propose an efficient and hardware-compatible token compression method called Prune and Merge. Our approach integrates token pruning and merging operations within transformer models to achieve layer-wise token compression. By introducing trainable merge and reconstruct matrices and utilizing shortcut connections, we efficiently merge tokens while preserving important information and enabling the restoration of pruned tokens. Additionally, we introduce a novel gradient-weighted attention scoring mechanism that computes token importance scores during the training phase, eliminating the need for separate computations during inference and enhancing compression efficiency. We also leverage gradient information to capture the global impact of tokens and automatically identify optimal compression structures. Extensive experiments on the ImageNet-1k and ADE20K datasets validate the effectiveness of our approach, achieving significant speed-ups with minimal accuracy degradation compared to state-of-the-art methods. For instance, on DeiT-Small, we achieve a 1.64× speed-up with only a 0.2% drop in accuracy on ImageNet-1k. Moreover, by compressing segmenter models and comparing with existing methods, we demonstrate the superior performance of our approach in terms of efficiency and effectiveness. Code and models have been made available at https://github.com/NUST-Machine-Intelligence-Laboratory/prune_and_merge.
- 14 January 2025
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 133 (6) , 3456-3480
UniCanvas: Affordance-Aware Unified Real Image Editing via Customized Text-to-Image Generation
- 14 January 2025
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 133 (6) , 3456-3480
UniCanvas: Affordance-Aware Unified Real Image Editing via Customized Text-to-Image Generation
- Open Access
Customized Generation Reimagined: Fidelity and Editability Harmonized
Customized generation aims to incorporate a novel concept into a pre-trained text-to-image model, enabling new generations of the concept in novel contexts guided by textual prompts. However, customized generation suffers from an inherent trade-off between concept fidelity and editability, i.e., between precisely modeling the concept and faithfully adhering to the prompts. Previous methods reluctantly seek a compromise and struggle to achieve both high concept fidelity and ideal prompt alignment simultaneously. In this paper, we propose a Divide, Conquer, then Integrate (DCI) framework, which performs a surgical adjustment in the early stage of denoising to liberate the fine-tuned model from the fidelity-editability trade-off at inference. The two conflicting components in the trade-off are decoupled and individually conquered by two collaborative branches, which are then selectively integrated to preserve high concept fidelity while achieving faithful prompt adherence. To obtain a better fine-tuned model, we introduce an Image-specific Context Optimization} (ICO) strategy for model customization. ICO replaces manual prompt templates with learnable image-specific contexts, providing an adaptive and precise fine-tuning direction to promote the overall performance. Extensive experiments demonstrate the effectiveness of our method in reconciling the fidelity-editability trade-off.
Open AccessCustomized Generation Reimagined: Fidelity and Editability Harmonized
Customized generation aims to incorporate a novel concept into a pre-trained text-to-image model, enabling new generations of the concept in novel contexts guided by textual prompts. However, customized generation suffers from an inherent trade-off between concept fidelity and editability, i.e., between precisely modeling the concept and faithfully adhering to the prompts. Previous methods reluctantly seek a compromise and struggle to achieve both high concept fidelity and ideal prompt alignment simultaneously. In this paper, we propose a Divide, Conquer, then Integrate (DCI) framework, which performs a surgical adjustment in the early stage of denoising to liberate the fine-tuned model from the fidelity-editability trade-off at inference. The two conflicting components in the trade-off are decoupled and individually conquered by two collaborative branches, which are then selectively integrated to preserve high concept fidelity while achieving faithful prompt adherence. To obtain a better fine-tuned model, we introduce an Image-specific Context Optimization} (ICO) strategy for model customization. ICO replaces manual prompt templates with learnable image-specific contexts, providing an adaptive and precise fine-tuning direction to promote the overall performance. Extensive experiments demonstrate the effectiveness of our method in reconciling the fidelity-editability trade-off.
- 1 November 2024
- Published by Springer Nature
- p. 410-426
Customized Generation Reimagined: Fidelity and Editability Harmonized
- 1 November 2024
- Published by Springer Nature
- p. 410-426
Customized Generation Reimagined: Fidelity and Editability Harmonized
- 30 July 2024
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 133 (1) , 372-397
Equiangular Basis Vectors: A Novel Paradigm for Classification Tasks
- 30 July 2024
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 133 (1) , 372-397
Equiangular Basis Vectors: A Novel Paradigm for Classification Tasks
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 9656-9659
Hawkeye: A PyTorch-based Library for Fine-Grained Image Recognition with Deep Learning
Fine-Grained Image Recognition (FGIR) is a fundamental and challenging task in computer vision and multimedia that plays a crucial role in Intellectual Economy and Industrial Internet applications. However, the absence of a unified open-source software library covering various paradigms in FGIR poses a significant challenge for researchers and practitioners in the field. To address this gap, we present Hawkeye, a PyTorch-based library for FGIR with deep learning. Hawkeye is designed with a modular architecture, emphasizing high-quality code and human-readable configuration, providing a comprehensive solution for FGIR tasks. In Hawkeye, we have implemented 16 state-of-the-art fine-grained methods, covering 6 different paradigms, enabling users to explore various approaches for FGIR. To the best of our knowledge, Hawkeye represents the first open-source PyTorch-based library dedicated to FGIR. It is publicly available at https://github.com/Hawkeye-FineGrained/Hawkeye/, providing researchers and practitioners with a powerful tool to advance their research and development in the field of FGIR.
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 9656-9659
Open AccessHawkeye: A PyTorch-based Library for Fine-Grained Image Recognition with Deep Learning
Fine-Grained Image Recognition (FGIR) is a fundamental and challenging task in computer vision and multimedia that plays a crucial role in Intellectual Economy and Industrial Internet applications. However, the absence of a unified open-source software library covering various paradigms in FGIR poses a significant challenge for researchers and practitioners in the field. To address this gap, we present Hawkeye, a PyTorch-based library for FGIR with deep learning. Hawkeye is designed with a modular architecture, emphasizing high-quality code and human-readable configuration, providing a comprehensive solution for FGIR tasks. In Hawkeye, we have implemented 16 state-of-the-art fine-grained methods, covering 6 different paradigms, enabling users to explore various approaches for FGIR. To the best of our knowledge, Hawkeye represents the first open-source PyTorch-based library dedicated to FGIR. It is publicly available at https://github.com/Hawkeye-FineGrained/Hawkeye/, providing researchers and practitioners with a powerful tool to advance their research and development in the field of FGIR.
- 28 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45 (11) , 13904-13920
Attribute-Aware Deep Hashing With Self-Consistency for Large-Scale Fine-Grained Image Retrieval
Our work focuses on tackling large-scale fine-grained image retrieval as ranking the images depicting the concept of interests (i.e., the same sub-category labels) highest based on the fine-grained details in the query. It is desirable to alleviate the challenges of both fine-grained nature of small inter-class variations with large intra-class variations and explosive growth of fine-grained data for such a practical task. In this paper, we propose attribute-aware hashing networks with self-consistency for generating attribute-aware hash codes to not only make the retrieval process efficient, but also establish explicit correspondences between hash codes and visual attributes. Specifically, based on the captured visual representations by attention, we develop an encoder-decoder structure network of a reconstruction task to unsupervisedly distill high-level attribute-specific vectors from the appearance-specific visual representations without attribute annotations. Our models are also equipped with a feature decorrelation constraint upon these attribute vectors to strengthen their representative abilities. Then, driven by preserving original entities' similarity, the required hash codes can be generated from these attribute-specific vectors and thus become attribute-aware. Furthermore, to combat simplicity bias in deep hashing, we consider the model design from the perspective of the self-consistency principle and propose to further enhance models' self-consistency by equipping an additional image reconstruction path. Comprehensive quantitative experiments under diverse empirical settings on six fine-grained retrieval datasets and two generic retrieval datasets show the superiority of our models over competing methods. Moreover, qualitative results demonstrate that not only the obtained hash codes can strongly correspond to certain kinds of crucial properties of fine-grained objects, but also our self-consistency designs can effectively overcome simplicity bias in fine-grained hashing.
- 28 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 45 (11) , 13904-13920
Attribute-Aware Deep Hashing With Self-Consistency for Large-Scale Fine-Grained Image Retrieval
Our work focuses on tackling large-scale fine-grained image retrieval as ranking the images depicting the concept of interests (i.e., the same sub-category labels) highest based on the fine-grained details in the query. It is desirable to alleviate the challenges of both fine-grained nature of small inter-class variations with large intra-class variations and explosive growth of fine-grained data for such a practical task. In this paper, we propose attribute-aware hashing networks with self-consistency for generating attribute-aware hash codes to not only make the retrieval process efficient, but also establish explicit correspondences between hash codes and visual attributes. Specifically, based on the captured visual representations by attention, we develop an encoder-decoder structure network of a reconstruction task to unsupervisedly distill high-level attribute-specific vectors from the appearance-specific visual representations without attribute annotations. Our models are also equipped with a feature decorrelation constraint upon these attribute vectors to strengthen their representative abilities. Then, driven by preserving original entities' similarity, the required hash codes can be generated from these attribute-specific vectors and thus become attribute-aware. Furthermore, to combat simplicity bias in deep hashing, we consider the model design from the perspective of the self-consistency principle and propose to further enhance models' self-consistency by equipping an additional image reconstruction path. Comprehensive quantitative experiments under diverse empirical settings on six fine-grained retrieval datasets and two generic retrieval datasets show the superiority of our models over competing methods. Moreover, qualitative results demonstrate that not only the obtained hash codes can strongly correspond to certain kinds of crucial properties of fine-grained objects, but also our self-consistency designs can effectively overcome simplicity bias in fine-grained hashing.
- Open Access
- 1 May 2023
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 2508 (1) , 012041
A Recommendation Model for Highway Horizontal Curve Indicators Based on Machine Learning
The horizontal alignment design is the result of the designer’s comprehensive consideration of smooth alignment and satisfying multiple design goals such as safety, environmental protection and economy.Through analyzing a large number of highway horizontal design cases,the statistical laws of intersection spacing, declination angle and the relationship between them are introduced. Based on this and the machine learning algorithms, a horizontal basic curve indicator recommendation model suitable for the single intersection design and the double intersection design process is built. Finally, the reliability of the model is verified and evaluated by using the calculation accuracy, test set error and average difference.
- 1 May 2023
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 2508 (1) , 012041
Open AccessA Recommendation Model for Highway Horizontal Curve Indicators Based on Machine Learning
The horizontal alignment design is the result of the designer’s comprehensive consideration of smooth alignment and satisfying multiple design goals such as safety, environmental protection and economy.Through analyzing a large number of highway horizontal design cases,the statistical laws of intersection spacing, declination angle and the relationship between them are introduced. Based on this and the machine learning algorithms, a horizontal basic curve indicator recommendation model suitable for the single intersection design and the double intersection design process is built. Finally, the reliability of the model is verified and evaluated by using the calculation accuracy, test set error and average difference.
- 23 October 2022
- Published by Springer Nature
- p. 531-548
SEMICON: A Learning-to-Hash Solution for Large-Scale Fine-Grained Image Retrieval
- 23 October 2022
- Published by Springer Nature
- p. 531-548
SEMICON: A Learning-to-Hash Solution for Large-Scale Fine-Grained Image Retrieval