Bosheng Qin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Bosheng Qin has a total of 26 co-authors. Here are the top co-authors by number of shared publications:
- Yueting Zhuang
Zhejiang University, China
4 shared publications - Siliang Tang
Zhejiang University, Hangzhou, Zhejiang, China
3 shared publications - Dongxiao Li
Zhejiang University
2 shared publications - Juncheng Li
Zhejiang University, Hangzhou, China
2 shared publications - Wenqiao Zhang
Zhejiang University, Hangzhou, China
2 shared publications - Chi Zhang
College of Computer Science and Technology, Zhejiang University, Hangzhou, Zhejiang, China
1 shared publication
Latest Publications
- 13 July 2026
- Published by Springer Nature
- p. 201-211
NutMuse: Inspiring Creative Ideation for Nut Carving with GenAI
- 13 July 2026
- Published by Springer Nature
- p. 201-211
NutMuse: Inspiring Creative Ideation for Nut Carving with GenAI
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 4137-4148
Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 4137-4148
Open AccessTowards Meta-Cognitive Knowledge Editing for Multimodal LLMs
- 4 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 7093-7108
Truncate Diffusion: Efficient Video Editing With Low-Rank Truncate
Diffusion models have demonstrated remarkable capabilities for text-to-video (T2V) editing tasks, relying on fine-tuning for pretrained text-to-image (T2I) diffusion models with only one video-prompt pair. However, conventional fine-tuning approaches require tuning and storing numerous parameters for each video, leading to substantial parameter and memory costs. To mitigate these issues, we propose Truncating Diffusion, an efficient fine-tuning method for video editing that optimizes both parameter and memory usage. Specifically, we propose the Truncating Diffusion module, which is designed with a focus on module architecture and initialization, specifically targeting optimization with a small training set, such as a single video-prompt pair. Theoretical analysis using the Johnson-Lindenstrauss lemma and the Eckart-Young-Mirsky theorem shows that Truncating Diffusion can achieve a minimal Frobenius norm distance to the original attention algorithm with appropriate initialization, which enhances the ease of optimization and improves video editing performance. During fine-tuning, the Truncating Diffusion module integrates seamlessly with the original diffusion model. We freeze the weights of the denoising network within the original pretrained diffusion model, updating only the introduced low-rank alternatives to ensure parameter and memory efficiency. Additionally, we propose Latent Flow Loss and Bidirectional Inter-Frame Attention (BIFA) to improve temporal consistency in synthesized videos. The Latent Flow Loss leverages global temporal information from the input video during training, while BIFA utilizes local temporal information from adjacent frames during inference. These enhancements do not incur additional memory or parameter costs during fine-tuning. Comparisons with state-of-the-art approaches demonstrate that Truncating Diffusion provides superior text alignment, video quality, and inter-frame temporal consistency in video editing. Importantly, Truncating Diffusion requires fine-tuning only 3.2% of the parameters and uses just 62% of the memory compared to the baseline model.
- 4 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 7093-7108
Truncate Diffusion: Efficient Video Editing With Low-Rank Truncate
Diffusion models have demonstrated remarkable capabilities for text-to-video (T2V) editing tasks, relying on fine-tuning for pretrained text-to-image (T2I) diffusion models with only one video-prompt pair. However, conventional fine-tuning approaches require tuning and storing numerous parameters for each video, leading to substantial parameter and memory costs. To mitigate these issues, we propose Truncating Diffusion, an efficient fine-tuning method for video editing that optimizes both parameter and memory usage. Specifically, we propose the Truncating Diffusion module, which is designed with a focus on module architecture and initialization, specifically targeting optimization with a small training set, such as a single video-prompt pair. Theoretical analysis using the Johnson-Lindenstrauss lemma and the Eckart-Young-Mirsky theorem shows that Truncating Diffusion can achieve a minimal Frobenius norm distance to the original attention algorithm with appropriate initialization, which enhances the ease of optimization and improves video editing performance. During fine-tuning, the Truncating Diffusion module integrates seamlessly with the original diffusion model. We freeze the weights of the denoising network within the original pretrained diffusion model, updating only the introduced low-rank alternatives to ensure parameter and memory efficiency. Additionally, we propose Latent Flow Loss and Bidirectional Inter-Frame Attention (BIFA) to improve temporal consistency in synthesized videos. The Latent Flow Loss leverages global temporal information from the input video during training, while BIFA utilizes local temporal information from adjacent frames during inference. These enhancements do not incur additional memory or parameter costs during fine-tuning. Comparisons with state-of-the-art approaches demonstrate that Truncating Diffusion provides superior text alignment, video quality, and inter-frame temporal consistency in video editing. Importantly, Truncating Diffusion requires fine-tuning only 3.2% of the parameters and uses just 62% of the memory compared to the baseline model.
- 15 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (8) , 14493-14507
DBA: Efficient Transformer With Dynamic Bilinear Low-Rank Attention
Many studies have aimed to improve Transformer model efficiency using low-rank-based methods that compress sequence length with predetermined or learned compression matrices. However, these methods fix compression coefficients for tokens in the same position during inference, ignoring sequence-specific variations. They also overlook the impact of hidden state dimensions on efficiency gains. To address these limitations, we propose dynamic bilinear low-rank attention (DBA), an efficient and effective attention mechanism that compresses sequence length using input-sensitive dynamic compression matrices. DBA achieves linear time and space complexity by jointly optimizing sequence length and hidden state dimension while maintaining state-of-the-art performance. Specifically, we demonstrate through experiments and the properties of low-rank matrices that sequence length can be compressed with compression coefficients dynamically determined by the input sequence. In addition, we illustrate that the hidden state dimension can be approximated by extending the Johnson–Lindenstrauss lemma, thereby introducing only a small amount of error. DBA optimizes the attention mechanism through bilinear forms that consider both the sequence length and hidden state dimension. Moreover, the theoretical analysis substantiates that DBA excels at capturing high-order relationships in cross-attention problems. Experimental results across different tasks with varied sequence length conditions demonstrate that DBA achieves state-of-the-art performance compared to several robust baselines. DBA also maintains higher processing speed and lower memory usage, highlighting its efficiency and effectiveness across diverse applications.
- 15 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (8) , 14493-14507
DBA: Efficient Transformer With Dynamic Bilinear Low-Rank Attention
Many studies have aimed to improve Transformer model efficiency using low-rank-based methods that compress sequence length with predetermined or learned compression matrices. However, these methods fix compression coefficients for tokens in the same position during inference, ignoring sequence-specific variations. They also overlook the impact of hidden state dimensions on efficiency gains. To address these limitations, we propose dynamic bilinear low-rank attention (DBA), an efficient and effective attention mechanism that compresses sequence length using input-sensitive dynamic compression matrices. DBA achieves linear time and space complexity by jointly optimizing sequence length and hidden state dimension while maintaining state-of-the-art performance. Specifically, we demonstrate through experiments and the properties of low-rank matrices that sequence length can be compressed with compression coefficients dynamically determined by the input sequence. In addition, we illustrate that the hidden state dimension can be approximated by extending the Johnson–Lindenstrauss lemma, thereby introducing only a small amount of error. DBA optimizes the attention mechanism through bilinear forms that consider both the sequence length and hidden state dimension. Moreover, the theoretical analysis substantiates that DBA excels at capturing high-order relationships in cross-attention problems. Experimental results across different tasks with varied sequence length conditions demonstrate that DBA achieves state-of-the-art performance compared to several robust baselines. DBA also maintains higher processing speed and lower memory usage, highlighting its efficiency and effectiveness across diverse applications.
- 6 May 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 25 (15209210) , 4282-4295
Deep Residual Weight-Sharing Attention Network With Low-Rank Attention for Visual Question Answering
The attention-based networks have become prevailing recently in visual question answering (VQA) due to their high performances. However, the extensive memory consumption of attention-based models poses excessive-high demand for the implementation equipment, raising concerns about their future application scenarios. Therefore, designing an efficient and lightweight VQA model is central to expanding possible application areas. Our work presents a novel lightweight attention-based VQA model, namely residual weight-sharing attention network (RWSAN), consisting of residual weight-sharing attention (RWSA) layers cascaded in depth. Each RWSA layer models the textual representation with self residual weight-sharing attention (SRWSA) and captures question features and question-image interactions with self-guided residual weight-sharing attention (SGRWSA). Inside each RWSA layer, the proposed low-rank attention units perform residual learning with learned connection patterns and shared parameters, and every stacked RWSA layer also uses the same parameters. Extensive ablation experiments with quantitative and qualitative analysis are conducted to illustrate the effectiveness and generality of RWSA. Experiments on VQA-v2, GQA, and CLEVR datasets show that the RWSAN achieves competitive performance with much fewer parameters over the state-of-the-art methods.
- 6 May 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 25 (15209210) , 4282-4295
Deep Residual Weight-Sharing Attention Network With Low-Rank Attention for Visual Question Answering
The attention-based networks have become prevailing recently in visual question answering (VQA) due to their high performances. However, the extensive memory consumption of attention-based models poses excessive-high demand for the implementation equipment, raising concerns about their future application scenarios. Therefore, designing an efficient and lightweight VQA model is central to expanding possible application areas. Our work presents a novel lightweight attention-based VQA model, namely residual weight-sharing attention network (RWSAN), consisting of residual weight-sharing attention (RWSA) layers cascaded in depth. Each RWSA layer models the textual representation with self residual weight-sharing attention (SRWSA) and captures question features and question-image interactions with self-guided residual weight-sharing attention (SGRWSA). Inside each RWSA layer, the proposed low-rank attention units perform residual learning with learned connection patterns and shared parameters, and every stacked RWSA layer also uses the same parameters. Extensive ablation experiments with quantitative and qualitative analysis are conducted to illustrate the effectiveness and generality of RWSA. Experiments on VQA-v2, GQA, and CLEVR datasets show that the RWSAN achieves competitive performance with much fewer parameters over the state-of-the-art methods.
- Open Access
Identifying Facemask-Wearing Condition Using Image Super-Resolution with Classification Network to Prevent COVID-19
The rapid worldwide spread of Coronavirus Disease 2019 (COVID-19) has resulted in a global pandemic. Correct facemask wearing is valuable for infectious disease control, but the effectiveness of facemasks has been diminished, mostly due to improper wearing. However, there have not been any published reports on the automatic identification of facemask-wearing conditions. In this study, we develop a new facemask-wearing condition identification method by combining image super-resolution and classification networks (SRCNet), which quantifies a three-category classification problem based on unconstrained 2D facial images. The proposed algorithm contains four main steps: Image pre-processing, facial detection and cropping, image super-resolution, and facemask-wearing condition identification. Our method was trained and evaluated on the public dataset Medical Masks Dataset containing 3835 images with 671 images of no facemask-wearing, 134 images of incorrect facemask-wearing, and 3030 images of correct facemask-wearing. Finally, the proposed SRCNet achieved 98.70% accuracy and outperformed traditional end-to-end image classification methods using deep learning without image super-resolution by over 1.5% in kappa. Our findings indicate that the proposed SRCNet can achieve high-accuracy identification of facemask-wearing conditions, thus having potential applications in epidemic prevention involving COVID-19.
Open AccessIdentifying Facemask-Wearing Condition Using Image Super-Resolution with Classification Network to Prevent COVID-19
The rapid worldwide spread of Coronavirus Disease 2019 (COVID-19) has resulted in a global pandemic. Correct facemask wearing is valuable for infectious disease control, but the effectiveness of facemasks has been diminished, mostly due to improper wearing. However, there have not been any published reports on the automatic identification of facemask-wearing conditions. In this study, we develop a new facemask-wearing condition identification method by combining image super-resolution and classification networks (SRCNet), which quantifies a three-category classification problem based on unconstrained 2D facial images. The proposed algorithm contains four main steps: Image pre-processing, facial detection and cropping, image super-resolution, and facemask-wearing condition identification. Our method was trained and evaluated on the public dataset Medical Masks Dataset containing 3835 images with 671 images of no facemask-wearing, 134 images of incorrect facemask-wearing, and 3030 images of correct facemask-wearing. Finally, the proposed SRCNet achieved 98.70% accuracy and outperformed traditional end-to-end image classification methods using deep learning without image super-resolution by over 1.5% in kappa. Our findings indicate that the proposed SRCNet can achieve high-accuracy identification of facemask-wearing conditions, thus having potential applications in epidemic prevention involving COVID-19.
- Open Access
- 17 July 2020
- Published by MDPI AG in Diagnostics
- Vol. 10 (7) , 487
Automatic Identification of Down Syndrome Using Facial Images with Deep Convolutional Neural Network
Down syndrome is one of the most common genetic disorders. The distinctive facial features of Down syndrome provide an opportunity for automatic identification. Recent studies showed that facial recognition technologies have the capability to identify genetic disorders. However, there is a paucity of studies on the automatic identification of Down syndrome with facial recognition technologies, especially using deep convolutional neural networks. Here, we developed a Down syndrome identification method utilizing facial images and deep convolutional neural networks, which quantified the binary classification problem of distinguishing subjects with Down syndrome from healthy subjects based on unconstrained two-dimensional images. The network was trained in two main steps: First, we formed a general facial recognition network using a large-scale face identity database (10,562 subjects) and then trained (70%) and tested (30%) a dataset of 148 Down syndrome and 257 healthy images curated through public databases. In the final testing, the deep convolutional neural network achieved 95.87% accuracy, 93.18% recall, and 97.40% specificity in Down syndrome identification. Our findings indicate that the deep convolutional neural network has the potential to support the fast, accurate, and fully automatic identification of Down syndrome and could add considerable value to the future of precision medicine.
- 17 July 2020
- Published by MDPI AG in Diagnostics
- Vol. 10 (7) , 487
Open AccessAutomatic Identification of Down Syndrome Using Facial Images with Deep Convolutional Neural Network
Down syndrome is one of the most common genetic disorders. The distinctive facial features of Down syndrome provide an opportunity for automatic identification. Recent studies showed that facial recognition technologies have the capability to identify genetic disorders. However, there is a paucity of studies on the automatic identification of Down syndrome with facial recognition technologies, especially using deep convolutional neural networks. Here, we developed a Down syndrome identification method utilizing facial images and deep convolutional neural networks, which quantified the binary classification problem of distinguishing subjects with Down syndrome from healthy subjects based on unconstrained two-dimensional images. The network was trained in two main steps: First, we formed a general facial recognition network using a large-scale face identity database (10,562 subjects) and then trained (70%) and tested (30%) a dataset of 148 Down syndrome and 257 healthy images curated through public databases. In the final testing, the deep convolutional neural network achieved 95.87% accuracy, 93.18% recall, and 97.40% specificity in Down syndrome identification. Our findings indicate that the deep convolutional neural network has the potential to support the fast, accurate, and fully automatic identification of Down syndrome and could add considerable value to the future of precision medicine.
- Open Access
- 2 July 2020
- Published by Wolters Kluwer Health in Medicine
- Vol. 99 (27) , e20989
Diagnostic performance of artificial intelligence to detect genetic diseases with facial phenotypes
Background: Many genetic diseases are known to have distinctive facial phenotypes, which are highly informative to provide an opportunity for automated detection. However, the diagnostic performance of artificial intelligence to identify genetic diseases with facial phenotypes requires further investigation. The objectives of this systematic review and meta-analysis are to evaluate the diagnostic accuracy of artificial intelligence to identify the genetic diseases with face phenotypes and then find the best algorithm. Methods: The systematic review will be conducted in accordance with the “Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols” guidelines. The following electronic databases will be searched: PubMed, Web of Science, IEEE, Ovid, Cochrane Library, EMBASE and China National Knowledge Infrastructure. Two reviewers will screen and select the titles and abstracts of the studies retrieved independently during the database searches and perform full-text reviews and extract available data. The main outcome measures include diagnostic accuracy, as defined by accuracy, recall, specificity, and precision. The descriptive forest plot and summary receiver operating characteristic curves will be used to represent the performance of diagnostic tests. Subgroup analysis will be performed for different algorithms aided diagnosis tests. The quality of study characteristics and methodology will be assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 tool. Data will be synthesized by RevMan 5.3 and Meta-disc 1.4 software. Results: The findings of this systematic review and meta-analysis will be disseminated in a relevant peer-reviewed journal and academic presentations. Conclusion: To our knowledge, there have not been any systematic review or meta-analysis relating to diagnosis performance of artificial intelligence in identifying the genetic diseases with face phenotypes. The findings would provide evidence to formulate a comprehensive understanding of applications using artificial intelligence in identifying the genetic diseases with face phenotypes and add considerable value in the future of precision medicine. OSF Registration: DOI 10.17605/OSF.IO/P9KUH.
- 2 July 2020
- Published by Wolters Kluwer Health in Medicine
- Vol. 99 (27) , e20989
Open AccessDiagnostic performance of artificial intelligence to detect genetic diseases with facial phenotypes
Background: Many genetic diseases are known to have distinctive facial phenotypes, which are highly informative to provide an opportunity for automated detection. However, the diagnostic performance of artificial intelligence to identify genetic diseases with facial phenotypes requires further investigation. The objectives of this systematic review and meta-analysis are to evaluate the diagnostic accuracy of artificial intelligence to identify the genetic diseases with face phenotypes and then find the best algorithm. Methods: The systematic review will be conducted in accordance with the “Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols” guidelines. The following electronic databases will be searched: PubMed, Web of Science, IEEE, Ovid, Cochrane Library, EMBASE and China National Knowledge Infrastructure. Two reviewers will screen and select the titles and abstracts of the studies retrieved independently during the database searches and perform full-text reviews and extract available data. The main outcome measures include diagnostic accuracy, as defined by accuracy, recall, specificity, and precision. The descriptive forest plot and summary receiver operating characteristic curves will be used to represent the performance of diagnostic tests. Subgroup analysis will be performed for different algorithms aided diagnosis tests. The quality of study characteristics and methodology will be assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 tool. Data will be synthesized by RevMan 5.3 and Meta-disc 1.4 software. Results: The findings of this systematic review and meta-analysis will be disseminated in a relevant peer-reviewed journal and academic presentations. Conclusion: To our knowledge, there have not been any systematic review or meta-analysis relating to diagnosis performance of artificial intelligence in identifying the genetic diseases with face phenotypes. The findings would provide evidence to formulate a comprehensive understanding of applications using artificial intelligence in identifying the genetic diseases with face phenotypes and add considerable value in the future of precision medicine. OSF Registration: DOI 10.17605/OSF.IO/P9KUH.