Zixin Yin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Zixin Yin has a total of 15 co-authors. Here are the top co-authors by number of shared publications:
- Jiakai Wang
Zhongguancun Laboratory, Beijing, China
2 shared publications - Xianglong Liu
Beihang University, Beijing, China
2 shared publications - Aishan Liu
Beihang University, Beijing, China
1 shared publication - Gang Li
University of the Chinese Academy of Sciences & Institute of Software, Chinese Academy of Sciences, Beijing, Beijing, China
1 shared publication - Hanqing Zhao
University of Science and Technology of China, Hefei, China
1 shared publication - Haotong Qin
ETH Zurich
1 shared publication
Latest Publications
- Open Access
- 22 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 863-871
Dual Intention Escape: Penetrating and Toxic Jailbreak Attack against Large Language Models
- 22 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 863-871
Open AccessDual Intention Escape: Penetrating and Toxic Jailbreak Attack against Large Language Models
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 11013-11021
DreamBooth++: Boosting Subject-Driven Generation via Region-Level References Packing
DreamBooth has demonstrated significant potential in subject-driven text-to-image generation, especially in scenarios requiring precise preservation of a subject's appearance. However, it still suffers from inefficiency and requires extensive iterative training to customize concepts using a small set of reference images. To address these issues, we introduce DreamBooth++, a region-level training strategy designed to significantly improve the efficiency and effectiveness of learning specific subjects. In particular, our approach employs a region-level data re-formulation technique that packs a set of reference images into a single sample, significantly reducing computational costs. Moreover, we adapt convolution and self-attention layers to ensure their processings are restricted within individual regions. Thus their operational scope (i.e., receptive field) can be preserved within a single subject, avoiding generating multiple sub-images within a single image. Last but not least, we design a text-guided prior regularization between our model and the pretrained one to preserve the original semantic generation ability. Comprehensive experiments demonstrate that our training strategy not only accelerates the subject-learning process but also significantly boosts fidelity to both subject and prompts in subject-driven generation.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 11013-11021
Open AccessDreamBooth++: Boosting Subject-Driven Generation via Region-Level References Packing
DreamBooth has demonstrated significant potential in subject-driven text-to-image generation, especially in scenarios requiring precise preservation of a subject's appearance. However, it still suffers from inefficiency and requires extensive iterative training to customize concepts using a small set of reference images. To address these issues, we introduce DreamBooth++, a region-level training strategy designed to significantly improve the efficiency and effectiveness of learning specific subjects. In particular, our approach employs a region-level data re-formulation technique that packs a set of reference images into a single sample, significantly reducing computational costs. Moreover, we adapt convolution and self-attention layers to ensure their processings are restricted within individual regions. Thus their operational scope (i.e., receptive field) can be preserved within a single subject, avoiding generating multiple sub-images within a single image. Last but not least, we design a text-guided prior regularization between our model and the pretrained one to preserve the original semantic generation ability. Comprehensive experiments demonstrate that our training strategy not only accelerates the subject-learning process but also significantly boosts fidelity to both subject and prompts in subject-driven generation.
- 19 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 26 (15209210) , 6785-6798
Improving Deepfake Detection Generalization by Invariant Risk Minimization
The abuse of deepfake techniques has raised serious concerns about social security and ethical problems, which motivates the development of deepfake detection. However, without fully addressing the domain gap issue, existing deepfake detection methods still show weak generalization ability among datasets belonging to different domains with domain-specific characteristics like identities and generation methods, limiting their practical applications. In this paper, we propose the Invariant Domain-oriented Deepfake Detection method (ID$_{3}$ ) , which improves the generalization of deepfake detection on multiple domains through invariant risk minimization, a novel learning paradigm that addresses the domain gap problem by jointly training a purified invariant predictor and learning an aligned invariant representation. To train a purified invariant predictor, we design the Domain Refinement Data Augmentation strategy with self-face-swapping and region-erasing approaches, which suppresses domain-specific features and encourages the models to focus on critical domain-invariant characteristics. To learn an aligned invariant representation, we propose the Domain Calibration Batch Normalization approach with multiple BN branches, which normalizes input features from different domains into aligned representations during both training and testing. Extensive experiments on multiple datasets demonstrate that our framework can boost the deepfake detection generalization ability and outperform other baselines by large margins. Our codes can be found here- 19 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 26 (15209210) , 6785-6798
Improving Deepfake Detection Generalization by Invariant Risk Minimization
The abuse of deepfake techniques has raised serious concerns about social security and ethical problems, which motivates the development of deepfake detection. However, without fully addressing the domain gap issue, existing deepfake detection methods still show weak generalization ability among datasets belonging to different domains with domain-specific characteristics like identities and generation methods, limiting their practical applications. In this paper, we propose the Invariant Domain-oriented Deepfake Detection method (ID$_{3}$ ) , which improves the generalization of deepfake detection on multiple domains through invariant risk minimization, a novel learning paradigm that addresses the domain gap problem by jointly training a purified invariant predictor and learning an aligned invariant representation. To train a purified invariant predictor, we design the Domain Refinement Data Augmentation strategy with self-face-swapping and region-erasing approaches, which suppresses domain-specific features and encourages the models to focus on critical domain-invariant characteristics. To learn an aligned invariant representation, we propose the Domain Calibration Batch Normalization approach with multiple BN branches, which normalizes input features from different domains into aligned representations during both training and testing. Extensive experiments on multiple datasets demonstrate that our framework can boost the deepfake detection generalization ability and outperform other baselines by large margins. Our codes can be found here