Chaoyi Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Chaoyi Zhang has a total of 62 co-authors. Here are the top co-authors by number of shared publications:
- Weidong Cai
School of Computer Science, The University of Sydney
16 shared publications - Jianhui Yu
School of Computer Science The University of Sydney
8 shared publications - Yang Song
University of New South Wales
7 shared publications - Dongnan Liu
The University of Sydney
6 shared publications Weidong Cai
University of Sydney
6 shared publications- Shunqi Mao
School of Computer Science, The University of Sydney
3 shared publications
Latest Publications
- 4 October 2026
- Published by Springer Nature
- p. 362-381
Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorations
- 4 October 2026
- Published by Springer Nature
- p. 362-381
Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorations
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 3487-3496
RealSyn : An Effective and Scalable Multimodal Interleaved Document Transformation Paradigm
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 3487-3496
Open AccessRealSyn : An Effective and Scalable Multimodal Interleaved Document Transformation Paradigm
- 8 December 2024
- Published by Springer Nature
- p. 280-297
Enhancing Robustness to Noise Corruption for Point Cloud Recognition via Spatial Sorting and Set-Mixing Aggregation Module
- 8 December 2024
- Published by Springer Nature
- p. 280-297
Enhancing Robustness to Noise Corruption for Point Cloud Recognition via Spatial Sorting and Set-Mixing Aggregation Module
- Open Access
- 3 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Exploring Annotation-free Image Captioning with Retrieval-augmented Pseudo Sentence Generation
- 3 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Open AccessExploring Annotation-free Image Captioning with Retrieval-augmented Pseudo Sentence Generation
- 3 December 2024
- Published by Springer Nature
- p. 259-274
Learning to Synthesize Graphics Programs for Geometric Artworks
- 3 December 2024
- Published by Springer Nature
- p. 259-274
Learning to Synthesize Graphics Programs for Geometric Artworks
- 20 November 2024
- Published by Springer Nature
- p. 269-280
3DSSG-Cap: A Caption Enhanced Dataset for 3D Visual Grounding
- 20 November 2024
- Published by Springer Nature
- p. 269-280
3DSSG-Cap: A Caption Enhanced Dataset for 3D Visual Grounding
- 1 November 2024
- Published by Springer Nature
- p. 464-481
Controllable Contextualized Image Captioning: Directing the Visual Narrative Through User-Defined Highlights
- 1 November 2024
- Published by Springer Nature
- p. 464-481
Controllable Contextualized Image Captioning: Directing the Visual Narrative Through User-Defined Highlights
- Open Access
Controllable Contextualized Image Captioning: Directing the Visual Narrative through User-Defined Highlights
Contextualized Image Captioning (CIC) evolves traditional image captioning into a more complex domain, necessitating the ability for multimodal reasoning. It aims to generate image captions given specific contextual information. This paper further introduces a novel domain of Controllable Contextualized Image Captioning (Ctrl-CIC). Unlike CIC, which solely relies on broad context, Ctrl-CIC accentuates a user-defined highlight, compelling the model to tailor captions that resonate with the highlighted aspects of the context. We present two approaches, Prompting-based Controller (P-Ctrl) and Recalibration-based Controller (R-Ctrl), to generate focused captions. P-Ctrl conditions the model generation on highlight by prepending captions with highlight-driven prefixes, whereas R-Ctrl tunes the model to selectively recalibrate the encoder embeddings for highlighted tokens. Additionally, we design a GPT-4V empowered evaluator to assess the quality of the controlled captions alongside standard assessment methods. Extensive experimental results demonstrate the efficient and effective controllability of our method, charting a new direction in achieving user-adaptive image captioning. Code is available at https://github.com/ShunqiM/Ctrl-CIC .
Open AccessControllable Contextualized Image Captioning: Directing the Visual Narrative through User-Defined Highlights
Contextualized Image Captioning (CIC) evolves traditional image captioning into a more complex domain, necessitating the ability for multimodal reasoning. It aims to generate image captions given specific contextual information. This paper further introduces a novel domain of Controllable Contextualized Image Captioning (Ctrl-CIC). Unlike CIC, which solely relies on broad context, Ctrl-CIC accentuates a user-defined highlight, compelling the model to tailor captions that resonate with the highlighted aspects of the context. We present two approaches, Prompting-based Controller (P-Ctrl) and Recalibration-based Controller (R-Ctrl), to generate focused captions. P-Ctrl conditions the model generation on highlight by prepending captions with highlight-driven prefixes, whereas R-Ctrl tunes the model to selectively recalibrate the encoder embeddings for highlighted tokens. Additionally, we design a GPT-4V empowered evaluator to assess the quality of the controlled captions alongside standard assessment methods. Extensive experimental results demonstrate the efficient and effective controllability of our method, charting a new direction in achieving user-adaptive image captioning. Code is available at https://github.com/ShunqiM/Ctrl-CIC .
- 26 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (9) , 6070-6081
Exploiting Structural Consistency of Chest Anatomy for Unsupervised Anomaly Detection in Radiography Images
Radiography imaging protocols focus on particular body regions, therefore producing images of great similarity and yielding recurrent anatomical structures across patients. Exploiting this structured information could potentially ease the detection of anomalies from radiography images. To this end, we propose a Sim ple S pace-Aware Memory Matrix for I n-painting and D etecting anomalies from radiography images (abbreviated as SimSID). We formulate anomaly detection as an image reconstruction task, consisting of a space-aware memory matrix and an in-painting block in the feature space. During the training, SimSID can taxonomize the ingrained anatomical structures into recurrent visual patterns, and in the inference, it can identify anomalies (unseen/modified visual patterns) from the test image. Our SimSID surpasses the state of the arts in unsupervised anomaly detection by +8.0%, +5.0%, and +9.9% AUC scores on ZhangLab, COVIDx, and CheXpert benchmark datasets, respectively.
- 26 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (9) , 6070-6081
Exploiting Structural Consistency of Chest Anatomy for Unsupervised Anomaly Detection in Radiography Images
Radiography imaging protocols focus on particular body regions, therefore producing images of great similarity and yielding recurrent anatomical structures across patients. Exploiting this structured information could potentially ease the detection of anomalies from radiography images. To this end, we propose a Sim ple S pace-Aware Memory Matrix for I n-painting and D etecting anomalies from radiography images (abbreviated as SimSID). We formulate anomaly detection as an image reconstruction task, consisting of a space-aware memory matrix and an in-painting block in the feature space. During the training, SimSID can taxonomize the ingrained anatomical structures into recurrent visual patterns, and in the inference, it can identify anomalies (unseen/modified visual patterns) from the test image. Our SimSID surpasses the state of the arts in unsupervised anomaly detection by +8.0%, +5.0%, and +9.9% AUC scores on ZhangLab, COVIDx, and CheXpert benchmark datasets, respectively.
- 23 February 2024
- Published by Elsevier in Medical Image Analysis
- Vol. 94, 103120
TractGeoNet: A geometric deep learning framework for pointwise analysis of tract microstructure to predict language assessment performance
- 23 February 2024
- Published by Elsevier in Medical Image Analysis
- Vol. 94, 103120
TractGeoNet: A geometric deep learning framework for pointwise analysis of tract microstructure to predict language assessment performance