Zheng Ge
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Zheng Ge has a total of 31 co-authors. Here are the top co-authors by number of shared publications:
- Chunrui Han
Megvii Technology, Beijing, China
4 shared publications - Xiangyu Zhang
Megvii Technology, Beijing, China
4 shared publications - Jianjian Sun
StepFun, Beijing, China
2 shared publications - Jinrong Yang
Huazhong University of Science and Technology, Wuhan, China
2 shared publications - Liang Zhao
Megvii Technology, Beijing, China
2 shared publications - Runpei Dong
Xi’an Jiaotong University
2 shared publications
Latest Publications
- Open Access
- 27 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 24-38
DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models
- 27 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 24-38
Open AccessDistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 147-155
OneChart: Purify the Chart Structural Extraction via One Auxiliary Token
Chart parsing poses a significant challenge due to the diversity of styles, values, texts, and so forth. Even advanced large vision-language models (LVLMs) with billions of parameters struggle to handle such tasks satisfactorily. To address this, we propose OneChart: a reliable agent specifically devised for the structural extraction of chart information. Similar to popular LVLMs, OneChart incorporates an autoregressive main body. Uniquely, to enhance the reliability of the numerical parts of the output, we introduce an auxiliary token placed at the beginning of the total tokens along with an additional decoder. The numerically optimized (auxiliary) token allows subsequent tokens for chart parsing to capture enhanced numerical features through causal attention. Furthermore, with the aid of the auxiliary token, we have devised a self-evaluation mechanism that enables the model to gauge the reliability of its chart parsing results by providing confidence scores for the generated content. Compared to current state-of-the-art (SOTA) chart parsing models, e.g., DePlot, ChartVLM, ChartAst, OneChart significantly outperforms in Average Precision (AP) for chart structural extraction across multiple public benchmarks, despite enjoying only 0.2 billion parameters. Moreover, as a chart parsing agent, it also brings 10%+ accuracy gains for the popular LVLM (LLaVA-1.6) in the downstream ChartQA benchmark. Our code, model and datasets ara available at https://onechartt.github.io.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 147-155
Open AccessOneChart: Purify the Chart Structural Extraction via One Auxiliary Token
Chart parsing poses a significant challenge due to the diversity of styles, values, texts, and so forth. Even advanced large vision-language models (LVLMs) with billions of parameters struggle to handle such tasks satisfactorily. To address this, we propose OneChart: a reliable agent specifically devised for the structural extraction of chart information. Similar to popular LVLMs, OneChart incorporates an autoregressive main body. Uniquely, to enhance the reliability of the numerical parts of the output, we introduce an auxiliary token placed at the beginning of the total tokens along with an additional decoder. The numerically optimized (auxiliary) token allows subsequent tokens for chart parsing to capture enhanced numerical features through causal attention. Furthermore, with the aid of the auxiliary token, we have devised a self-evaluation mechanism that enables the model to gauge the reliability of its chart parsing results by providing confidence scores for the generated content. Compared to current state-of-the-art (SOTA) chart parsing models, e.g., DePlot, ChartVLM, ChartAst, OneChart significantly outperforms in Average Precision (AP) for chart structural extraction across multiple public benchmarks, despite enjoying only 0.2 billion parameters. Moreover, as a chart parsing agent, it also brings 10%+ accuracy gains for the popular LVLM (LLaVA-1.6) in the downstream ChartQA benchmark. Our code, model and datasets ara available at https://onechartt.github.io.
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 291-300
Self-Supervised Visual Preference Alignment
This paper makes the first attempt towards unsupervised preference alignment in Vision-Language Models (VLMs). We generate chosen and rejected responses with regard to the original and augmented image pairs, and conduct preference alignment with direct preference optimization. It is based on a core idea: properly designed augmentation to the image input will induce VLM to generate false but hard negative responses, which helps the model to learn from and produce more robust and powerful answers. The whole pipeline no longer hinges on supervision from GPT-4 or human involvement during alignment, and is highly efficient with few lines of code. With only 8k randomly sampled unsupervised data, it achieves 90% relative score to GPT-4 on complex reasoning in LLaVA-Bench, and improves LLaVA-7B/13B by 6.7%/5.6% score on complex multi-modal benchmark MM-Vet. Visualizations shows its improved ability to align with user-intentions. A series of ablations are firmly conducted to reveal the latent mechanism of the approach, which also indicates its potential towards further scaling.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 291-300
Open AccessSelf-Supervised Visual Preference Alignment
This paper makes the first attempt towards unsupervised preference alignment in Vision-Language Models (VLMs). We generate chosen and rejected responses with regard to the original and augmented image pairs, and conduct preference alignment with direct preference optimization. It is based on a core idea: properly designed augmentation to the image input will induce VLM to generate false but hard negative responses, which helps the model to learn from and produce more robust and powerful answers. The whole pipeline no longer hinges on supervision from GPT-4 or human involvement during alignment, and is highly efficient with few lines of code. With only 8k randomly sampled unsupervised data, it achieves 90% relative score to GPT-4 on complex reasoning in LLaVA-Bench, and improves LLaVA-7B/13B by 6.7%/5.6% score on complex multi-modal benchmark MM-Vet. Visualizations shows its improved ability to align with user-intentions. A series of ablations are firmly conducted to reveal the latent mechanism of the approach, which also indicates its potential towards further scaling.
- 7 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (11) , 10487-10494
GroupLane: End-to-End 3D Lane Detection With Channel-Wise Grouping
Efficiency is quite important for 3D lane detection while previous detectors are either computationally expensive or difficult for optimization. To bridge this gap, we propose a fully convolutional detector named GroupLane, which is simple, fast, and still maintains high detection precision. Specifically, we first propose to split extracted feature into multiple groups along the channel dimension and employ every group to represent a prediction. In this way, GroupLane realizes end-to-end detection like DETR based on pure convolutional neural networks. Then, we propose to represent lanes by performing row-wise classification in bird's eye view and devise a set of detection heads. Compared with existing row-wise classification implementations that only support recognizing vertical lanes, ours can detect both vertical and horizontal ones. Additionally, a matching algorithm named single-win one-to-one matching is developed to associate predictions with labels during training. Extensive experiments are conducted to verify the effectiveness of the proposed strategies, and the results suggest that GroupLane achieves the best performance with high inference speed on both the popular OpenLane and Once-3DLanes benchmarks. In addition, GroupLane is the first fully convolutional 3D lane detector that achieves end-to-end detection without post-processing.
- 7 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (11) , 10487-10494
GroupLane: End-to-End 3D Lane Detection With Channel-Wise Grouping
Efficiency is quite important for 3D lane detection while previous detectors are either computationally expensive or difficult for optimization. To bridge this gap, we propose a fully convolutional detector named GroupLane, which is simple, fast, and still maintains high detection precision. Specifically, we first propose to split extracted feature into multiple groups along the channel dimension and employ every group to represent a prediction. In this way, GroupLane realizes end-to-end detection like DETR based on pure convolutional neural networks. Then, we propose to represent lanes by performing row-wise classification in bird's eye view and devise a set of detection heads. Compared with existing row-wise classification implementations that only support recognizing vertical lanes, ours can detect both vertical and horizontal ones. Additionally, a matching algorithm named single-win one-to-one matching is developed to associate predictions with labels during training. Extensive experiments are conducted to verify the effectiveness of the proposed strategies, and the results suggest that GroupLane achieves the best performance with high inference speed on both the popular OpenLane and Once-3DLanes benchmarks. In addition, GroupLane is the first fully convolutional 3D lane detector that achieves end-to-end detection without post-processing.
- Top Cited Papers
- 30 September 2024
- Published by Springer Nature
- p. 214-238
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction
- 30 September 2024
- Published by Springer Nature
- p. 214-238
Top Cited PapersShapeLLM: Universal 3D Object Understanding for Embodied Interaction
- Top Cited Papers
- 15 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (7) , 6544-6551
Exploring Recurrent Long-Term Temporal Fusion for Multi-View 3D Perception
Long-term temporal fusion is a crucial but often overlooked technique in camera-based Bird's-Eye-View (BEV) 3D perception. Existing methods are mostly in a parallel manner. While parallel fusion can benefit from long-term information, it suffers from increasing computational and memory overheads as the fusion window size grows. Alternatively, BEVFormer adopts a recurrent fusion pipeline so that history information can be efficiently integrated, yet it fails to benefit from longer temporal frames. In this letter, we explore an embarrassingly simple long-term recurrent fusion strategy built upon the LSS-based methods and find it already able to enjoy the merits from both sides, i.e., rich long-term information and efficient fusion pipeline. A temporal embedding module is further proposed to improve the model's robustness against occasionally missed frames in practical scenarios. We name this simple but effective fusing pipeline VideoBEV. Experimental results on the nuScenes benchmark show that VideoBEV obtains strong performance on various camera-based 3D perception tasks, including object detection ( 55.4% mAP and 62.9% NDS), segmentation ( 48.6% vehicle mIoU), tracking ( 54.8% AMOTA), and motion prediction ( 0.80 m minADE and 0.463 EPA).
- 15 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (7) , 6544-6551
Top Cited PapersExploring Recurrent Long-Term Temporal Fusion for Multi-View 3D Perception
Long-term temporal fusion is a crucial but often overlooked technique in camera-based Bird's-Eye-View (BEV) 3D perception. Existing methods are mostly in a parallel manner. While parallel fusion can benefit from long-term information, it suffers from increasing computational and memory overheads as the fusion window size grows. Alternatively, BEVFormer adopts a recurrent fusion pipeline so that history information can be efficiently integrated, yet it fails to benefit from longer temporal frames. In this letter, we explore an embarrassingly simple long-term recurrent fusion strategy built upon the LSS-based methods and find it already able to enjoy the merits from both sides, i.e., rich long-term information and efficient fusion pipeline. A temporal embedding module is further proposed to improve the model's robustness against occasionally missed frames in practical scenarios. We name this simple but effective fusing pipeline VideoBEV. Experimental results on the nuScenes benchmark show that VideoBEV obtains strong performance on various camera-based 3D perception tasks, including object detection ( 55.4% mAP and 62.9% NDS), segmentation ( 48.6% vehicle mIoU), tracking ( 54.8% AMOTA), and motion prediction ( 0.80 m minADE and 0.463 EPA).