DataVisT5: A Pre-Trained Language Model for Jointly Understanding Text and Data Visualization
Authors:
- 19 May 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1704-1717
Abstract
Data visualization (DV) is the fundamental and premise tool to improve the efficiency in conveying the insights behind the big data, which has been widely accepted in existing data-driven world. Task automation in DV, such as converting natural language queries to visualizations (i.e., text-to-vis), gener-ating explanations from visualizations (i.e., vis-to-text), answering DV-related questions in free form (i.e. Fe VisQA), and explicating tabular data (i.e., table-to-text), is vital for advancing the field. Despite their potential, the application of pre-trained language models (PLMs) like T5 and BERT in DV has been limited by high costs and challenges in handling cross-modal information, leading to few studies on PLMs for DV. We introduce Data VisT5, a novel PLM tailored for DV that enhances the T5 architecture through a hybrid objective pre-training and multi-task fine-tuning strategy, integrating text and DV datasets to effectively interpret cross-modal semantics. Extensive evaluations on public datasets show that Data VisT5 consistently outperforms current state-of-the-art models and higher-parameter Large Language Models (LLMs) on various DV-related tasks. We anticipate that Data VisT5 will not only inspire further research on vertical PLMs but also expand the range of applications for PLMs.Keywords
All Related Versions
This publication has 41 references indexed in Scilit:
- Demonstration of FeVisQA: Free-Form Question Answering over Data Visualization2024
- Natural Language Generation Meets Data Visualization: Vis-to-Text and its Duality with Text-to-Vis2023
- Making data visualization more efficient and effective: a surveyThe VLDB Journal, 2019
- Data2Vis: Automatic Generation of Data Visualizations Using Sequence-to-Sequence Recurrent Neural NetworksIEEE Computer Graphics and Applications, 2019
- ECharts: A declarative framework for rapid construction of web-based visualizationVisual Informatics, 2018
- DeepEye: Towards Automatic Data Visualization2018
- Storytelling with Data2015
- SINA: Semantic interpretation of user queries for question answering on interlinked dataJournal of Web Semantics, 2015
- From keywords to semantic queries—Incremental query construction on the semantic webJournal of Web Semantics, 2009
- A Brief History of Data Visualization2008