Interactive Table Synthesis With Natural Language
- 1 November 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 30 (9) , 6130-6145
Abstract
Tables are a ubiquitous data format for insight communication. However, transforming data into consumable tabular views remains a challenging and time-consuming task. To lower the barrier of such a task, research efforts have been devoted to developing interactive approaches for data transformation, but many approaches still presume that their users have considerable knowledge of various data transformation concepts and functions. In this study, we leverage natural language (NL) as the primary interaction modality to improve the accessibility of average users to performing complex data transformation and facilitate intuitive table generation and editing. Designing an NL-driven data transformation approach introduces two challenges: 1) NL-driven synthesis of interpretable pipelines and 2) incremental refinement of synthesized tables. To address these challenges, we present NL2Rigel, an interactive tool that assists users in synthesizing and improving tables from semi-structured text with NL instructions. Based on a large language model and prompting techniques, NL2Rigel can interpret the given NL instructions into a table synthesis pipeline corresponding to Rigel specifications, a declarative language for tabular data transformation. An intuitive interface is designed to visualize the synthesis pipeline and the generated tables, helping users understand the transformation process and refine the results efficiently with targeted NL instructions. The comprehensiveness of NL2Rigel is demonstrated with an example gallery, and we further confirmed NL2Rigel's usability with a comparative user study by showing that the task completion time with NL2Rigel is significantly shorter than that with the original version of Rigel with comparable completion rates.Keywords
Funding Information
- National Key R&D Program of China (2022YFE0137800)
- National Natural Science Foundation of China (U22A2032)
- Collaborative Innovation Center of Artificial Intelligence by MOE and Zhejiang Provincial Government
This publication has 65 references indexed in Scilit:
- Can language models automate data wrangling?Machine Learning, 2022
- FlowSense: A Natural Language Interface for Visual Data Exploration within a Dataflow SystemIEEE Transactions on Visualization and Computer Graphics, 2019
- Explaining Queries Over Web Tables to Non-experts2019
- Orko: Facilitating Multimodal Interaction for Visual Exploration and Analysis of NetworksIEEE Transactions on Visualization and Computer Graphics, 2017
- Tabular Data Anomaly Patterns2017
- Vega-Lite: A Grammar of Interactive GraphicsIEEE Transactions on Visualization and Computer Graphics, 2016
- DataXFormer: A robust transformation discovery system2016
- Initial Implementation of Natural Language Turn-Based Dialog SystemProcedia Computer Science, 2016
- Natural language interfaces to databases – an introductionNatural Language Engineering, 1995
- The Pegasus heterogeneous multidatabase systemComputer, 1991