SalChartQA: Question-driven Saliency on Information Visualisations
Open Access
- 11 May 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-14
Abstract
Understanding the link between visual attention and users’ information needs when visually exploring information visualisations is under-explored due to a lack of large and diverse datasets to facilitate these analyses. To fill this gap we introduce SalChartQA – a novel crowd-sourced dataset that uses the BubbleView interface to track user attention and a question-answering (QA) paradigm to induce different information needs in users. SalChartQA contains 74,340 answers to 6,000 questions on 3,000 visualisations. Informed by our analyses demonstrating the close correlation between information needs and visual saliency, we propose the first computational method to predict question-driven saliency on visualisations. Our method outperforms state-of-the-art saliency models for several metrics, such as the correlation coefficient and the Kullback-Leibler divergence. These results show the importance of information needs for shaping attentive behaviour and pave the way for new applications, such as task-driven optimisation of visualisations or explainable AI in chart question-answering.Keywords
Funding Information
- Ministry of Science, Research and the Arts Baden-Württemberg in the Artificial Intelligence Software Academy (AISA)
- Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy -- EXC 2075 (390740016)
- Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) (251654672 -- TRR 161)
- Swiss National Science Foundation (SNSF), Postdoc. Mobility Fellowship (214434)
- European Research Council (ERC) (801708)
This publication has 58 references indexed in Scilit:
- TranSalNet: Towards perceptually relevant visual saliency predictionNeurocomputing, 2022
- Adaptive feature guidance: Modelling visual search with graphical layoutsInternational Journal of Human-Computer Studies, 2020
- How to Fine-Tune BERT for Text Classification?2019
- Task-Driven Webpage Saliency2018
- Exploring visual attention and saliency modeling for task-based visual analysisComputers & Graphics, 2018
- Patterns of Attention: How Data Visualizations Are Read2017
- Efficient saliency analysis based on wavelet transform and entropy theoryJournal of Visual Communication and Image Representation, 2015
- ImageNet: A large-scale hierarchical image database2009
- Modeling the influence of task on attentionVision Research, 2005
- A model of saliency-based visual attention for rapid scene analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998