From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews
Open Access
- 22 March 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 61-65
Abstract
Although numerous studies have examined how researchers use Large Language Models (LLMs) in academic writing, their concrete implementation, limitations, and design challenges in the literature review process remain underexplored. In this paper, we present findings from a user study with researchers across multiple disciplines that characterize current practices, benefits, and pain points when using LLMs to investigate related work. We identified three recurring gaps: (i) lack of trust in outputs, (ii) a persistent verification burden, and (iii) fragmentation across tools. To address these gaps, we propose six design goals and a high-level framework that operationalizes them through improved visualization of related papers, verification at every step, and human-feedback alignment with generation-guided explanationsKeywords
All Related Versions
Funding Information
- Notre Dame-IBM Technology Ethics Lab (11199)
This publication has 20 references indexed in Scilit:
- A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open QuestionsACM Transactions on Information Systems, 2025
- Use of large language models as artificial intelligence tools in academic research and publishing among global clinical researchersScientific Reports, 2024
- Large Language Models for Literature Reviews - An Exemplary Comparison of LLM-based Approaches with Manual Methods2024
- AI in Literature Reviews: a survey of current and emerging methods2024
- Artificial intelligence for literature reviews: opportunities and challengesArtificial Intelligence Review, 2024
- Can Generative AI improve social science?Proceedings of the National Academy of Sciences, 2024
- The application of large language models in medicine: A scoping reviewiScience, 2024
- Three ways ChatGPT helps me in my academic writingNature, 2024
- Large Language Models are Zero-Shot Rankers for Recommender Systems2024
- Target-aware Abstractive Related Work Generation with Contrastive Learning | Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval2022