What is the Role of Large Language Models in the Evolution of Astronomy Research?
Authors:
- Morgan FouesneauMFMorgan Fouesneau
- Ivelina G. MomchevaIMIvelina G. Momcheva
- Urmila ChadayammuriUCUrmila Chadayammuri
- Mariia DemianenkoMDMariia Demianenko
- Antoine DumontADAntoine Dumont
- Raphael E. HvidingRHRaphael E. Hviding
- K. Angelique KahleKKK. Angelique Kahle
- Bhavesh RajpootBRBhavesh Rajpoot
- Marten B. ScheuckMSMarten B. Scheuck
- Rhys SeeburgerRSRhys Seeburger
- Dmitry SemenovDSDmitry Semenov
Preprint
Abstract
ChatGPT and other state-of-the-art large language models (LLMs) are rapidly transforming multiple fields, offering powerful tools for a wide range of applications. These models, commonly trained on vast datasets, exhibit human-like text generation capabilities, making them useful for research tasks such as ideation, literature review, coding, drafting, and outreach. We conducted a study involving 13 astronomers at different career stages and research fields to explore LLM applications across diverse tasks over several months and to evaluate their performance in research-related activities. This work was accompanied by an anonymous survey assessing participants' experiences and attitudes towards LLMs. We provide a detailed analysis of the tasks attempted and the survey answers, along with specific output examples. Our findings highlight both the potential and limitations of LLMs in supporting research while also addressing general and research-specific ethical considerations. We conclude with a series of recommendations, emphasizing the need for researchers to complement LLMs with critical thinking and domain expertise, ensuring these tools serve as aids rather than substitutes for rigorous scientific inquiry.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Friend or foe? Exploring the implications of large language models on the science systemAI & SOCIETY, 2023
- Accuracy of Chatbots in Citing Journal ArticlesJAMA Network Open, 2023
- GPT detectors are biased against non-native English writersPatterns, 2023
- How to cheat on your final paper: Assigning AI for student writingAI & SOCIETY, 2022
- A study of gender in user reviews on the Google Play StoreEmpirical Software Engineering, 2021
- Speeding up to keep up: exploring the use of AI in the research processAI & SOCIETY, 2021
- When eliminating bias isn’t fair: Algorithmic reductionism and procedural justice in human resource decisionsOrganizational Behavior and Human Decision Processes, 2020
- Gender differences in participation and reward on Stack OverflowEmpirical Software Engineering, 2019
- Gender and the GeoWeb: divisions in the production of user-generated cartographic informationGeoJournal, 2013
- Steam power, establishment size, and labor productivity growth in nineteenth century American manufacturingExplorations in Economic History, 2008