Comments on “AI and the advent of the cyborg behavioral scientist”
- 16 March 2025
- journal article
- Published by Wiley in Journal of Consumer Psychology
- Vol. 35 (2) , 316-328
Abstract
Below are comments on Tomaino, Cooke, and Hoover by four teams of collaborative reviewers that helped clarify and focus its original version. Their comments on the refined version articulate how the fast‐moving world of generative AI can alter authors, readers, reviewers, and consumer behavior journals. In the first comment, Blythe, Kulis, and McGraw propose that Generative AI requires substantial effort to generate research that is fast, cost‐effective, and of high quality. They articulate three recommendations: to ask, to train, and to check the system. Asking builds on GenAI's ability to reveal its own capabilities at different stages of the research process. Training allows the system to be customized with relevant context, domain‐specific documents, and tailored examples, enhancing its accuracy and reducing errors. Checking is strongly advised to validate that the outputs are both reasonable and robust. Haenlein, Hewett, and Yoo build on the capabilities of Large Language Models that go beyond the research practices central to consumer psychology. They outline strategic prompting strategies: starting broadly and gradually narrowing to specific domains, downloading information from relevant articles and data that is unlikely to be part of the current corpus, and evoking specific theories, methods, or presentation formats. They also elaborate on the ways the apparent magic of GenAI may raise learning or ethical challenges. The third comment by Stacy Wood focuses less on the capabilities of GenAI and more on how its adoption will depend on researcher feelings—in other words, how different aspects of its use may alter researchers' experiences of doing research and their identities as scholars. GenAI has the potential to both build (through increased productivity or increased accessibility) and limit (through loss of agency or faster production) pride of purpose in research. She argues that feelings from using GenAI are likely to differ across research steps, from developing novel concepts, processes, analyses, and writing of the paper. Wherever GenAI may lessen the excitement, satisfaction, motivation, and perceived status of the researcher, barriers to its use are likely to be erected. Finally, Vicki Morwitz identifies new AI capabilities beyond those explored in Tomaino et al. Those include the ability to generate synthetic data that can guide empirical experiments, a facility to create audio and visual stimuli, a capability to study group behavior, and a capacity to reliably interpret complex human statements. The comment then closes with important questions for editorial policies, raising issues about limitations on AI use by authors, its appropriate applications by review teams, and possible publishers' restrictions on uploading copyrighted articles.Keywords
This publication has 23 references indexed in Scilit:
- Can Generative AI improve social science?Proceedings of the National Academy of Sciences, 2024
- Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelinesPsychology & Marketing, 2024
- Systematic testing of three Language Models reveals low language accuracy, absence of response stability, and a yes-response biasProceedings of the National Academy of Sciences, 2023
- Uncovering the semantics of concepts using GPT-4Proceedings of the National Academy of Sciences, 2023
- ChatGPT outperforms crowd workers for text-annotation tasksProceedings of the National Academy of Sciences, 2023
- Can AI language models replace human participants?Trends in Cognitive Sciences, 2023
- On ChatGPT and beyond: How generative artificial intelligence may affect research, teaching, and practiceInternational Journal of Research in Marketing, 2023
- Out of One, Many: Using Language Models to Simulate Human SamplesPolitical Analysis, 2023
- Should Birds of a Feather Flock Together? Understanding Self-Control Decisions in DyadsJournal of Consumer Research, 2014
- The weirdest people in the world?Behavioral and Brain Sciences, 2010