Human vs. Machine: Comparing AI-Generated and Human-Written Psychological Reports
- 12 June 2025
- journal article
- Published by SAGE Publications in Journal of Psychoeducational Assessment
- Vol. 43 (6) , 559-573
Abstract
This study examines the effectiveness of artificial intelligence (AI) in psychological report writing by comparing reports generated by human psychologists with those produced by OpenAI’s Generative Pre-trained Transformer Version 4 (ChatGPT-4). A total of 249 licensed psychologists evaluated the reports based on overall quality, readability, writing style, organization, summary quality, recommendations, preference, and willingness to sign off on the reports. Although human-generated reports were generally rated more favorably and participants expressed greater comfort in approving them, effect sizes were typically small. Two exceptions were noted: moderate effect sizes were found in favor of human-written summaries, while AI-generated reports showed moderate effect sizes for the quality of their recommendations. These findings suggest that AI shows potential for augmenting report writing. Comprehensive guidelines are necessary for the ethical and effective integration of AI into psychological practice. Further research is needed to enhance our understanding of AI’s role and capabilities in psychological assessment and reporting.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Artificial intelligence–derived dermatology case reports are indistinguishable from those written by humans: A single-blinded observer studyJournal of the American Academy of Dermatology, 2023
- The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual realityWorld Psychiatry, 2021
- The better-than-average effect in comparative self-evaluation: A comprehensive review and meta-analysis.Psychological Bulletin, 2020
- Artificial Intelligence for Mental Health and Mental Illnesses: an OverviewCurrent Psychiatry Reports, 2019
- Internet, Phone, Mail, and Mixed‐Mode Surveys2014
- Writing Useful, Accessible, and Legally Defensible Psychoeducational Reports2014
- Variables affecting the clarity of psychological reportsJournal of Clinical Psychology, 2005
- The psychological report: A review of current controversiesJournal of Clinical Psychology, 2005
- What Makes a Competent Psychologist?European Psychologist, 2002
- Comparison of Open and Closed Questionnaire Formats in Obtaining Demographic Information From Canadian General InternistsJournal of Clinical Epidemiology, 1999