Development and Assessment of an Accessible AI-Powered Tool for Manuscript Evaluation Through Iterative Optimization in Plastic Surgery Research
Preprint
- 28 June 2026
- preprint
- Published by Springer Nature in Research Square
Abstract
Purpose Authors from low- and middle- income countries and non-English speaking regions face systemic disadvantages in accessing expert guidance for manuscript preparation. This study developed and evaluated an accessible, iteratively optimized artificial intelligence tool to provide structured feedback for manuscripts, aimed at assisting authors to refine their work prior to submission. Methods Sixty manuscript evaluations from fifteen plastic surgery articles (5 clinical studies, 5 clinical trials, 5 reviews) were assessed using three progressively refined Artificial Intelligence (AI) approaches accessible via standard ChatGPT account: Tool 1 (baseline GPT-5), Tool 2 (Custom GPT with peer review knowledge base), and Tool 3 (iteratively optimized with enhanced system instructions through meta-prompting and journal-specific guidelines). AI responses were compared with human peer reviews from open peer review journals. Two blinded raters scored evaluations using the validated Review Quality Instrument (maximum score = 40). Statistical analyses included descriptive statistics, intraclass correlation coefficients, and Friedman tests with post-hoc comparisons. Results The iteratively optimized Custom GPT (Tool 3) provided significantly higher mean scores for manuscript evaluation compared to all other review types (Tool 3 = 28.53 ± 3.76, Human = 25.17 ± 4.87, Tool 2 = 24.30 ± 5.02, Tool 1 = 24.00 ± 1.77; p < 0.001). Tool 3 demonstrated strengths in methodological assessment, evidence provision, and interpretation evaluation with the largest effects (Kendall's W = 0.69, 0.59, and 0.63). The tool maintained consistent performance across manuscript types (score range = 28.00-29.10), while human reviews showed greater variability (range = 22.10–27.00). Conclusions Systematic instruction optimization through meta-prompting and domain-specific knowledge integration significantly improves manuscript review quality. The iteratively optimized tool offers an accessible, scalable solution that provides structured, actionable feedback and increases access to manuscript preparation guidance, particularly benefiting researchers in resource-limited settings without technical expertise or Application Programming Interface (API) costs.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Artificial Intelligence (AI)-Assisted Patient Education and Concerns Following Facelift Surgery: A Study on ChatGPT-4 and GeminiAesthetic Plastic Surgery, 2025
- Comparing AI-generated and human peer reviews: A study on 11 articlesHand Surgery and Rehabilitation, 2025
- How to review a manuscript for journal publication: A primer for surgery residentsSurgery Open Science, 2025
- Generative artificial intelligence and scientific publishing: urgent questions, difficult answersThe Lancet, 2024
- Using artificial intelligence in academic writing and research: An essential productivity toolComputer Methods and Programs in Biomedicine Update, 2024
- ChatGPT for scientific writing — The coexistence of opportunities and challengesJournal of Orthopaedic Translation, 2024
- Association of International Editorial Staff With Published Articles From Low- and Middle-Income CountriesJAMA Network Open, 2022
- Rejection Resilience—Quantifying Faculty Experience With Submitting Papers Multiple Times After a RejectionAcademic Pediatrics, 2022
- Preliminary evidence of linguistic bias in academic reviewingJournal of English for Academic Purposes, 2020
- Rejection Blues: Why Do Research Papers Get Rejected?The Journal of Obstetrics and Gynecology of India, 2018