Development and Assessment of an Accessible AI-Powered Tool for Manuscript Evaluation Through Iterative Optimization in Plastic Surgery Research

Authors:

Abstract
Purpose Authors from low- and middle- income countries and non-English speaking regions face systemic disadvantages in accessing expert guidance for manuscript preparation. This study developed and evaluated an accessible, iteratively optimized artificial intelligence tool to provide structured feedback for manuscripts, aimed at assisting authors to refine their work prior to submission. Methods Sixty manuscript evaluations from fifteen plastic surgery articles (5 clinical studies, 5 clinical trials, 5 reviews) were assessed using three progressively refined Artificial Intelligence (AI) approaches accessible via standard ChatGPT account: Tool 1 (baseline GPT-5), Tool 2 (Custom GPT with peer review knowledge base), and Tool 3 (iteratively optimized with enhanced system instructions through meta-prompting and journal-specific guidelines). AI responses were compared with human peer reviews from open peer review journals. Two blinded raters scored evaluations using the validated Review Quality Instrument (maximum score = 40). Statistical analyses included descriptive statistics, intraclass correlation coefficients, and Friedman tests with post-hoc comparisons. Results The iteratively optimized Custom GPT (Tool 3) provided significantly higher mean scores for manuscript evaluation compared to all other review types (Tool 3 = 28.53 ± 3.76, Human = 25.17 ± 4.87, Tool 2 = 24.30 ± 5.02, Tool 1 = 24.00 ± 1.77; p < 0.001). Tool 3 demonstrated strengths in methodological assessment, evidence provision, and interpretation evaluation with the largest effects (Kendall's W = 0.69, 0.59, and 0.63). The tool maintained consistent performance across manuscript types (score range = 28.00-29.10), while human reviews showed greater variability (range = 22.10–27.00). Conclusions Systematic instruction optimization through meta-prompting and domain-specific knowledge integration significantly improves manuscript review quality. The iteratively optimized tool offers an accessible, scalable solution that provides structured, actionable feedback and increases access to manuscript preparation guidance, particularly benefiting researchers in resource-limited settings without technical expertise or Application Programming Interface (API) costs.

This publication has 33 references indexed in Scilit: