LLMs in the Laboratory: Benchmarking Conversational AI for Multivariate Analysis of NMR Spectroscopic Data
Open Access
- 27 October 2026
- journal article
- Published by American Chemical Society (ACS) in Analytical Chemistry
- Vol. 98 (45) , 32167-32179
Abstract
This study presents a comparative evaluation of five state-of-the-art large language models (LLMs) (Claude Opus 4.6, Gemini 3.1 Pro, Grok-4, ChatGPT (GPT-5.4 Thinking), and Qwen 3.5) for their ability to perform key chemometric tasks through natural language interaction. Using a low-field nuclear magnetic resonance (LF-NMR) dataset of edible vegetable oils, the models were prompted to execute a complete multivariate analysis workflow, including data exploration, unsupervised and supervised classification, model validation, and interpretation. Results reveal a clear performance hierarchy: Claude Opus 4.6 was the only model to autonomously execute the entire pipeline with the highest analytical rigor, correctly predicting all nine unlabeled samples. Gemini 3.1 Pro and ChatGPT (GPT-5.4 Thinking) also demonstrated strong capabilities, achieving 100% or near-100% classification accuracy within fallback or native workflows, positioning them as effective analytical co-pilots. Grok-4 produced near-perfect classification results but showed limitations in data handling autonomy and exploratory depth. Qwen 3.5, the only open-source model, suffered a critical, unrecoverable failure at the supervised classification stage after repeated attempts, preventing workflow completion. Overall, the results indicate that linguistic fluency should not be taken as a reliable indicator of analytical competence and that general-purpose benchmarks may have limitations when assessing domain-specific chemometric performance. Critical validation and human oversight remain essential for the trustworthy deployment of these tools in spectroscopy-based workflows.Keywords
This publication has 25 references indexed in Scilit:
- Expanding horizons: Toward the classification of extra virgin olive oil using 1H NMR and machine learningFood Control, 2025
- Large Language Models (such as ChatGPT) as Tools for Machine Learning-Based Data Insights in Analytical ChemistryAnalytical Chemistry, 2025
- Benchtop NMR spectroscopy for quantitative determination of milk fat and qualitative determination of lactose: From calibration curve to deep learningLWT, 2024
- Applicability of ChatGPT 3.5 in the development of new analytical proceduresMicrochemical Journal, 2024
- Augmenting large language models with chemistry toolsNature Machine Intelligence, 2024
- AI in analytical chemistry: Advancements, challenges, and future directionsTalanta, 2024
- How ChatGPT is transforming the postdoc experienceNature, 2023
- Can ChatGPT provide intelligent diagnoses? A comparative study between predictive models and ChatGPT to define a new medical diagnostic botExpert Systems with Applications, 2023
- The future of chemistry is languageNature Reviews Chemistry, 2023
- Deep metabolome: Applications of deep learning in metabolomicsComputational and Structural Biotechnology Journal, 2020