Evaluating large language model agents for automation of atomic force microscopy
Top Cited Papers
Open Access
- 14 October 2025
- journal article
- research article
- Published by Springer Nature in Nature Communications
- Vol. 16 (1) , 1-15
Abstract
Large language models (LLMs) are transforming laboratory automation by enabling self-driving laboratories (SDLs) that could accelerate materials research. However, current SDL implementations rely on rigid protocols that fail to capture the adaptability and intuition of expert scientists in dynamic experimental settings. Here, we show that LLM agents can automate atomic force microscopy (AFM) through our Artificially Intelligent Lab Assistant (AILA) framework. Further, we develop AFMBench—a comprehensive evaluation suite challenging LLM agents across the complete scientific workflow from experimental design to results analysis. We find that state-of-the-art LLMs struggle with basic tasks and coordination scenarios. Notably, models excelling at materials science question-answering perform poorly in laboratory settings, showing that domain knowledge does not translate to experimental capabilities. Additionally, we observe that LLM agents can deviate from instructions, a phenomenon referred to as sleepwalking, raising safety alignment concerns for SDL applications. Our ablations reveal that multi-agent frameworks significantly outperform single-agent approaches, though both remain sensitive to minor changes in instruction formatting or prompting. Finally, we evaluate AILA’s effectiveness in increasingly advanced experiments—AFM calibration, feature detection, mechanical property measurement, graphene layer counting, and indenter detection. These findings establish the necessity for benchmarking and robust safety protocols before deploying LLM agents as autonomous laboratory assistants across scientific disciplines.Keywords
All Related Versions
Funding Information
- Alexander von Humboldt-Stiftung
This publication has 45 references indexed in Scilit:
- Autonomous chemical research with large language modelsNature, 2023
- Machine learning-enabled autonomous operation for atomic force microscopesReview of Scientific Instruments, 2023
- An autonomous laboratory for the accelerated synthesis of inorganic materialsNature, 2023
- Demonstration of an AI-driven workflow for autonomous high-resolution scanning microscopyNature Communications, 2023
- Autonomous Molecular Structure Imaging with High-Resolution Atomic Force Microscopy for Molecular Mixture DiscoveryThe Journal of Physical Chemistry A, 2023
- Towards artefact‐free AFM image presentation and interpretationJournal of Microscopy, 2023
- Disentangling Electronic Transport and Hysteresis at Individual Grain Boundaries in Hybrid Perovskites via Automated Scanning Probe MicroscopyACS Nano, 2023
- Automated and Autonomous Experiments in Electron and Scanning Probe MicroscopyACS Nano, 2021
- Autonomous Experiments in Scanning Probe Microscopy and Spectroscopy: Choosing Where to Explore Polarization Dynamics in FerroelectricsACS Nano, 2021
- Baseline correction of AFM force curves in the force-time representationMicroscopy Research and Technique, 2016