A Lightweight Data-Augmented Deep Learning Framework for Real-Time Instance Segmentation in Liquid-Phase In Situ Transmission Electron Microscopy
Open Access
- 13 February 2026
- journal article
- Published by American Chemical Society (ACS) in ACS Measurement Science Au
- Vol. 6 (2) , 476-487
Abstract
Liquid-phase transmission electron microscopy (LPTEM) offers critical insights into the dynamic behaviors of materials, but automated analysis is hindered by the scarcity of annotated data and the high cost of image acquisition. To address these challenges, we propose an LPTEM deep learning framework that (i) closes the annotation gap, (ii) localizes nanoparticles through real-time detection, and (iii) preserves per-pixel detail. Through CycleGAN, we are able to generate a large number of images that are style-consistent with real LPTEM frames, effectively expanding the data set without manual labeling. After augmenting the data set, our fine-tuned YOLOv11n model achieves 97.66% precision and 99.05% mAP50 for detecting nanoparticles, closely approaching the performance of models trained on real data. Three Mobile-UNet variants, optimized for different computational efficiency and accuracy trade-offs, demonstrate instance segmentation for in situ TEM with intersection over union (IoU) scores ranging from 0.8685 to 0.9207 (on a scale where 1.0 indicates perfect overlap with the ground truth), indicating highly accurate particle shapes. Our integrated framework significantly enhances real-time detection and segmentation performance of nanoparticles in LPTEM, with YOLO + Mobile-UNet Slim achieving a real-time processing speed of 102.34 FPS. This framework establishes a scalable, annotation-efficient route to high-precision, real-time LPTEM analysis, opening the door to autonomous in situ experiments and accelerated materials discovery.Keywords
Funding Information
- Nvidia
- Google Research
- National Science and Technology Council (NSTC 112-2221-E-006-151-MY3)
This publication has 49 references indexed in Scilit:
- Applications of liquid-phase TEM in the fields of electrocatalysis and photocatalysisNano Today, 2025
- Exploring Biomineralization Processes Using In Situ Liquid Transmission Electron Microscopy: A ReviewSmall, 2024
- SAM-I-Am: Semantic boosting for zero-shot atomic-scale electron micrograph segmentationComputational Materials Science, 2024
- Using In situ Transmission Electron Microscopy to Study Strong Metal‐Support Interactions in Heterogeneous CatalysisAngewandte Chemie International Edition, 2024
- Thermal‐Induced Structure Evolution at the Interface between Cathode and Solid‐State ElectrolyteSmall Structures, 2023
- Artificial Scanning Electron Microscopy Images Created by Generative Adversarial Networks from Simulated Particle AssembliesAdvanced Intelligent Systems, 2023
- In Situ TEM Studies on Electrochemical Mechanisms of Rechargeable Ion Battery CathodesSmall Structures, 2023
- Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images2022
- A deep learning based automatic defect analysis framework for In-situ TEM ion irradiationsComputational Materials Science, 2021
- Deep MR to CT Synthesis Using Unpaired Data2017