SMaRTAban: a voice-controlled LLM agent for quadruped mobile robotics with integrated vision
Open Access
- 21 September 2026
- journal article
- research article
- Published by Springer Nature in International Journal of Intelligent Robotics and Applications
- p. 1-17
Abstract
Large Language Models (LLMs) have emerged as powerful tools for enabling intuitive human-robot interaction through natural language. However, existing approaches often lack evaluated voice control in languages other than English, integrated vision, and comprehensive safety mechanisms. We present SMaRTAban (Smart Multi-legged Robotic Transformer Agent Artaban), a system that combines a LangGraph ReAct agent with the quadruped mobile robot Artaban for natural language control. Our system integrates: (1) voice commands with automatic language detection via Whisper, evaluated on English and Slovak, (2) on-demand scene interpretation via GPT-4o vision, (3) a safety-focused system prompt with interruptible tool execution, and (4) a React-based GUI for transparent operation monitoring. A pilot voice-control evaluation on 120 samples showed robust semantic understanding across English and Slovak (BERTScore F1: 0.87–1.00), despite transcription challenges in Slovak. We further conducted 4,800 accepted tool-calling trials across two models, six agent and prompting configurations, two languages, and 10 command scenarios. The strongest configuration combined iterative ReAct execution, an explicit system prompt, and few-shot prompting with three task-specific movement examples, achieving a tool-call F1 of 0.9977 and 96.5% exact-sequence success with GPT-4o. A separate performance evaluation showed that remote transcription and model processing dominated response time, whereas local ROS communication contributed only a few milliseconds.Keywords
Funding Information
- NextGenerationEU (09I05-03-V02-00039)
This publication has 21 references indexed in Scilit:
- Benefit-cost frontier-aware semantic reasoning for zero-shot object navigationApplied Intelligence, 2026
- Reducing Latency in LLM-Based Natural Language Commands Processing for Robot Navigation2025
- LLM-driven agent for speech-enabled control of industrial robots: A case study in snow-crab quality inspectionResults in Engineering, 2025
- LLM-Driven Augmented Reality Puppeteer: Controller-Free Voice-Commanded Robot Teleoperation2025
- Enabling Novel Mission Operations and Interactions with ROSA: The Robot Operating System Agent2025
- Deployment of Large Language Models to Control Mobile Robots at the Edge2025
- Prompt a Robot to Walk with Large Language Models2024
- Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents2024
- ChatGPT for Robotics: Design Principles and Model AbilitiesIEEE Access, 2024
- Code as Policies: Language Model Programs for Embodied Control2023