AgentInspect: Diagnosing Behavioral Failures in Artificial Intelligence Agents

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Effectively testing Artificial Intelligence (AI) agents remains a fundamental challenge due to their stochastic reasoning, vast and diverse input space, reliance on external tools, and operation in dynamic execution environments; factors that demand new testing methodologies explicitly tailored to the complex and interactive nature of agent-based systems. This work presents a novel methodology for testing AI agents, with a particular focus on assessing their behavioral robustness under varied operational conditions. Our approach relies on following key technical innovations: (1) a coverage-guided test input generation strategy based on agent- specific coverage objectives, (2) a capture-and-simulate mechanism that systematically emulates abnormal tool behaviors to mimic real-world execution failures, and (3) a deterministic behavioral failure detection approach that enables consistent identification of failures across different test inputs. We developed AgentInspect, a framework that automatically detects six types of behavioral failures in LangChain-based AI agents by analyzing their execution trajectories across three evaluation settings: a baseline setting using real tool responses, a simulated setting incorporating synthetic tool responses, and a hybrid setting that combines the real and simulated tool responses. To evaluate our approach, we curated a benchmark of 35 AI agents obtained from GitHub. Our results show that AgentInspect consistently identifies different behavioral failures with high precision and recall across all three execution settings. In particular, the simulated and hybrid settings expose failure modes that do not emerge during baseline execution with real tool responses, thereby enabling a more comprehensive assessment of agent robustness. Our findings highlight AgentInspect’s effectiveness in revealing critical failures and its practical utility for systematic robustness evaluation of AI agents.
Funding Information
  • National Science Foundation (CCF-2223812)
  • National Science Foundation (CNS-2120448)
  • National Science Foundation (IIS-2419882)
  • National Science Foundation (IIS-2419883)
  • Canadian Institute for Advanced Research (334134)
  • Canadian Institute for Advanced Research (10945 )
  • Natural Sciences and Engineering Research Council of Canada (RGPIN-2019-06956)
  • Canada Research Chairs Program (CRC-2022-00339)

This publication has 23 references indexed in Scilit: