Abstract
LLM agents increasingly rely on reinforcement learning to improve multi-turn reasoning, tool use, and environment interaction. However, existing training pipelines often treat an agent trajectory as a flat token sequence or a reconstructed message trace, making it difficult to preserve rollout fidelity, manage long contexts, and inspect step-level credit signals. We present Agent-R1, an open-source modular framework for agentic reinforcement learning (RL). Agent-R1 uses interaction steps as the native unit for trajectory representation, replay, and environment interaction, and decouples exact trajectory storage from flexible context construction. The system supports context-management policies, step-level rollout inspection, action-mask visualization, tool-feedback analysis, and unified comparison of related RL algorithms. Through demonstrations on multiple tasks, Agent-R1 enables users to configure, train, inspect, and compare agentic RL workflows within a single interactive interface. Code and project resources are available at https://github.com/AgentR1/Agent-R1.
Funding Information
  • National Natural Science Foundation of China (62502486)
  • National Natural Science Foundation of China (U25B2072)
  • Fundamental Research Funds for the Central Universities of China (N\/A)
  • USTC Kunpeng-Ascend Scientific and Education Innovation Excellence Center (N\/A)