Abstract
Environment setup remains a critical bottleneck in software development, with Java projects presenting unique challenges due to complex build tool configurations, transitive dependencies, and compilation requirements. While existing solutions like Installamatic achieve 55% success on Python repositories, Java/JVM repositories show only 29.47% success rates with current approaches. We present SAG (Setup Agent), an LLM-based system specifically designed for Java project environment setup within Docker containers. SAG employs three key innovations: a dual-model ReAct architecture separating reasoning from action, hierarchical context management with a "Trunk/Branch" context system for complex task chains, and independent file validation ensuring objective assessment while maintaining autonomous discovery. Through evaluation on 15 Apache projects across four model configurations, SAG achieves 78.9% overall build success and 48.4% test execution rate. The o4-mini+GPT-4.1-mini configuration achieves the highest 84.4% success using only 17,956 tokens, compared to GPT-5+GPT-5’s 75.6% success with 99,400 tokens—a 5.5× reduction in computational resources with 8.8 percentage points higher success. The extreme 12.09 thinking-to-action token ratio validates our architectural separation. Our results establish that role specialization and focused model pairing outperform uniformly applying powerful models, paving the way for efficient, task-specific automation solutions.