Abstract
Multi-agent systems (MAS) based on large language models (LLMs) are on the rise due to their improved capabilities compared to single agents. In particular, among the emerging MAS for software development, those establishing automated agentic workflows contribute to the idea that software development is an essentially solved task. This study aims at critically assessing five MAS for software development that cover multiple phases and support automated workflow: MetaGPT, ChatDev, AgileCoder, HyperAgent and AgentLite. Here, 14 research questions are identified, specifically referring to architectural-, technical-, operational-, human interaction- and responsible AI-aspects. The findings from the study can help interested researchers and practitioners to understand which design choices bring advantages for particular aspects and where, on the contrary, existing solutions are suboptimal and what could be suggestions for addressing the gaps. Due to its research question-oriented layout, this paper is ideal as a reference which, for a particular aspect, facilitates grasping the overview of main design choices, capabilities, limitation and reconciliation approaches.