Once Upon a Team: Investigating Bias in LLM-Driven Software Team Composition and Task Allocation
Preprint
Abstract
LLMs are increasingly used to boost productivity and support software engineering tasks. However, when applied to socially sensitive decisions such as team composition and task allocation, they raise concerns of fairness. Prior studies have revealed that LLMs may reproduce stereotypes; however, these analyses remain exploratory and examine sensitive attributes in isolation. This study investigates whether LLMs exhibit bias in team composition and task assignment by analyzing the combined effects of candidates' country and pronouns. Using three LLMs and 3,000 simulated decisions, we find systematic disparities: demographic attributes significantly shaped both selection likelihood and task allocation, even when accounting for expertise-related factors. Task distributions further reflected stereotypes, with technical and leadership roles unevenly assigned across groups. Our findings indicate that LLMs exacerbate demographic inequities in software engineering contexts, underscoring the need for fairness-aware assessment.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Fairness on a budget, across the board: A cost-effective evaluation of fairness-aware practices across contexts, tasks, and sensitive attributesInformation and Software Technology, 2025
- Fairness-aware practices from developers’ perspective: A surveyInformation and Software Technology, 2025
- Enhancing understanding and addressing gender bias in IT/SE job advertisementsJournal of Systems and Software, 2024
- Foundation and large language models: fundamentals, challenges, opportunities, and social impactsCluster Computing, 2023
- Large Language Models for Software Engineering: Survey and Open Problems2023
- Like, dislike, or just do it? How developers approach software development tasksInformation and Software Technology, 2022
- Learning Sentiment Analysis for Accessibility User Reviews2021
- Variance Inflation Factor and Condition Number in multiple linear regressionJournal of Statistical Computation and Simulation, 2018
- Introducing scrum process optimization2017
- An Introduction to Logistic Regression Analysis and ReportingThe Journal of Educational Research, 2002