Abstract
Accurate effort estimation in Agile and DevOps software projects remains a persistent challenge, as traditional models fail to account for the iterative, continuous delivery nature of modern development environments. Existing single-technique estimation approaches, whether algorithmic, expert-based, or machine learning-based, each capture only a partial view of project complexity, leading to systematic underestimation in Agile DevOps contexts. This study proposes the Hybrid Ensemble Effort Estimation Model (HEEEM), which integrates Use Case Points (UCP) as an algorithmic anchor, the Delphi method as an expert-based judgment layer, and Extra Trees as a data-driven machine learning component. A two-stage framework is adopted: candidate machine learning models are first benchmarked across eight public datasets, spanning 945 projects, providing external validation of the data-driven component and confirming Extra Trees as the most consistent performer (MAE: 0.288, RMSE: 0.398, R2: 0.815), before being embedded into the hybrid ensemble. Empirical evaluation is conducted through case studies on five industrial Agile DevOps projects using standard performance metrics including Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Mean Magnitude of Relative Error (MMRE) and R2.The findings show that HEEEM significantly improved effort estimation accuracy, achieving an in-sample MAE of 21.60, RMSE of 25.32, MMRE of 0.0053, and an in-sample R2 of 0.9983, representing reductions of 69% in MAE, 71.9% in RMSE, and 72% in MMRE compared to individual baseline methods;under Leave-One-Out Cross-Validation, the corresponding out-of-sample metrics are MAE = 72.88, RMSE = 100.42, MMRE= 0.0192, and R2 = 0.9731, representing 45–52% reductions in MAE compared to the algorithmic and expert-based individual baselines. Statistical validation through the Friedman test confirmed these improvements are statistically significant (p = 0.0136). The proposed model offers actionable guidance for project managers and development teams regarding sprint planning, resource allocation, and risk management, and is particularly suited for continuous delivery environments where estimates require regular updating.
Funding Information
  • United Arab Emirates University (12T051)
  • United Arab Emirates University (G00004635)

This publication has 55 references indexed in Scilit: