OACE: An Adaptive Multicriteria Approach for Deep Learning Model Selection Across Assertiveness-Cost Trade-Offs

Authors:

Abstract
The advancement of deep learning has imposed new challenges on model evaluation, requiring a balance between assertiveness and computational cost. The state of the art reveals that this validation still depends on subjective and manual analyses that cannot systematically quantify the balance between these criteria. This paper proposes the Optimized Assertiveness-Cost Evaluation (OACE) method, an approach based on Multiple Criteria Decision Making (MCDM) aimed at the simultaneous evaluation of these two criteria. OACE combines performance metrics through a parameterizable objective function, making it possible to adjust the relative weight between assertiveness and cost. To evaluate its effectiveness, the methodology was integrated with Random Walk, generating multiple training configurations with varying learning rates. The experiments were conducted on the CIFAR-10 dataset in three different scenarios: (i) prioritization of assertiveness, (ii) balance between accuracy and cost, and (iii) optimization for computational cost. The results demonstrate the potential of OACE to dynamically adapt model selection to different deployment environments: InceptionV3 stood out in scenario (i) by achieving maximum assertiveness, MobileNetV2 led scenario (ii) by achieving the best balance between assertiveness and cost, while EfficientNetB0 was superior in scenario (iii) by achieving maximum cost efficiency.
Funding Information
  • Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (001)
  • University of Para
  • National Council of Technological and Scientific Development

This publication has 50 references indexed in Scilit: