Aestimo: a feedback-directed optimization evaluation tool

Authors:
  • PBP. Berube
    P. Berube
  • JAJ.N. Amaral
    J.N. Amaral

Abstract
Published studies that use feedback-directed optimiza- tion (FDO) techniques use either a single input for both training and performance evaluation, or a single input for training and a single input for evaluation. Thus an impor- tant question is if the FDO results published in the literature are sensitive to the training and testing input selection. Aestimo is a new evaluation tool that uses a workload of inputs to evaluate the sensitivity of specific code transfor- mations to the choice of inputs in the training and testing phases. Aestimo uses optimization logs to isolate the effects of individual code transformations. It incorporates metrics to determine the effect of training input selection on indi- vidual compiler decisions. Besides describing the structure of Aestimo, this paper presents a case study that uses SPEC CINT2000 benchmark programs with the Open Research Compiler (ORC) to in- vestigate the effect of training/testing input selection on in- lining and if-conversion. The experimental results indicate that: (1) training input selection affects the compiler de- cisions made for these code transformation; (2) the choice of training/testing inputs can have a significant impact on measured performance.