Simulating mismatch between calibration and target population in AI for mammography the retrospective VAIB study

Authors:
Open Access
Publisher website
Google Scholar
Abstract
AI cancer detection models require calibration to attain the desired balance between cancer detection rate (CDR) and false positive rate. In this study, we simulate the impact of six types of mismatches between the calibration population and the clinical target population, by creating purposefully non-representative datasets to calibrate AI for clinical settings. Mismatching the acquisition year between healthy and cancer-diagnosed screening participants led to a distortion in CDR between −3% to +19%. Mismatching age led to a distortion in CDR between −0.2% to +27%. Mismatching breast density distribution led to a distortion in CDR between +1% to 16%. Mismatching mammography vendors lead to a distortion in CDR between −32% to + 33%. Mismatches between calibration population and target clinical population lead to clinically important deviations. It is vital for safe clinical AI integration to ensure that important aspects of the calibration population are representative of the target population.
Funding Information
  • Swedish Innovation Agency VINNOVA
  • Regional Cancer Centers in Collaboration
  • Region Stockholm
  • Swedish Cancer Society
  • Swedish Research Council
  • Emil and Wera Cornell foundation
  • Lillian Sagens and Curt Ericssons Foundation

This publication has 38 references indexed in Scilit: