Abstract
Calzone is a Python package for evaluating the calibration of probabilistic outputs of classifier models. It provides a set of functions for visualizing calibration and computing calibration metrics given a representative dataset with the model's predictions and the true class labels. The metrics provided in Calzone include: expected calibration error (ECE), maximum calibration error (MCE), Hosmer-Lemeshow (HL) test, integrated calibration index (ICI), Spiegelhalter's Z-statistic and Cox's calibration slope/intercept. The package is designed with versatility in mind. Many metrics allow users to adjust binning schemes and choose between top-class or class-wise calculations.

This publication has 22 references indexed in Scilit: