Calzone: A Python package for measuring calibration of probabilistic models for classification
Open Access
- 24 October 2025
- journal article
- Published by The Open Journal in Journal of Open Source Software
- Vol. 10 (114) , 8026
Abstract
Calzone is a Python package for evaluating the calibration of probabilistic outputs of classifier models. It provides a set of functions for visualizing calibration and computing calibration metrics given a representative dataset with the model's predictions and the true class labels. The metrics provided in Calzone include: expected calibration error (ECE), maximum calibration error (MCE), Hosmer-Lemeshow (HL) test, integrated calibration index (ICI), Spiegelhalter's Z-statistic and Cox's calibration slope/intercept. The package is designed with versatility in mind. Many metrics allow users to adjust binning schemes and choose between top-class or class-wise calculations.Keywords
This publication has 22 references indexed in Scilit:
- The Integrated Calibration Index (ICI) and related metrics for quantifying the calibration of logistic regression modelsStatistics in Medicine, 2019
- Calibration of Prognostic Risk Scores2018
- Applied Logistic Regression2013
- Reliability, sufficiency, and the decomposition of proper scoresQuarterly Journal of the Royal Meteorological Society, 2009
- Prevalence ScalingAcademic Radiology, 2008
- The Elements of Statistical Learning2001
- What price perfection? Calibration and discrimination of clinical prediction modelsJournal of Clinical Epidemiology, 1992
- Probabilistic prediction in patient management and clinical trialsStatistics in Medicine, 1986
- Goodness of fit tests for the multiple logistic regression modelCommunications in Statistics - Theory and Methods, 1980
- Two further applications of a model for binary regressionBiometrika, 1958