mapie.metrics
.kolmogorov_smirnov_statistic¶
- mapie.metrics.kolmogorov_smirnov_statistic(y_true: ndarray[Any, dtype[_ScalarType_co]], y_score: ndarray[Any, dtype[_ScalarType_co]]) float [source]¶
Compute Kolmogorov-smirnov’s statistic for calibration test. Also called ECCE-MAD (Estimated Cumulative Calibration Errors - Maximum Absolute Deviation). The closer to zero, the better the scores are calibrated. Indeed, if the scores are perfectly calibrated, the cumulative differences between
y_true
andy_score
should share the same properties of a standard Brownian motion asymptotically.- Parameters
- y_trueNDArray of shape (n_samples,)
An array of ground truth.
- y_scoreNDArray of shape (n_samples,)
An array of scores..
- Returns
- float
Kolmogorov-smirnov’s statistic.
References
Arrieta-Ibarra I, Gujral P, Tannen J, Tygert M, Xu C. Metrics of calibration for probabilistic predictions. The Journal of Machine Learning Research. 2022 Jan 1;23(1):15886-940.