mapie.metrics.kuiper_statistic

mapie.metrics.kuiper_statistic(y_true: ndarray[Any, dtype[_ScalarType_co]], y_score: ndarray[Any, dtype[_ScalarType_co]]) float[source]

Compute Kuiper’s statistic for calibration test. Also called ECCE-R (Estimated Cumulative Calibration Errors - Range). The closer to zero, the better the scores are calibrated. Indeed, if the scores are perfectly calibrated, the cumulative differences between y_true and y_score should share the same properties of a standard Brownian motion asymptotically.

Parameters
y_trueNDArray of shape (n_samples,)

An array of ground truth.

y_scoreNDArray of shape (n_samples,)

An array of scores.

Returns
float

Kuiper’s statistic.

References

Arrieta-Ibarra I, Gujral P, Tannen J, Tygert M, Xu C. Metrics of calibration for probabilistic predictions. The Journal of Machine Learning Research. 2022 Jan 1;23(1):15886-940.