mapie.metrics.spiegelhalter_statistic

mapie.metrics.spiegelhalter_statistic(y_true: ndarray[Any, dtype[_ScalarType_co]], y_score: ndarray[Any, dtype[_ScalarType_co]]) → float

Compute Spiegelhalter’s statistic for calibration test. The closer to zero, the better the scores are calibrated. Indeed, if the scores are perfectly calibrated, the Brier score simplifies to an expression whose expectancy and variance are easy to compute. The statistic is no more that a z-score on this normalized expression.

Parameters

y_trueNDArray of shape (n_samples,)

An array of ground truth.

y_scoreNDArray of shape (n_samples,)

An array of scores.

Returns

float

Spiegelhalter’s statistic.

References

Spiegelhalter DJ. Probabilistic prediction in patient management and clinical trials. Statistics in medicine. 1986 Sep;5(5):421-33.

Examples

>>> import numpy as np
>>> from mapie.metrics import spiegelhalter_statistic
>>> y_true = np.array([0, 1, 0, 1, 0])
>>> y_score = np.array([0.1, 0.9, 0.21, 0.9, 0.5])
>>> print(np.round(spiegelhalter_statistic(y_true, y_score), 3))
-0.757