mapie.metrics.spiegelhalter_p_value

mapie.metrics.spiegelhalter_p_value(y_true: ndarray[Any, dtype[_ScalarType_co]], y_score: ndarray[Any, dtype[_ScalarType_co]]) float[source]

Compute Spiegelhalter statistic p-value. Deduced from the corresponding statistic and CDF, which is no more than the normal distribution. It represents the probability of the observed statistic under the null hypothesis of perfect calibration.

Parameters
y_trueNDArray of shape (n_samples,)

An array of ground truth.

y_scoreNDArray of shape (n_samples,)

An array of scores.

Returns
float

The Spiegelhalter statistic p_value.

References

Spiegelhalter DJ. Probabilistic prediction in patient management and clinical trials. Statistics in medicine. 1986 Sep;5(5):421-33.

Examples

>>> import numpy as np
>>> from mapie.metrics import spiegelhalter_p_value
>>> y_true = np.array([1, 0, 1, 0, 1, 0])
>>> y_score = np.array([0.8, 0.3, 0.5, 0.5, 0.7, 0.1])
>>> sp_p_value = spiegelhalter_p_value(y_true, y_score)
>>> print(np.round(sp_p_value, 4))
0.8486

Examples using mapie.metrics.spiegelhalter_p_value