mapie.metrics.kolmogorov_smirnov_p_value

mapie.metrics.kolmogorov_smirnov_p_value(y_true: numpy.ndarray[Any, numpy.dtype[numpy._typing._array_like._ScalarType_co]], y_score: numpy.ndarray[Any, numpy.dtype[numpy._typing._array_like._ScalarType_co]]) → float

Compute Kolmogorov Smirnov p-value. Deduced from the corresponding statistic and CDF. It represents the probability of the observed statistic under the null hypothesis of perfect calibration.

Parameters

y_trueNDArray of shape (n_samples,)

An array of ground truth.

y_scoreNDArray of shape (n_samples,)

An array of scores.

Returns

float

The Kolmogorov Smirnov p-value.

References

Tygert M. Calibration of P-values for calibration and for deviation of a subpopulation from the full population. arXiv preprint arXiv:2202.00100. 2022 Jan 31.

D. A. Darling. A. J. F. Siegert. The First Passage Problem for a Continuous Markov Process. Ann. Math. Statist. 24 (4) 624 - 639, December, 1953.

Examples using mapie.metrics.kolmogorov_smirnov_p_value

Testing for calibration in binary classification settings

Evaluating the asymptotic convergence of p-values