mapie.metrics.regression_ssc

mapie.metrics.regression_ssc(y_true: ndarray[Any, dtype[_ScalarType_co]], y_intervals: ndarray[Any, dtype[_ScalarType_co]], num_bins: int = 3) → ndarray[Any, dtype[_ScalarType_co]]

Compute Size-Stratified Coverage metrics proposed in [3] that is the conditional coverage conditioned by the size of the intervals. The intervals are ranked by their size (ascending) and then divided into num_bins groups: one value of coverage by groups is computed.

Warning: This metric should be used only with non constant intervals (intervals of different sizes), with constant intervals the result may be misinterpreted.

[3] Angelopoulos, A. N., & Bates, S. (2021). A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511.

Parameters

y_true: NDArray of shape (n_samples,)

True labels.

y_intervals: NDArray of shape (n_samples, 2, n_alpha) or (n_samples, 2)

Prediction intervals given by booleans of labels.

num_bins: int n

Number of groups. Should be less than the number of different interval widths.

Returns

NDArray of shape (n_alpha, num_bins)

Examples

>>> from mapie.metrics import regression_ssc
>>> import numpy as np
>>> y_true = np.array([5, 7.5, 9.5])
>>> y_intervals = np.array([
... [4, 6],
... [6.0, 9.0],
... [9, 10.0]
... ])
>>> print(regression_ssc(y_true, y_intervals, num_bins=2))
[[1. 1.]]

Examples using mapie.metrics.regression_ssc

Estimating conditional coverage