mapie.metrics.classification_ssc_score

mapie.metrics.classification_ssc_score(y_true: ndarray[Any, dtype[_ScalarType_co]], y_pred_set: ndarray[Any, dtype[_ScalarType_co]], num_bins: Optional[int] = None) → ndarray[Any, dtype[_ScalarType_co]]

Aggregate by the minimum for each alpha the Size-Stratified Coverage [3]: returns the maximum violation of the conditional coverage (with the groups defined).

Parameters

y_true: NDArray of shape (n_samples,)

True labels.

y_pred_set: NDArray of shape (n_samples, n_class, n_alpha)

or (n_samples, n_class)

Prediction sets given by booleans of labels.

num_bins: int or None

Number of groups. If None, one value of coverage by possible size of sets (n_classes +1) is computed. Should be less than the number of different set sizes.

Returns

NDArray of shape (n_alpha,)

Examples

>>> from mapie.metrics import classification_ssc_score
>>> import numpy as np
>>> y_true = y_true_class = np.array([3, 3, 1, 2, 2])
>>> y_pred_set = np.array([
...    [True, True, True, True],
...    [False, True, False, True],
...    [True, True, True, False],
...    [False, False, True, True],
...    [True, True, False, True]])
>>> print(classification_ssc_score(y_true, y_pred_set, num_bins=2))
[0.66666667]