mapie.metrics.classification_coverage_score

mapie.metrics.classification_coverage_score(y_true: Union[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]], numpy._typing._nested_sequence._NestedSequence[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]]], bool, int, float, complex, str, bytes, numpy._typing._nested_sequence._NestedSequence[Union[bool, int, float, complex, str, bytes]]], y_pred_set: Union[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]], numpy._typing._nested_sequence._NestedSequence[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]]], bool, int, float, complex, str, bytes, numpy._typing._nested_sequence._NestedSequence[Union[bool, int, float, complex, str, bytes]]]) → float

Effective coverage score obtained by the prediction sets.

The effective coverage is obtained by estimating the fraction of true labels that lie within the prediction sets.

Parameters

y_true: ArrayLike of shape (n_samples,)

True labels.

y_pred_set: ArrayLike of shape (n_samples, n_class)

Prediction sets given by booleans of labels.

Returns

float

Effective coverage obtained by the prediction sets.

Examples

>>> from mapie.metrics import classification_coverage_score
>>> import numpy as np
>>> y_true = np.array([3, 3, 1, 2, 2])
>>> y_pred_set = np.array([
...     [False, False,  True,  True],
...     [False,  True, False,  True],
...     [False,  True,  True, False],
...     [False, False,  True,  True],
...     [False,  True, False,  True]
... ])
>>> print(classification_coverage_score(y_true, y_pred_set))
0.8

Examples using mapie.metrics.classification_coverage_score

Comparing prediction sets on a two-dimensional dataset

Tutorial for classification

Tutorial for set prediction

Cross-conformal for classification