mapie.conformity_scores.TopKConformityScore

class mapie.conformity_scores.TopKConformityScore

Top-K method-based non-conformity score.

It is based on the sorted index of the probability of the true label in the softmax outputs, on the conformalization set. In case two probabilities are equal, both are taken, thus, the size of some prediction sets may be different from the others.

Attributes:

classes: Optional[ArrayLike]

Names of the classes.

random_state: Optional[Union[int, np.random.RandomState]]

Pseudo random number generator state.

quantiles_: ArrayLike of shape (n_alpha)

The quantiles estimated from get_sets method.

References

[1] Anastasios Nikolas Angelopoulos, Stephen Bates, Michael Jordan and Jitendra Malik. “Uncertainty Sets for Image Classifiers using Conformal Prediction.” International Conference on Learning Representations 2021.

__init__() → None

get_conformity_score_quantiles(conformity_scores: ndarray[tuple[Any, ...], dtype[_ScalarT]], alpha_np: ndarray[tuple[Any, ...], dtype[_ScalarT]], cv: int | str | BaseCrossValidator | None, **kwargs) → ndarray[tuple[Any, ...], dtype[_ScalarT]]

Get the quantiles of the conformity scores for each uncertainty level.

Parameters:

conformity_scores: NDArray of shape (n_samples,)

Conformity scores for each sample.

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, representing the uncertainty of the confidence interval.

cv: Optional[Union[int, str, BaseCrossValidator]]

Cross-validation strategy used by the estimator (not used here).

Returns:

NDArray

Array of quantiles with respect to alpha_np.

get_conformity_scores(y: ndarray[tuple[Any, ...], dtype[_ScalarT]], y_pred: ndarray[tuple[Any, ...], dtype[_ScalarT]], y_enc: ndarray[tuple[Any, ...], dtype[_ScalarT]] | None = None, **kwargs) → ndarray[tuple[Any, ...], dtype[_ScalarT]]

Get the conformity score.

Parameters:

y: NDArray of shape (n_samples,)

Observed target values.

y_pred: NDArray of shape (n_samples,)

Predicted target values.

y_enc: NDArray of shape (n_samples,)

Target values as normalized encodings.

Returns:

NDArray of shape (n_samples,)

Conformity scores.

get_prediction_sets(y_pred_proba: ndarray[tuple[Any, ...], dtype[_ScalarT]], conformity_scores: ndarray[tuple[Any, ...], dtype[_ScalarT]], alpha_np: ndarray[tuple[Any, ...], dtype[_ScalarT]], cv: int | str | BaseCrossValidator | None, **kwargs) → ndarray[tuple[Any, ...], dtype[_ScalarT]]

Generate prediction sets based on the probability predictions, the conformity scores and the uncertainty level.

Parameters:

y_pred_proba: NDArray of shape (n_samples, n_classes)

Target prediction.

conformity_scores: NDArray of shape (n_samples,)

Conformity scores for each sample (not used here).

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, representing the uncertainty of the confidence interval (not used here).

cv: Optional[Union[int, str, BaseCrossValidator]]

Cross-validation strategy used by the estimator (not used here).

Returns:

NDArray

Array of quantiles with respect to alpha_np.

get_predictions(X: ndarray[tuple[Any, ...], dtype[_ScalarT]], alpha_np: ndarray[tuple[Any, ...], dtype[_ScalarT]], y_pred_proba: ndarray[tuple[Any, ...], dtype[_ScalarT]], cv: int | str | BaseCrossValidator | None, **kwargs) → ndarray[tuple[Any, ...], dtype[_ScalarT]]

Just processes the passed y_pred_proba.

Parameters:

X: NDArray of shape (n_samples, n_features)

Observed feature values (not used since predictions are passed).

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, represents the uncertainty of the confidence interval.

y_pred_proba: NDArray

Predicted probabilities from the estimator.

cv: Optional[Union[int, str, BaseCrossValidator]]

Cross-validation strategy used by the estimator (not used here).

Returns:

NDArray

Array of predictions.