mapie.conformity_scores.RAPSConformityScore

class mapie.conformity_scores.RAPSConformityScore(size_raps: float | None = 0.2)

Regularized Adaptive Prediction Sets (RAPS) method-based non-conformity score. It uses the same technique as APSConformityScore class but with a penalty term to reduce the size of prediction sets. See [1] for more details. For now, this method only works with “prefit” and “split” strategies.

Parameters:

size_raps: Optional[float]

Percentage of the data to be used for choosing lambda_star and k_star for the RAPS method.

Attributes:

classes: ArrayLike

Names of the classes.

random_state: Union[int, np.random.RandomState]

Pseudo random number generator state.

quantiles_: ArrayLike of shape (n_alpha)

The quantiles estimated from get_sets method.

label_encoder: LabelEncoder

The label encoder used to encode the labels.

size_raps: float

Percentage of the data to be used for choosing lambda_star and k_star for the RAPS method.

References

[1] Anastasios Nikolas Angelopoulos, Stephen Bates, Michael Jordan and Jitendra Malik. “Uncertainty Sets for Image Classifiers using Conformal Prediction.” International Conference on Learning Representations 2021.

__init__(size_raps: float | None = 0.2) → None

get_conformity_score_quantiles(conformity_scores: ndarray[tuple[Any, ...], dtype[_ScalarT]], alpha_np: ndarray[tuple[Any, ...], dtype[_ScalarT]], cv: int | str | BaseCrossValidator | None, agg_scores: str | None = 'mean', include_last_label: bool | str | None = True, **kwargs) → ndarray[tuple[Any, ...], dtype[_ScalarT]]

Get the quantiles of the conformity scores for each uncertainty level.

Parameters:

conformity_scores: NDArray of shape (n_samples,)

Conformity scores for each sample.

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, representing the uncertainty of the confidence interval.

cv: Optional[Union[int, str, BaseCrossValidator]]

Cross-validation strategy used by the estimator (not used here).

agg_scores: Optional[str]

Method to aggregate the scores from the base estimators. If “mean”, the scores are averaged. If “crossval”, the scores are obtained from cross-validation (not used here).

By default, “mean”.

include_last_label: Optional[Union[bool, str]]

Whether or not to include last label in prediction sets. Choose among False, True or “randomized”.

By default, True.

See the docstring of APSConformityScore.get_prediction_sets for more details.

Returns:

NDArray

Array of quantiles with respect to alpha_np.

get_conformity_scores(y: ndarray[tuple[Any, ...], dtype[_ScalarT]], y_pred: ndarray[tuple[Any, ...], dtype[_ScalarT]], y_enc: ndarray[tuple[Any, ...], dtype[_ScalarT]] | None = None, **kwargs) → ndarray[tuple[Any, ...], dtype[_ScalarT]]

Get the conformity score.

Parameters:

y: NDArray of shape (n_samples,)

Observed target values.

y_pred: NDArray of shape (n_samples,)

Predicted target values.

y_enc: Optional[NDArray] of shape (n_samples,)

Target values as normalized encodings.

Returns:

NDArray of shape (n_samples,)

Conformity scores.

set_external_attributes(*, label_encoder: LabelEncoder | None = None, size_raps: float | None = None, **kwargs) → None

Set attributes that are not provided by the user.

Parameters:

label_encoder: Optional[LabelEncoder]

The label encoder used to encode the labels.

By default None.

size_raps: Optional[float]

Percentage of the data to be used for choosing lambda_star and k_star for the RAPS method.

By default None.

split_data(X: ndarray[tuple[Any, ...], dtype[_ScalarT]], y: ndarray[tuple[Any, ...], dtype[_ScalarT]], y_enc: ndarray[tuple[Any, ...], dtype[_ScalarT]], sample_weight: ndarray[tuple[Any, ...], dtype[_ScalarT]] | None = None, groups: ndarray[tuple[Any, ...], dtype[_ScalarT]] | None = None)

Split data. Keeps part of the data for the calibration estimator (separate from the calibration data).

Parameters:

X: NDArray

Observed values.

y: NDArray

Target values.

y_enc: NDArray

Target values as normalized encodings.

sample_weight: Optional[NDArray] of shape (n_samples,)

Non-null sample weights.

groups: Optional[NDArray] of shape (n_samples,)

Group labels for the samples used while splitting the dataset into train/test set. By default None.

Returns:

Tuple[NDArray, NDArray, NDArray, NDArray, Optional[NDArray],

Optional[NDArray]]

• X: NDArray of shape (n_samples, n_features)

• y: NDArray of shape (n_samples,)

• y_enc: NDArray of shape (n_samples,)

• sample_weight: Optional[NDArray] of shape (n_samples,)

• groups: Optional[NDArray] of shape (n_samples,)