mapie.conformity_scores.RAPSConformityScore

class mapie.conformity_scores.RAPSConformityScore(size_raps: Optional[float] = 0.2)[source]

Regularized Adaptive Prediction Sets (RAPS) method-based non-conformity score. It uses the same technique as APSConformityScore class but with a penalty term to reduce the size of prediction sets. See [1] for more details. For now, this method only works with "prefit" and "split" strategies.

Parameters
size_raps: Optional[float]

Percentage of the data to be used for choosing lambda_star and k_star for the RAPS method.

References

[1] Anastasios Nikolas Angelopoulos, Stephen Bates, Michael Jordan and Jitendra Malik. “Uncertainty Sets for Image Classifiers using Conformal Prediction.” International Conference on Learning Representations 2021.

Attributes
classes: ArrayLike

Names of the classes.

random_state: Union[int, RandomState]

Pseudo random number generator state.

quantiles_: ArrayLike of shape (n_alpha)

The quantiles estimated from get_sets method.

label_encoder: LabelEncoder

The label encoder used to encode the labels.

size_raps: float

Percentage of the data to be used for choosing lambda_star and k_star for the RAPS method.

__init__(size_raps: Optional[float] = 0.2) → None[source]
get_conformity_score_quantiles(conformity_scores: ndarray[Any, dtype[_ScalarType_co]], alpha_np: ndarray[Any, dtype[_ScalarType_co]], estimator: EnsembleClassifier, agg_scores: Optional[str] = 'mean', include_last_label: Optional[Union[bool, str]] = True, **kwargs) → ndarray[Any, dtype[_ScalarType_co]][source]

Get the quantiles of the conformity scores for each uncertainty level.

Parameters
conformity_scores: NDArray of shape (n_samples,)

Conformity scores for each sample.

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, representing the uncertainty of the confidence interval.

estimator: EnsembleClassifier

Estimator that is fitted to predict y from X.

agg_scores: Optional[str]

Method to aggregate the scores from the base estimators. If “mean”, the scores are averaged. If “crossval”, the scores are obtained from cross-validation.

By default, "mean".

include_last_label: Optional[Union[bool, str]]

Whether or not to include last label in prediction sets. Choose among False, True or "randomized".

By default, True.

See the docstring of conformity_scores.sets.aps.APSConformityScore.get_prediction_sets() for more details.

X_raps: NDArray of shape (n_samples, n_features)

Observed feature values for the RAPS method (split data).

By default, “None” but must be set to work.

y_raps_no_enc: NDArray of shape (n_samples,)

Observed labels for the RAPS method (split data).

By default, “None” but must be set to work.

y_pred_proba_raps: NDArray of shape (n_samples, n_classes)

Predicted probabilities for the RAPS method (split data).

By default, “None” but must be set to work.

position_raps: NDArray of shape (n_samples,)

Position of the points in the split set for the RAPS method (split data). These positions are returned by the function get_true_label_position.

By default, “None” but must be set to work.

Returns
NDArray

Array of quantiles with respect to alpha_np.

get_conformity_scores(y: ndarray[Any, dtype[_ScalarType_co]], y_pred: ndarray[Any, dtype[_ScalarType_co]], y_enc: Optional[ndarray[Any, dtype[_ScalarType_co]]] = None, **kwargs) → ndarray[Any, dtype[_ScalarType_co]][source]

Get the conformity score.

Parameters
y: NDArray of shape (n_samples,)

Observed target values.

y_pred: NDArray of shape (n_samples,)

Predicted target values.

y_enc: NDArray of shape (n_samples,)

Target values as normalized encodings.

Returns
NDArray of shape (n_samples,)

Conformity scores.

set_external_attributes(*, label_encoder: Optional[LabelEncoder] = None, size_raps: Optional[float] = None, **kwargs) → None[source]

Set attributes that are not provided by the user.

Parameters
label_encoder: Optional[LabelEncoder]

The label encoder used to encode the labels.

By default None.

size_raps: Optional[float]

Percentage of the data to be used for choosing lambda_star and k_star for the RAPS method.

By default None.

split_data(X: ndarray[Any, dtype[_ScalarType_co]], y: ndarray[Any, dtype[_ScalarType_co]], y_enc: ndarray[Any, dtype[_ScalarType_co]], sample_weight: Optional[ndarray[Any, dtype[_ScalarType_co]]] = None, groups: Optional[ndarray[Any, dtype[_ScalarType_co]]] = None)[source]

Split data. Keeps part of the data for the calibration estimator (separate from the calibration data).

Parameters
X: ArrayLike

Observed values.

y: ArrayLike

Target values.

y_enc: ArrayLike

Target values as normalized encodings.

sample_weight: Optional[ArrayLike] of shape (n_samples,)

Non-null sample weights.

groups: Optional[ArrayLike] of shape (n_samples,)

Group labels for the samples used while splitting the dataset into train/test set. By default None.

Returns
Tuple[NDArray, NDArray, NDArray, NDArray, Optional[NDArray],
Optional[NDArray]]

  • X: NDArray of shape (n_samples, n_features)

  • y: NDArray of shape (n_samples,)

  • y_enc: NDArray of shape (n_samples,)

  • sample_weight: Optional[NDArray] of shape (n_samples,)

  • groups: Optional[NDArray] of shape (n_samples,)