mapie.conformity_scores.BaseClassificationScore

class mapie.conformity_scores.BaseClassificationScore

Base conformity score class for classification task.

This class should not be used directly. Use derived classes instead.

Attributes:

classes: Optional[ArrayLike]

Names of the classes.

random_state: Optional[Union[int, np.random.RandomState]]

Pseudo random number generator state.

quantiles_: ArrayLike of shape (n_alpha)

The quantiles estimated from get_sets method.

__init__() → None

abstract get_conformity_score_quantiles(conformity_scores: ndarray[tuple[Any, ...], dtype[_ScalarT]], alpha_np: ndarray[tuple[Any, ...], dtype[_ScalarT]], cv: int | str | BaseCrossValidator | None, **kwargs) → ndarray[tuple[Any, ...], dtype[_ScalarT]]

Abstract method to get quantiles of the conformity scores.

This method should be implemented by any subclass of the current class.

Parameters:

conformity_scores: NDArray of shape (n_samples,)

Conformity scores for each sample.

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, representing the uncertainty of the confidence set.

cv: Optional[Union[int, str, BaseCrossValidator]]

Cross-validation strategy used by the estimator.

Returns:

NDArray

Array of quantiles with respect to alpha_np.

abstract get_prediction_sets(y_pred_proba: ndarray[tuple[Any, ...], dtype[_ScalarT]], conformity_scores: ndarray[tuple[Any, ...], dtype[_ScalarT]], alpha_np: ndarray[tuple[Any, ...], dtype[_ScalarT]], cv: int | str | BaseCrossValidator | None, **kwargs) → ndarray[tuple[Any, ...], dtype[_ScalarT]]

Abstract method to generate prediction sets based on the probability predictions, the conformity scores and the uncertainty level.

This method should be implemented by any subclass of the current class.

Parameters:

y_pred_proba: NDArray of shape (n_samples, n_classes)

Target prediction.

conformity_scores: NDArray of shape (n_samples,)

Conformity scores for each sample.

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, representing the uncertainty of the confidence set.

cv: Optional[Union[int, str, BaseCrossValidator]]

Cross-validation strategy used by the estimator.

Returns:

NDArray

Array of quantiles with respect to alpha_np.

abstract get_predictions(X: ndarray[tuple[Any, ...], dtype[_ScalarT]], alpha_np: ndarray[tuple[Any, ...], dtype[_ScalarT]], y_pred_proba: ndarray[tuple[Any, ...], dtype[_ScalarT]], cv: int | str | BaseCrossValidator | None, **kwargs) → ndarray[tuple[Any, ...], dtype[_ScalarT]]

Abstract method to get predictions from an EnsembleClassifier.

This method should be implemented by any subclass of the current class.

Parameters:

X: NDArray of shape (n_samples, n_features)

Observed feature values.

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, represents the uncertainty of the confidence set.

y_pred_proba: NDArray

Predicted probabilities from the estimator.

cv: Optional[Union[int, str, BaseCrossValidator]]

Cross-validation strategy used by the estimator.

Returns:

NDArray

Array of predictions.

get_sets(X: ndarray[tuple[Any, ...], dtype[_ScalarT]], alpha_np: ndarray[tuple[Any, ...], dtype[_ScalarT]], y_pred_proba: ndarray[tuple[Any, ...], dtype[_ScalarT]], cv: int | str | BaseCrossValidator | None, conformity_scores: ndarray[tuple[Any, ...], dtype[_ScalarT]], **kwargs) → ndarray[tuple[Any, ...], dtype[_ScalarT]]

Compute classes of the prediction sets from the observed values, the predicted probabilities and the conformity scores.

Parameters:

X: NDArray of shape (n_samples, n_features)

Observed feature values.

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, representing the uncertainty of the confidence set.

y_pred_proba: NDArray

Predicted probabilities from the estimator.

cv: Optional[Union[int, str, BaseCrossValidator]]

Cross-validation strategy used by the estimator.

conformity_scores: NDArray of shape (n_samples,)

Conformity scores.

Returns:

NDArray of shape (n_samples, n_classes, n_alpha)

Prediction sets (Booleans indicate whether classes are included).

predict_set(X: ndarray[tuple[Any, ...], dtype[_ScalarT]], alpha_np: ndarray[tuple[Any, ...], dtype[_ScalarT]], **kwargs)

Compute the prediction sets on new samples based on the uncertainty of the target confidence set.

Parameters:

X: NDArray of shape (n_samples,)

The input data or samples for prediction.

alpha_np: NDArray of shape (n_alpha, )

Represents the uncertainty of the confidence set to produce.

**kwargs: dict

Additional keyword arguments.

Returns:

result

The prediction sets for each sample and each alpha level. The output structure depends on the get_sets method.

set_external_attributes(*, classes: ArrayLike | None = None, random_state: int | RandomState | None = None, **kwargs) → None

Set attributes that are not provided by the user.

Parameters:

classes: Optional[ArrayLike]

Names of the classes.

By default None.

random_state: Optional[Union[int, np.random.RandomState]]

Pseudo random number generator state.