mapie.conformity_scores.LACConformityScore

class mapie.conformity_scores.LACConformityScore[source]

Least Ambiguous set-valued Classifier (LAC) method-based non conformity score (also formerly called "score").

It is based on the scores (i.e. 1 minus the softmax score of the true label) on the calibration set.

References

[1] Mauricio Sadinle, Jing Lei, and Larry Wasserman. “Least Ambiguous Set-Valued Classifiers with Bounded Error Levels.”, Journal of the American Statistical Association, 114, 2019.

Attributes
classes: Optional[ArrayLike]

Names of the classes.

random_state: Optional[Union[int, RandomState]]

Pseudo random number generator state.

quantiles_: ArrayLike of shape (n_alpha)

The quantiles estimated from get_sets method.

__init__() None[source]
get_conformity_score_quantiles(conformity_scores: ndarray[Any, dtype[_ScalarType_co]], alpha_np: ndarray[Any, dtype[_ScalarType_co]], estimator: EnsembleClassifier, agg_scores: Optional[str] = 'mean', **kwargs) ndarray[Any, dtype[_ScalarType_co]][source]

Get the quantiles of the conformity scores for each uncertainty level.

Parameters
conformity_scores: NDArray of shape (n_samples,)

Conformity scores for each sample.

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, representing the uncertainty of the confidence interval.

estimator: EnsembleClassifier

Estimator that is fitted to predict y from X.

agg_scores: Optional[str]

Method to aggregate the scores from the base estimators. If “mean”, the scores are averaged. If “crossval”, the scores are obtained from cross-validation.

By default "mean".

Returns
NDArray

Array of quantiles with respect to alpha_np.

get_conformity_scores(y: ndarray[Any, dtype[_ScalarType_co]], y_pred: ndarray[Any, dtype[_ScalarType_co]], y_enc: Optional[ndarray[Any, dtype[_ScalarType_co]]] = None, **kwargs) ndarray[Any, dtype[_ScalarType_co]][source]

Get the conformity score.

Parameters
y: NDArray of shape (n_samples,)

Observed target values.

y_pred: NDArray of shape (n_samples,)

Predicted target values.

y_enc: NDArray of shape (n_samples,)

Target values as normalized encodings.

Returns
NDArray of shape (n_samples,)

Conformity scores.

get_prediction_sets(y_pred_proba: ndarray[Any, dtype[_ScalarType_co]], conformity_scores: ndarray[Any, dtype[_ScalarType_co]], alpha_np: ndarray[Any, dtype[_ScalarType_co]], estimator: EnsembleClassifier, agg_scores: Optional[str] = 'mean', **kwargs) ndarray[Any, dtype[_ScalarType_co]][source]

Generate prediction sets based on the probability predictions, the conformity scores and the uncertainty level.

Parameters
y_pred_proba: NDArray of shape (n_samples, n_classes)

Target prediction.

conformity_scores: NDArray of shape (n_samples,)

Conformity scores for each sample.

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, representing the uncertainty of the confidence interval.

estimator: EnsembleClassifier

Estimator that is fitted to predict y from X.

agg_scores: Optional[str]

Method to aggregate the scores from the base estimators. If “mean”, the scores are averaged. If “crossval”, the scores are obtained from cross-validation.

By default "mean".

Returns
NDArray

Array of quantiles with respect to alpha_np.

get_predictions(X: ndarray[Any, dtype[_ScalarType_co]], alpha_np: ndarray[Any, dtype[_ScalarType_co]], estimator: EnsembleClassifier, agg_scores: Optional[str] = 'mean', **kwargs) ndarray[Any, dtype[_ScalarType_co]][source]

Get predictions from an EnsembleClassifier.

Parameters
X: NDArray of shape (n_samples, n_features)

Observed feature values.

alpha_np: NDArray of shape (n_alpha,)

NDArray of floats between 0 and 1, represents the uncertainty of the confidence interval.

estimator: EnsembleClassifier

Estimator that is fitted to predict y from X.

agg_scores: Optional[str]

Method to aggregate the scores from the base estimators. If “mean”, the scores are averaged. If “crossval”, the scores are obtained from cross-validation.

By default "mean".

Returns
NDArray

Array of predictions.