mapie.conformity_scores
.LACConformityScore¶
- class mapie.conformity_scores.LACConformityScore[source]¶
Least Ambiguous set-valued Classifier (LAC) method-based non conformity score (also formerly called
"score"
).It is based on the scores (i.e. 1 minus the softmax score of the true label) on the calibration set.
References
[1] Mauricio Sadinle, Jing Lei, and Larry Wasserman. “Least Ambiguous Set-Valued Classifiers with Bounded Error Levels.”, Journal of the American Statistical Association, 114, 2019.
- Attributes
- classes: Optional[ArrayLike]
Names of the classes.
- random_state: Optional[Union[int, RandomState]]
Pseudo random number generator state.
- quantiles_: ArrayLike of shape (n_alpha)
The quantiles estimated from
get_sets
method.
- get_conformity_score_quantiles(conformity_scores: ndarray[Any, dtype[_ScalarType_co]], alpha_np: ndarray[Any, dtype[_ScalarType_co]], estimator: EnsembleClassifier, agg_scores: Optional[str] = 'mean', **kwargs) ndarray[Any, dtype[_ScalarType_co]] [source]¶
Get the quantiles of the conformity scores for each uncertainty level.
- Parameters
- conformity_scores: NDArray of shape (n_samples,)
Conformity scores for each sample.
- alpha_np: NDArray of shape (n_alpha,)
NDArray of floats between 0 and 1, representing the uncertainty of the confidence interval.
- estimator: EnsembleClassifier
Estimator that is fitted to predict y from X.
- agg_scores: Optional[str]
Method to aggregate the scores from the base estimators. If “mean”, the scores are averaged. If “crossval”, the scores are obtained from cross-validation.
By default
"mean"
.
- Returns
- NDArray
Array of quantiles with respect to alpha_np.
- get_conformity_scores(y: ndarray[Any, dtype[_ScalarType_co]], y_pred: ndarray[Any, dtype[_ScalarType_co]], y_enc: Optional[ndarray[Any, dtype[_ScalarType_co]]] = None, **kwargs) ndarray[Any, dtype[_ScalarType_co]] [source]¶
Get the conformity score.
- Parameters
- y: NDArray of shape (n_samples,)
Observed target values.
- y_pred: NDArray of shape (n_samples,)
Predicted target values.
- y_enc: NDArray of shape (n_samples,)
Target values as normalized encodings.
- Returns
- NDArray of shape (n_samples,)
Conformity scores.
- get_prediction_sets(y_pred_proba: ndarray[Any, dtype[_ScalarType_co]], conformity_scores: ndarray[Any, dtype[_ScalarType_co]], alpha_np: ndarray[Any, dtype[_ScalarType_co]], estimator: EnsembleClassifier, agg_scores: Optional[str] = 'mean', **kwargs) ndarray[Any, dtype[_ScalarType_co]] [source]¶
Generate prediction sets based on the probability predictions, the conformity scores and the uncertainty level.
- Parameters
- y_pred_proba: NDArray of shape (n_samples, n_classes)
Target prediction.
- conformity_scores: NDArray of shape (n_samples,)
Conformity scores for each sample.
- alpha_np: NDArray of shape (n_alpha,)
NDArray of floats between 0 and 1, representing the uncertainty of the confidence interval.
- estimator: EnsembleClassifier
Estimator that is fitted to predict y from X.
- agg_scores: Optional[str]
Method to aggregate the scores from the base estimators. If “mean”, the scores are averaged. If “crossval”, the scores are obtained from cross-validation.
By default
"mean"
.
- Returns
- NDArray
Array of quantiles with respect to alpha_np.
- get_predictions(X: ndarray[Any, dtype[_ScalarType_co]], alpha_np: ndarray[Any, dtype[_ScalarType_co]], estimator: EnsembleClassifier, agg_scores: Optional[str] = 'mean', **kwargs) ndarray[Any, dtype[_ScalarType_co]] [source]¶
Get predictions from an EnsembleClassifier.
- Parameters
- X: NDArray of shape (n_samples, n_features)
Observed feature values.
- alpha_np: NDArray of shape (n_alpha,)
NDArray of floats between
0
and1
, represents the uncertainty of the confidence interval.- estimator: EnsembleClassifier
Estimator that is fitted to predict y from X.
- agg_scores: Optional[str]
Method to aggregate the scores from the base estimators. If “mean”, the scores are averaged. If “crossval”, the scores are obtained from cross-validation.
By default
"mean"
.
- Returns
- NDArray
Array of predictions.