MAPIE API

Regression

regression.MapieRegressor([estimator, ...])	Prediction interval with out-of-fold conformity scores.
regression.MapieQuantileRegressor([...])	This class implements the conformalized quantile regression strategy as proposed by Romano et al. (2019) to make conformal predictions.
regression.MapieTimeSeriesRegressor([...])	Prediction intervals with out-of-fold residuals for time series.

Classification

classification.MapieClassifier([estimator, ...])

Prediction sets for classification.

Multi-Label Classification

multi_label_classification.MapieMultiLabelClassifier([...])

Prediction sets for multilabel-classification.

Calibration

calibration.MapieCalibrator([estimator, ...])

Calibration for multi-class problems.

Metrics

metrics.regression_coverage_score(y_true, ...)	Effective coverage score obtained by the prediction intervals.
metrics.classification_coverage_score(...)	Effective coverage score obtained by the prediction sets.
metrics.regression_mean_width_score(...)	Effective mean width score obtained by the prediction intervals.
metrics.classification_mean_width_score(...)	Mean width of prediction set output by MapieClassifier.
metrics.expected_calibration_error(y_true, ...)	The expected calibration error, which is the difference between the confidence scores and accuracy per bin [1].
metrics.top_label_ece(y_true, y_scores[, ...])	The Top-Label ECE which is a method adapted to fit the ECE to a Top-Label setting [2].
metrics.regression_coverage_score_v2(y_true, ...)	Effective coverage score obtained by the prediction intervals.
metrics.classification_coverage_score_v2(...)	Effective coverage score obtained by the prediction sets.
metrics.regression_ssc(y_true, y_intervals)	Compute Size-Stratified Coverage metrics proposed in [3] that is the conditional coverage conditioned by the size of the intervals.
metrics.regression_ssc_score(y_true, y_intervals)	Aggregate by the minimum for each alpha the Size-Stratified Coverage [3]: returns the maximum violation of the conditional coverage (with the groups defined).
metrics.classification_ssc(y_true, y_pred_set)	Compute Size-Stratified Coverage metrics proposed in [3] that is the conditional coverage conditioned by the size of the predictions sets.
metrics.classification_ssc_score(y_true, ...)	Aggregate by the minimum for each alpha the Size-Stratified Coverage [3]: returns the maximum violation of the conditional coverage (with the groups defined).
metrics.hsic(y_true, y_intervals[, kernel_sizes])	Compute the square root of the hsic coefficient.

Conformity scores

conformity_scores.AbsoluteConformityScore()	Absolute conformity score.
conformity_scores.GammaConformityScore()	Gamma conformity score.

Resampling

subsample.BlockBootstrap([n_resamplings, ...])	Generate a sampling method, that block bootstraps the training set.
subsample.Subsample([n_resamplings, ...])	Generate a sampling method, that resamples the training set with possible bootstraps.