MAPIE API

Regression

regression.MapieRegressor([estimator, ...])	Prediction interval with out-of-fold conformity scores.
regression.MapieQuantileRegressor([...])	This class implements the conformalized quantile regression strategy as proposed by Romano et al. (2019) to make conformal predictions.
regression.MapieTimeSeriesRegressor([...])	Prediction intervals with out-of-fold residuals for time series.

Classification

classification.MapieClassifier([estimator, ...])

Prediction sets for classification.

Multi-Label Classification

multi_label_classification.MapieMultiLabelClassifier([...])

Prediction sets for multilabel-classification.

Calibration

calibration.MapieCalibrator([estimator, ...])

Calibration for multi-class problems.

Metrics

metrics.classification_coverage_score(...)	Effective coverage score obtained by the prediction sets.
metrics.classification_coverage_score_v2(...)	Effective coverage score obtained by the prediction sets.
metrics.classification_mean_width_score(...)	Mean width of prediction set output by MapieClassifier.
metrics.classification_ssc(y_true, y_pred_set)	Compute Size-Stratified Coverage metrics proposed in [3] that is the conditional coverage conditioned by the size of the predictions sets.
metrics.classification_ssc_score(y_true, ...)	Aggregate by the minimum for each alpha the Size-Stratified Coverage [3]: returns the maximum violation of the conditional coverage (with the groups defined).
metrics.cumulative_differences(y_true, y_score)	Compute the cumulative difference between y_true and y_score, both ordered according to y_scores array.
metrics.expected_calibration_error(y_true, ...)	The expected calibration error, which is the difference between the confidence scores and accuracy per bin [1].
metrics.hsic(y_true, y_intervals[, kernel_sizes])	Compute the square root of the hsic coefficient.
metrics.kolmogorov_smirnov_cdf(x)	Compute the Kolmogorov-smirnov cumulative distribution function (CDF) for the float x.
metrics.kolmogorov_smirnov_p_value(y_true, ...)	Compute Kolmogorov Smirnov p-value.
metrics.kolmogorov_smirnov_statistic(y_true, ...)	Compute Kolmogorov-smirnov's statistic for calibration test.
metrics.kuiper_cdf(x)	Compute the Kuiper cumulative distribution function (CDF) for the float x.
metrics.kuiper_p_value(y_true, y_score)	Compute Kuiper statistic p-value.
metrics.kuiper_statistic(y_true, y_score)	Compute Kuiper's statistic for calibration test.
metrics.length_scale(s)	Compute the mean square root of the sum of s * (1 - s).
metrics.regression_coverage_score(y_true, ...)	Effective coverage score obtained by the prediction intervals.
metrics.regression_coverage_score_v2(y_true, ...)	Effective coverage score obtained by the prediction intervals.
metrics.regression_mean_width_score(...)	Effective mean width score obtained by the prediction intervals.
metrics.regression_ssc(y_true, y_intervals)	Compute Size-Stratified Coverage metrics proposed in [3] that is the conditional coverage conditioned by the size of the intervals.
metrics.regression_ssc_score(y_true, y_intervals)	Aggregate by the minimum for each alpha the Size-Stratified Coverage [3]: returns the maximum violation of the conditional coverage (with the groups defined).
metrics.spiegelhalter_p_value(y_true, y_score)	Compute Spiegelhalter statistic p-value.
metrics.spiegelhalter_statistic(y_true, y_score)	Compute Spiegelhalter's statistic for calibration test.
metrics.top_label_ece(y_true, y_scores[, ...])	The Top-Label ECE which is a method adapted to fit the ECE to a Top-Label setting [2].

Conformity scores

conformity_scores.AbsoluteConformityScore([sym])	Absolute conformity score.
conformity_scores.GammaConformityScore([sym])	Gamma conformity score.
conformity_scores.ResidualNormalisedScore([...])	Residual Normalised score.

Resampling

subsample.BlockBootstrap([n_resamplings, ...])	Generate a sampling method, that block bootstraps the training set.
subsample.Subsample([n_resamplings, ...])	Generate a sampling method, that resamples the training set with possible bootstraps.