MAPIE API

Regression

Conformalizers

mapie.regression.SplitConformalRegressor([...])

Computes prediction intervals using the split conformal regression technique:

mapie.regression.CrossConformalRegressor([...])

Computes prediction intervals using the cross conformal regression technique:

mapie.regression.JackknifeAfterBootstrapRegressor([...])

Computes prediction intervals using the jackknife-after-bootstrap technique:

mapie.regression.ConformalizedQuantileRegressor([...])

Computes prediction intervals using the conformalized quantile regression technique:

mapie.regression.TimeSeriesRegressor([...])

Prediction intervals with out-of-fold residuals for time series.

Metrics

mapie.metrics.regression.regression_coverage_score(...)

Effective coverage obtained by the prediction intervals.

mapie.metrics.regression.regression_mean_width_score(...)

Effective mean width score obtained by the prediction intervals.

mapie.metrics.regression.regression_ssc(...)

Compute Size-Stratified Coverage metrics proposed in [3] that is the conditional coverage conditioned by the size of the intervals.

mapie.metrics.regression.regression_ssc_score(...)

Aggregate by the minimum for each confidence level the Size-Stratified Coverage [3]: returns the maximum violation of the conditional coverage (with the groups defined).

mapie.metrics.regression.hsic(y_true, ...[, ...])

Compute the square root of the hsic coefficient.

mapie.metrics.regression.coverage_width_based(...)

Coverage Width-based Criterion (CWC) obtained by the prediction intervals.

mapie.metrics.regression.regression_mwi_score(...)

The Winkler score, proposed by Winkler (1972), is a measure used to evaluate prediction intervals, combining the length of the interval with a penalty that increases proportionally to the distance of an observation outside the interval.

Conformity Scores

mapie.conformity_scores.BaseRegressionScore(sym)

Base conformity score class for regression task.

mapie.conformity_scores.AbsoluteConformityScore([sym])

Absolute conformity score.

mapie.conformity_scores.GammaConformityScore([sym])

Gamma conformity score.

mapie.conformity_scores.ResidualNormalisedScore([...])

Residual Normalised score.

Resampling

mapie.subsample.BlockBootstrap([...])

Generate a sampling method, that block bootstraps the training set.

mapie.subsample.Subsample([n_resamplings, ...])

Generate a sampling method, that resamples the training set with possible bootstraps.

Classification

Conformalizers

mapie.classification.SplitConformalClassifier([...])

Computes prediction sets using the split conformal classification technique:

mapie.classification.CrossConformalClassifier([...])

Computes prediction sets using the cross conformal classification technique:

Metrics

mapie.metrics.classification.classification_coverage_score(...)

Effective coverage score obtained by the prediction sets.

mapie.metrics.classification.classification_mean_width_score(...)

Mean width of prediction set output by mapie.classification._MapieClassifier.

mapie.metrics.classification.classification_ssc(...)

Compute Size-Stratified Coverage metrics proposed in [3] that is the conditional coverage conditioned by the size of the predictions sets.

mapie.metrics.classification.classification_ssc_score(...)

Aggregate by the minimum for each confidence level the Size-Stratified Coverage [3]: returns the maximum violation of the conditional coverage (with the groups defined).

Conformity Scores

mapie.conformity_scores.BaseClassificationScore()

Base conformity score class for classification task.

mapie.conformity_scores.NaiveConformityScore()

Naive classification non-conformity score method that is based on the cumulative sum of probabilities until the 1-alpha threshold.

mapie.conformity_scores.LACConformityScore()

Least Ambiguous set-valued Classifier (LAC) method-based non conformity score (also formerly called "score").

mapie.conformity_scores.APSConformityScore()

Adaptive Prediction Sets (APS) method-based non-conformity score.

mapie.conformity_scores.RAPSConformityScore([...])

Regularized Adaptive Prediction Sets (RAPS) method-based non-conformity score.

mapie.conformity_scores.TopKConformityScore()

Top-K method-based non-conformity score.

Risk Control

mapie.risk_control.MultiLabelClassificationController(...)

Prediction sets for multilabel-classification.

mapie.risk_control.BinaryClassificationController(...)

Controls the risk or performance of a binary classifier.

mapie.risk_control.SemanticSegmentationController(...)

Risk controller for semantic segmentation tasks, inheriting from MultiLabelClassificationController.

mapie.risk_control.BinaryRisk(...)

Define a risk (or a performance metric) to be used with the BinaryClassificationController.

mapie.risk_control.BinaryClassificationRisk(...)

Deprecated alias for BinaryRisk.

Calibration

Conformalizer

mapie.calibration.TopLabelCalibrator([...])

Top-label calibration for multi-class problems.

Metrics

mapie.metrics.calibration.expected_calibration_error(...)

The expected calibration error, which is the difference between the confidence scores and accuracy per bin [1].

mapie.metrics.calibration.top_label_ece(...)

The Top-Label ECE which is a method adapted to fit the ECE to a Top-Label setting [2].

mapie.metrics.calibration.cumulative_differences(...)

Compute the cumulative difference between y_true and y_score, both ordered according to y_scores array.

mapie.metrics.calibration.kolmogorov_smirnov_cdf(x)

Compute the Kolmogorov-smirnov cumulative distribution function (CDF) for the float x.

mapie.metrics.calibration.kolmogorov_smirnov_p_value(...)

Compute Kolmogorov Smirnov p-value.

mapie.metrics.calibration.kolmogorov_smirnov_statistic(...)

Compute Kolmogorov-smirnov's statistic for calibration test.

mapie.metrics.calibration.kuiper_cdf(x)

Compute the Kuiper cumulative distribution function (CDF) for the float x.

mapie.metrics.calibration.kuiper_p_value(...)

Compute Kuiper statistic p-value.

mapie.metrics.calibration.kuiper_statistic(...)

Compute Kuiper's statistic for calibration test.

mapie.metrics.calibration.length_scale(s)

Compute the mean square root of the sum of s * (1 - s).

mapie.metrics.calibration.spiegelhalter_p_value(...)

Compute Spiegelhalter statistic p-value.

mapie.metrics.calibration.spiegelhalter_statistic(...)

Compute Spiegelhalter's statistic for calibration test.

Utils

mapie.utils.train_conformalize_test_split(X, ...)

Split arrays or matrices into train, conformalization and test subsets.