mapie.metrics.calibration.top_label_ece

mapie.metrics.calibration.top_label_ece(y_true: ArrayLike, y_scores: ArrayLike, y_score_arg: ArrayLike | None = None, num_bins: int = 50, split_strategy: str | None = None, classes: ArrayLike | None = None) → float

The Top-Label ECE which is a method adapted to fit the ECE to a Top-Label setting [2].

[2] Gupta, Chirag, and Aaditya K. Ramdas. “Top-label calibration and multiclass-to-binary reductions.” arXiv preprint arXiv:2107.08353 (2021).

Parameters:

y_true: ArrayLike of shape (n_samples,)

The target values for the calibrator.

y_scores: ArrayLike of shape (n_samples, n_classes)

or (n_samples,)

The predictions scores, either the maximum score and the argmax needs to be inputted or in the form of the prediction probabilities.

y_score_arg: Optional[ArrayLike] of shape (n_samples,)

If only the maximum is provided in the y_scores, the argmax must be provided here. This is optional and could be directly infered from the y_scores.

num_bins: int

Number of bins to make the split in the y_score. The allowed values are num_bins above 0.

split_strategy: str

The way of splitting the predictions into different bins. The allowed split strategies are “uniform”, “quantile” and “array split”.

classes: ArrayLike of shape (n_samples,)

The different classes, in order of the indices that would be present in a pred_proba.

Returns:

float

The ECE score adapted in the top label setting.