mapie.metrics
.top_label_ece¶
- mapie.metrics.top_label_ece(y_true: Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]], y_scores: Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]], y_score_arg: Optional[Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]]] = None, num_bins: int = 50, split_strategy: Optional[str] = None, classes: Optional[Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]]] = None) float [source]¶
The Top-Label ECE which is a method adapted to fit the ECE to a Top-Label setting [2].
[2] Gupta, Chirag, and Aaditya K. Ramdas. “Top-label calibration and multiclass-to-binary reductions.” arXiv preprint arXiv:2107.08353 (2021).
- Parameters
- y_true: ArrayLike of shape (n_samples,)
The target values for the calibrator.
- y_scores: ArrayLike of shape (n_samples, n_classes)
- or (n_samples,)
The predictions scores, either the maximum score and the argmax needs to be inputted or in the form of the prediction probabilities.
- y_score_arg: Optional[ArrayLike] of shape (n_samples,)
If only the maximum is provided in the y_scores, the argmax must be provided here. This is optional and could be directly infered from the y_scores.
- num_bins: int
Number of bins to make the split in the y_score. The allowed values are num_bins above 0.
- split_strategy: str
The way of splitting the predictions into different bins. The allowed split strategies are “uniform”, “quantile” and “array split”.
- classes: ArrayLike of shape (n_samples,)
The different classes, in order of the indices that would be present in a pred_proba.
- Returns
- float
The ECE score adapted in the top label setting.