mapie.metrics.top_label_ece

mapie.metrics.top_label_ece(y_true: Union[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]], numpy._typing._nested_sequence._NestedSequence[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]]], bool, int, float, complex, str, bytes, numpy._typing._nested_sequence._NestedSequence[Union[bool, int, float, complex, str, bytes]]], y_scores: Union[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]], numpy._typing._nested_sequence._NestedSequence[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]]], bool, int, float, complex, str, bytes, numpy._typing._nested_sequence._NestedSequence[Union[bool, int, float, complex, str, bytes]]], y_score_arg: Optional[Union[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]], numpy._typing._nested_sequence._NestedSequence[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]]], bool, int, float, complex, str, bytes, numpy._typing._nested_sequence._NestedSequence[Union[bool, int, float, complex, str, bytes]]]] = None, num_bins: int = 50, split_strategy: Optional[str] = None, classes: Optional[Union[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]], numpy._typing._nested_sequence._NestedSequence[numpy._typing._array_like._SupportsArray[numpy.dtype[Any]]], bool, int, float, complex, str, bytes, numpy._typing._nested_sequence._NestedSequence[Union[bool, int, float, complex, str, bytes]]]] = None) → float

The Top-Label ECE which is a method adapted to fit the ECE to a Top-Label setting [2].

[2] Gupta, Chirag, and Aaditya K. Ramdas. “Top-label calibration and multiclass-to-binary reductions.” arXiv preprint arXiv:2107.08353 (2021).

Parameters

y_true: ArrayLike of shape (n_samples,)

The target values for the calibrator.

y_scores: ArrayLike of shape (n_samples, n_classes)

or (n_samples,)

The predictions scores, either the maximum score and the argmax needs to be inputted or in the form of the prediction probabilities.

y_score_arg: Optional[ArrayLike] of shape (n_samples,)

If only the maximum is provided in the y_scores, the argmax must be provided here. This is optional and could be directly infered from the y_scores.

num_bins: int

Number of bins to make the split in the y_score. The allowed values are num_bins above 0.

split_strategy: str

The way of splitting the predictions into different bins. The allowed split strategies are “uniform”, “quantile” and “array split”.

classes: ArrayLike of shape (n_samples,)

The different classes, in order of the indices that would be present in a pred_proba.

Returns

float

The ECE score adapted in the top label setting.