mapie.metrics.cumulative_differences¶
- mapie.metrics.cumulative_differences(y_true: ndarray[Any, dtype[_ScalarType_co]], y_score: ndarray[Any, dtype[_ScalarType_co]], noise_amplitude: float = 1e-08, random_state: Optional[Union[int, RandomState]] = 1) ndarray[Any, dtype[_ScalarType_co]][source]¶
Compute the cumulative difference between y_true and y_score, both ordered according to y_scores array.
- Parameters
- y_trueNDArray of size (n_samples,)
An array of ground truths.
- y_scoreNDArray of size (n_samples,)
An array of scores.
- noise_amplitudefloat, optional
The tiny relative noise amplitude to add, by default 1e-8.
- random_state: Optional[Union[int, RandomState]]
Pseudo random number generator state used for random sampling. Pass an int for reproducible output across multiple function calls.
- Returns
- NDArray
The mean cumulative difference between y_true and y_score.
References
Arrieta-Ibarra I, Gujral P, Tannen J, Tygert M, Xu C. Metrics of calibration for probabilistic predictions. The Journal of Machine Learning Research. 2022 Jan 1;23(1):15886-940.
Examples
>>> import numpy as np >>> from mapie.metrics import cumulative_differences >>> y_true = np.array([1, 0, 0]) >>> y_score = np.array([0.7, 0.3, 0.6]) >>> cum_diff = cumulative_differences(y_true, y_score) >>> print(len(cum_diff)) 3 >>> print(np.max(cum_diff) <= 1) True >>> print(np.min(cum_diff) >= -1) True >>> cum_diff array([-0.1, -0.3, -0.2])
