# `mapie.metrics`.cumulative_differences¶

mapie.metrics.cumulative_differences(y_true: ndarray[Any, dtype[_ScalarType_co]], y_score: ndarray[Any, dtype[_ScalarType_co]], noise_amplitude: float = 1e-08, random_state: = 1) ndarray[Any, dtype[_ScalarType_co]][source]

Compute the cumulative difference between y_true and y_score, both ordered according to y_scores array.

Parameters
y_trueNDArray of size (n_samples,)

An array of ground truths.

y_scoreNDArray of size (n_samples,)

An array of scores.

noise_amplitudefloat, optional

The tiny relative noise amplitude to add, by default 1e-8.

random_state: Optional[Union[int, RandomState]]

Pseudo random number generator state used for random sampling. Pass an int for reproducible output across multiple function calls.

Returns
NDArray

The mean cumulative difference between y_true and y_score.

References

Arrieta-Ibarra I, Gujral P, Tannen J, Tygert M, Xu C. Metrics of calibration for probabilistic predictions. The Journal of Machine Learning Research. 2022 Jan 1;23(1):15886-940.

Examples

```>>> import numpy as np
>>> from mapie.metrics import cumulative_differences
>>> y_true = np.array([1, 0, 0])
>>> y_score = np.array([0.7, 0.3, 0.6])
>>> cum_diff = cumulative_differences(y_true, y_score)
>>> print(len(cum_diff))
3
>>> print(np.max(cum_diff) <= 1)
True
>>> print(np.min(cum_diff) >= -1)
True
>>> cum_diff
array([-0.1, -0.3, -0.2])
```