.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "examples_regression/1-quickstart/plot_toy_model.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_examples_regression_1-quickstart_plot_toy_model.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_examples_regression_1-quickstart_plot_toy_model.py:


Plot prediction intervals
=========================

An example plot of `SplitConformalRegressor` used
in the Quickstart.

.. GENERATED FROM PYTHON SOURCE LINES 10-12

We will use MAPIE to estimate prediction intervals on a one-dimensional,
non-linear regression problem.

.. GENERATED FROM PYTHON SOURCE LINES 12-24

.. code-block:: Python


    import numpy as np
    from matplotlib import pyplot as plt
    from numpy.typing import NDArray
    from sklearn.neural_network import MLPRegressor

    from mapie.metrics.regression import regression_coverage_score
    from mapie.regression import SplitConformalRegressor
    from mapie.utils import train_conformalize_test_split

    RANDOM_STATE = 1








.. GENERATED FROM PYTHON SOURCE LINES 25-26

Firstly, let us create our dataset:

.. GENERATED FROM PYTHON SOURCE LINES 26-50

.. code-block:: Python



    def f(x: NDArray) -> NDArray:
        """Polynomial function used to generate one-dimensional data."""
        return np.array(5 * x + 5 * x**4 - 9 * x**2)


    rng = np.random.default_rng(1)
    sigma = 0.1
    n_samples = 10000
    X = np.linspace(0, 1, n_samples)
    y = f(X) + rng.normal(0, sigma, n_samples)
    X = X.reshape(-1, 1)
    (X_train, X_conformalize, X_test, y_train, y_conformalize, y_test) = (
        train_conformalize_test_split(
            X,
            y,
            train_size=0.8,
            conformalize_size=0.1,
            test_size=0.1,
            random_state=RANDOM_STATE,
        )
    )








.. GENERATED FROM PYTHON SOURCE LINES 51-57

We fit our training data with a MLPRegressor.
Then, we initialize a `SplitConformalRegressor`
using our estimator, indicating that it has already been fitted with
`prefit=True`.
Lastly, we compute the prediction intervals with the desired confidence level using
the `conformalize` and `predict_interval` methods.

.. GENERATED FROM PYTHON SOURCE LINES 57-68

.. code-block:: Python


    regressor = MLPRegressor(activation="relu", random_state=RANDOM_STATE)
    regressor.fit(X_train, y_train)

    confidence_level = 0.95
    mapie_regressor = SplitConformalRegressor(
        estimator=regressor, confidence_level=confidence_level, prefit=True
    )
    mapie_regressor.conformalize(X_conformalize, y_conformalize)
    y_pred, y_pred_interval = mapie_regressor.predict_interval(X_test)








.. GENERATED FROM PYTHON SOURCE LINES 69-73

`y_pred` represents the point predictions as a `np.ndarray` of shape
`(n_samples)`.
`y_pred_interval` corresponds to the prediction intervals as a `np.ndarray` of
shape `(n_samples, 2, 1)`, giving the lower and upper bounds of the intervals.

.. GENERATED FROM PYTHON SOURCE LINES 75-77

Finally, we can easily compute the coverage score (i.e., the proportion of times the
true labels fall within the predicted intervals).

.. GENERATED FROM PYTHON SOURCE LINES 77-85

.. code-block:: Python


    coverage_score = regression_coverage_score(y_test, y_pred_interval)
    print(
        f"For a confidence level of {confidence_level:.2f}, "
        f"the target coverage is {confidence_level:.3f}, "
        f"and the effective coverage is {coverage_score[0]:.3f}."
    )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    For a confidence level of 0.95, the target coverage is 0.950, and the effective coverage is 0.951.




.. GENERATED FROM PYTHON SOURCE LINES 86-90

In this example, the effective coverage is slightly above the target coverage
(i.e., 0.95), indicating that the confidence level we set has been reached.
Therefore, we can confirm that the prediction intervals effectively contain the
true label more than 95% of the time.

.. GENERATED FROM PYTHON SOURCE LINES 92-93

Now, let us plot the estimated prediction intervals.

.. GENERATED FROM PYTHON SOURCE LINES 93-111

.. code-block:: Python


    plt.xlabel("x")
    plt.ylabel("y")
    plt.scatter(X_test, y_test, alpha=0.3)
    X_test = X_test.ravel()
    order = np.argsort(X_test)
    plt.plot(X_test[order], y_pred[order], color="C1")
    plt.plot(X_test[order], y_pred_interval[order][:, 0, 0], color="C1", ls="--")
    plt.plot(X_test[order], y_pred_interval[order][:, 1, 0], color="C1", ls="--")
    plt.fill_between(
        X_test[order],
        y_pred_interval[:, 0, 0][order].ravel(),
        y_pred_interval[:, 1, 0][order].ravel(),
        alpha=0.2,
    )
    plt.title("Estimated prediction intervals with MLPRegressor")
    plt.show()




.. image-sg:: /examples_regression/1-quickstart/images/sphx_glr_plot_toy_model_001.png
   :alt: Estimated prediction intervals with MLPRegressor
   :srcset: /examples_regression/1-quickstart/images/sphx_glr_plot_toy_model_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 112-114

On the plot above, the dots represent the samples from our dataset, while the
orange area corresponds to the estimated prediction intervals for each `x` value.


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 1.294 seconds)


.. _sphx_glr_download_examples_regression_1-quickstart_plot_toy_model.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_toy_model.ipynb <plot_toy_model.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_toy_model.py <plot_toy_model.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_toy_model.zip <plot_toy_model.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_