.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples_regression/1-quickstart/plot_toy_model.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_regression_1-quickstart_plot_toy_model.py: ===================================================================================== Use MAPIE to plot prediction intervals ===================================================================================== An example plot of :class:`~mapie.regression.SplitConformalRegressor` used in the Quickstart. .. GENERATED FROM PYTHON SOURCE LINES 10-12 We will use MAPIE to estimate prediction intervals on a one-dimensional, non-linear regression problem. .. GENERATED FROM PYTHON SOURCE LINES 12-23 .. code-block:: Python import numpy as np from matplotlib import pyplot as plt from numpy.typing import NDArray from sklearn.neural_network import MLPRegressor from mapie.metrics.regression import regression_coverage_score from mapie.regression import SplitConformalRegressor from mapie.utils import train_conformalize_test_split RANDOM_STATE = 1 .. GENERATED FROM PYTHON SOURCE LINES 24-25 Firstly, let us create our dataset: .. GENERATED FROM PYTHON SOURCE LINES 25-49 .. code-block:: Python def f(x: NDArray) -> NDArray: """Polynomial function used to generate one-dimensional data.""" return np.array(5 * x + 5 * x**4 - 9 * x**2) rng = np.random.default_rng(1) sigma = 0.1 n_samples = 10000 X = np.linspace(0, 1, n_samples) y = f(X) + rng.normal(0, sigma, n_samples) X = X.reshape(-1, 1) (X_train, X_conformalize, X_test, y_train, y_conformalize, y_test) = ( train_conformalize_test_split( X, y, train_size=0.8, conformalize_size=0.1, test_size=0.1, random_state=RANDOM_STATE, ) ) .. GENERATED FROM PYTHON SOURCE LINES 50-56 We fit our training data with a MLPRegressor. Then, we initialize a :class:`~mapie.regression.SplitConformalRegressor` using our estimator, indicating that it has already been fitted with `prefit=True`. Lastly, we compute the prediction intervals with the desired confidence level using the ``conformalize`` and ``predict_interval`` methods. .. GENERATED FROM PYTHON SOURCE LINES 56-67 .. code-block:: Python regressor = MLPRegressor(activation="relu", random_state=RANDOM_STATE) regressor.fit(X_train, y_train) confidence_level = 0.95 mapie_regressor = SplitConformalRegressor( estimator=regressor, confidence_level=confidence_level, prefit=True ) mapie_regressor.conformalize(X_conformalize, y_conformalize) y_pred, y_pred_interval = mapie_regressor.predict_interval(X_test) .. GENERATED FROM PYTHON SOURCE LINES 68-72 ``y_pred`` represents the point predictions as a ``np.ndarray`` of shape ``(n_samples)``. ``y_pred_interval`` corresponds to the prediction intervals as a ``np.ndarray`` of shape ``(n_samples, 2, 1)``, giving the lower and upper bounds of the intervals. .. GENERATED FROM PYTHON SOURCE LINES 74-76 Finally, we can easily compute the coverage score (i.e., the proportion of times the true labels fall within the predicted intervals). .. GENERATED FROM PYTHON SOURCE LINES 76-84 .. code-block:: Python coverage_score = regression_coverage_score(y_test, y_pred_interval) print( f"For a confidence level of {confidence_level:.2f}, " f"the target coverage is {confidence_level:.3f}, " f"and the effective coverage is {coverage_score[0]:.3f}." ) .. rst-class:: sphx-glr-script-out .. code-block:: none For a confidence level of 0.95, the target coverage is 0.950, and the effective coverage is 0.951. .. GENERATED FROM PYTHON SOURCE LINES 85-89 In this example, the effective coverage is slightly above the target coverage (i.e., 0.95), indicating that the confidence level we set has been reached. Therefore, we can confirm that the prediction intervals effectively contain the true label more than 95% of the time. .. GENERATED FROM PYTHON SOURCE LINES 91-92 Now, let us plot the estimated prediction intervals. .. GENERATED FROM PYTHON SOURCE LINES 92-110 .. code-block:: Python plt.xlabel("x") plt.ylabel("y") plt.scatter(X_test, y_test, alpha=0.3) X_test = X_test.ravel() order = np.argsort(X_test) plt.plot(X_test[order], y_pred[order], color="C1") plt.plot(X_test[order], y_pred_interval[order][:, 0, 0], color="C1", ls="--") plt.plot(X_test[order], y_pred_interval[order][:, 1, 0], color="C1", ls="--") plt.fill_between( X_test[order], y_pred_interval[:, 0, 0][order].ravel(), y_pred_interval[:, 1, 0][order].ravel(), alpha=0.2, ) plt.title("Estimated prediction intervals with MLPRegressor") plt.show() .. image-sg:: /examples_regression/1-quickstart/images/sphx_glr_plot_toy_model_001.png :alt: Estimated prediction intervals with MLPRegressor :srcset: /examples_regression/1-quickstart/images/sphx_glr_plot_toy_model_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 111-113 On the plot above, the dots represent the samples from our dataset, while the orange area corresponds to the estimated prediction intervals for each ``x`` value. .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.577 seconds) .. _sphx_glr_download_examples_regression_1-quickstart_plot_toy_model.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_toy_model.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_toy_model.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_toy_model.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_