:orphan:

.. _calibration_examples:

Calibration examples
=======================


.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. thumbnail-parent-div-close

.. raw:: html

    </div>


1. Quickstart examples
----------------------

The following examples present the main functionalities of MAPIE through basic quickstart calibration problems.


.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="We compare an uncalibrated model to its Venn-ABERS calibrated version using reliability diagrams and Brier scores.">

.. only:: html

  .. image:: /examples_calibration/1-quickstart/images/thumb/sphx_glr_plot_calibration_venn_abers_binary_thumb.png
    :alt:

  :ref:`sphx_glr_examples_calibration_1-quickstart_plot_calibration_venn_abers_binary.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Calibrating binary classifier with Venn-ABERS</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="These statistical tests are based on the following references:">

.. only:: html

  .. image:: /examples_calibration/1-quickstart/images/thumb/sphx_glr_plot_calibration_hypothesis_testing_thumb.png
    :alt:

  :ref:`sphx_glr_examples_calibration_1-quickstart_plot_calibration_hypothesis_testing.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Testing for calibration in binary classification settings</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Calibrating multi-class classifier with Venn-ABERS">

.. only:: html

  .. image:: /examples_calibration/1-quickstart/images/thumb/sphx_glr_plot_calibration_venn_abers_multiclass_thumb.png
    :alt:

  :ref:`sphx_glr_examples_calibration_1-quickstart_plot_calibration_venn_abers_multiclass.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Calibrating multi-class classifier with Venn-ABERS</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


2. Advanced analysis
--------------------

The following examples use MAPIE for discussing more complex calibration problems.


.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="To this end, we generate many datasets that are calibrated by nature, and plot the distribution of the p-values. A p-value must follow a uniform distribution when computed on statistical samples following the null hypothesis.">

.. only:: html

  .. image:: /examples_calibration/2-advanced-analysis/images/thumb/sphx_glr_plot_asymptotic_convergence_of_p_values_thumb.png
    :alt:

  :ref:`sphx_glr_examples_calibration_2-advanced-analysis_plot_asymptotic_convergence_of_p_values.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Evaluating the asymptotic convergence of p-values</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:
   :includehidden:


   /examples_calibration/1-quickstart/index.rst
   /examples_calibration/2-advanced-analysis/index.rst


.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-gallery

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download all examples in Python source code: examples_calibration_python.zip </examples_calibration/examples_calibration_python.zip>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download all examples in Jupyter notebooks: examples_calibration_jupyter.zip </examples_calibration/examples_calibration_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_