|
1 | 1 | ################################################################ |
2 | | -The calibration (or "conformity") set |
| 2 | +The conformity (or "calibration") set |
3 | 3 | ################################################################ |
4 | 4 |
|
5 | | -**MAPIE** is based on two types of techniques: |
| 5 | +**MAPIE** is based on two types of techniques for measuring uncertainty in regression and classification: |
6 | 6 |
|
7 | 7 | - the split-conformal predictions, |
8 | 8 | - the cross-conformal predictions. |
9 | 9 |
|
10 | | -In all cases, the training/calibration process can be broken down as follows: |
| 10 | +In all cases, the training/conformalization process can be broken down as follows: |
11 | 11 |
|
12 | | -- Identify a basic model (or pre-trained model). |
13 | | -- Wrap it with the MAPIE class. |
14 | | -- Fit new model to calibration data (or full data if cross-validation) to estimate conformity scores. |
15 | | -- Predict target on test data to obtain prediction intervals/sets based on conformity scores. |
| 12 | +- Train a model using the training set (or full dataset if cross-conformal). |
| 13 | +- Estimate conformity scores using the conformity set (or full dataset if cross-conformal). |
| 14 | +- Predict target on test data to obtain prediction intervals/sets based on these conformity scores. |
16 | 15 |
|
17 | 16 |
|
18 | 17 | 1. Split conformal predictions |
19 | 18 | ============================== |
20 | 19 |
|
21 | | -- Construction of a conformity score. |
22 | | -- Calibration of the conformity score on a calibration set not seen by the model during training. |
| 20 | +- Compute conformity scores ("conformalization") on a conformity set not seen by the model during training. |
23 | 21 |
|
24 | | -**MAPIE** then uses the calibrated conformity scores to estimate sets associated with the desired coverage on new data with strong theoretical guarantees. |
| 22 | +**MAPIE** then uses the conformity scores to estimate sets associated with the desired coverage on new data with strong theoretical guarantees. |
25 | 23 |
|
26 | | -.. image:: images/cp_split.png |
| 24 | +Split conformal predictions with a pre-trained model |
| 25 | +------------------------------------------------------------------------------------ |
| 26 | + |
| 27 | +.. image:: images/cp_prefit.png |
27 | 28 | :width: 600 |
28 | 29 | :align: center |
29 | 30 |
|
30 | 31 |
|
31 | | -Prefit mode of split conformal predictions |
32 | | ------------------------------------------- |
| 32 | +Split conformal predictions with an untrained model |
| 33 | +------------------------------------------------------------------------------------ |
33 | 34 |
|
34 | | -.. image:: images/cp_prefit.png |
| 35 | +.. image:: images/cp_split.png |
35 | 36 | :width: 600 |
36 | 37 | :align: center |
37 | 38 |
|
38 | 39 |
|
39 | 40 | 2. Cross conformal predictions |
40 | 41 | ============================== |
41 | 42 |
|
42 | | -- Conformity scores on the whole training set obtained by cross-validation, |
| 43 | +- Conformity scores on the whole dataset obtained by cross-validation, |
43 | 44 | - Perturbed models generated during the cross-validation. |
44 | 45 |
|
45 | 46 | **MAPIE** then combines all these elements in a way that provides prediction intervals on new data with strong theoretical guarantees. |
|
0 commit comments