Parameter optimize_beta is not working

[Valentin-Laurent]

After investigating issue #484, I realized our implementation of the optimize_beta option is incorrect, and has been from its introduction in MAPIE v0.4. See also issue #583.

Code to fix:

• In function mapie.conformity_scores.regression.BaseRegressionScore.get_bounds (the only place where beta optimization is performed), we pass a reshaped version of conformity scores to _beta_optimize, and we probably should not.

        if optimize_beta:
            beta_np = self._beta_optimize(
                alpha_np,
                conformity_scores.reshape(1, -1),
                conformity_scores.reshape(1, -1),
            )

• The implementation of _beta_optimize seems OK-ish (see below). However, we get conformity_scores.reshape(1, -1) for parameters lower_bounds and upper_bounds, and the reshape always produces arrays of shape (1, n), so the value of len(lower_bounds) is always 1.
• In _beta_optimize, there may be an issue with the type of _alpha in the enumerate.
• In _beta_optimize, we distinguish lower_bounds and upper_bounds even though this value is the same, so it could be simplified.

After fixing this, we should:

• Implement tests to make sure optimize_beta works as expected
• Verify example notebooks that are using this feature (are the results still consistent?)
• Make sure issue Error when computing multiple prediction intervals in Time Series Tutorial #484 is indeed fixed

The MAPIE core team is currently in discussion regarding the priority of this fix VS other ongoing developments. Contributions are welcome :)

Note: may be helpful: when using optimize_beta with a list of alphas, the prediction intervals shape is currently incorrect.

[C-BdB]

The parameter optimize_beta is removed from the examples of the documentation for version v1-alpha. PR #624
Examples need to be updated when parameter is fixed.