Add quadratic-type means & implement linear operations for means

[pjpollot]

Quadratic mean for GP models

Overview

Standard version

$$ \mu (x) = x A x^\top, $$

$$ \nabla \mu (x) = \frac12 x (A + A^\top), $$

$$ \nabla^2 \mu(x) = A. $$

Positive version

We define $A = L L^\top$ and $L$ is a lower triangular matrix. We also that the gradient becomes $\nabla \mu(x) = x A$ as A is now symmetric, and the Hessian remains unchanged.

Linear operations

Add the possibilities to do linear operations with means. For all mean modules $\mu, \nu$ and for all $\lambda \in \mathbb{R}$, we can define the following mean:

$$ \eta(x) := \mu(x) + \lambda \cdot \nu(x) $$

$$ = \mu(x) + \nu(x) \cdot \lambda. $$

It is especially useful to combine a linear mean with a quadratic mean.

Motivation

Using a quadratic mean could be an interesting alternative to a Polynomial kernel of degree two, reducing the risk getting less singular matrixes when fitting a GP model to some rough training datasets by considering simpler kernels conjointly with this mean module.

Checklist

• Add the gradient and the hessian
• Allow linear operations (esp. Quadratic + Linear)
• Add unit tests
• Give examples in 1D
• Check if the documentation is well generated

I have run pip install -e . but I cannot generate locally the docs. Do you have any idea on how to install all the dependencies correctly for sphinx?

Ideas for further PRs

• Reduce the number of files inside the means folder, e.g. gather *.py, *_grad.py and *_gradgrad.py into a single file.
• Refactor the unit tests for means as the base test for means is always re-implemented. We could think of a way to shorten the boiler plate code.

Example

access to the notebook