[Bug]: Dimensionality of posterior statistics

[AdrianSosic]

What happened?

Hi, I'm not sure if this is a bug, and if it is, whether it's even possible to fix it without breaking things. But I just noticed that the different posterior statistics one can request from a GPyTorchPosterior have rather inconsistent dimensionality.

Please provide a minimal, reproducible example of the unexpected behavior.

import torch
from botorch.posteriors import GPyTorchPosterior
from gpytorch.distributions import MultivariateNormal

mvn = MultivariateNormal(
    torch.tensor([0.0, 1.0]),
    covariance_matrix=torch.tensor([[1.0, 0.5], [0.5, 1.0]]),
)
posterior = GPyTorchPosterior(mvn)

print("Dim mean: ", posterior.mean.ndim)  # --> 2
print("Dim mode: ", posterior.mode.ndim)  # --> 1
print("Dim variance: ", posterior.variance.ndim)  # --> 2
print("Dim stddev: ", posterior.stddev.ndim)  # --> 1
print("Dim quantile: ", posterior.quantile(torch.tensor(0.5)).ndim)  # --> 2

BoTorch Version

0.14.0

Python Version

3.10

Operating System

macOS

(Optional) Describe any potential fixes you've considered to the issue outlined above.

No response

Pull Request

None

Code of Conduct

• I agree to follow BoTorch's Code of Conduct

[saitcakmak]

Hi @AdrianSosic. I believe the reason for this is GPyTorchPosterior will call getattr(self.distribution, name) for any attribute that is not explicitly defined. The mean, variance and quantile are all defined at the posterior level (with an explicit output dimension as is BoTorch convention) but not the others.

This behavior is defined at the TorchPosterior level, with the motivation being to allow people to use different posterior distributions from PyTorch without having to define a custom method for each.

[AdrianSosic]

I see, already expected something like that, hence was careful even calling it a "bug" 🙃 Thanks for the answer, though! So basically I just wanted to report the situation, it's of course up to you to decided whether to take any action or not. Maybe one way to harmonize things would be to "correct" the obtained object dimensions from the BoTorch side to make the returned objects follow the BoTorch convention. However, potentially this would break things in many other places, which is hard for me to oversee...

[saitcakmak]

Yeah, that does sound great on the surface but it is tricky to cover all attributes like that. Mean, variance, mode etc will have a standard shape of batch x num_points x num_outcomes. But then the covariance matrix has a completely different shape, very much dependent on the underlying distribution. If the distribution is defined jointly over points & outcomes, it might be batch x (num_points * num_outcomes) x (num_points * num_outcomes) or it can be batch x num_outcomes x num_points x num_points for independent outcomes.

It is quite difficult to enforce these consistently without adding some restrictions. We can either keep the current state of deferring to the distribution attributes, or require the user to implement a custom Posterior subclass for each new distribution they want to use. There isn't a ton of non-Gaussian posterior usage in BoTorch really, so it might be ok to just leave it to the user for the non-standard cases.