First of all I wanted to thank the people on the Pyro team for creating the GP contrib section. I find it nice and really easy to use. It sets a nice standard for good software to allow researchers to explore GP algorithms - something we desperate need in the GP community.
I am trying to verify some things that have been said in this paper (Damianou 2014) for my research. They claim that we can use the BGPLVM model in the case of modeling uncertain inputs. So, instead of having deterministic inputs X, the inputs come from a distribution e.g. N(mu_x, Sigma_x).
In the paper, they claim that we can do one of two things:
- Set a prior distribution to X (the model(?) as you call it in Pyro)
- Set a prior distribution to the variational parameter q(X) (or the guide as you call it in Pyro)
I like the example you put in the tutorial (from this paper GrandPrix 2019) as it is something very similar in the sense that you are explicitly putting priors on X. The problem I’m doing is slightly different as I’m not reducing the dimensionality but I think it’s a similar situation regardless; I imagine that I just need to finess the priors and which parameters require a grad function. However, I’m having trouble understanding how we are setting the priors and what exactly they represent within our GPLVM model.
Code - Setting Priors
Continuing from the tutorial where you set a prior to the
gplvm class (
#... # we use `.to_event()` to tell Pyro that the prior distribution for X has no batch_shape gplvm.set_prior("X", dist.Normal(X_prior_mean, 0.1).to_event()) gplvm.autoguide("X", dist.Normal)
autoguide puzzles me a bit as I cannot really understand where to access the parameters. I could be wrong but doing a simple inspection of the model attributes
gplvm.mode = 'model' model_X_loc = gplvm.X_loc.cpu().detach().numpy() model_X_scale_unconstrained = gplvm.X_scale_unconstrained.cpu().detach().numpy()
and similarly a simple inspection of the guide attributes
gplvm.mode = 'guide' guide_X_loc = gplvm.X_loc.cpu().detach().numpy() guide_X_scale_unconstrained = gplvm.X_scale_unconstrained.cpu().detach().numpy()
gives the exact same output
assert(model_X_loc.all() == guide_X_loc.all()) assert(model_X_scale_unconstrained.all() == guide_X_scale_unconstrained.all())
So my intuition is that I just don’t understand where the parameters are stored in this model nor the guide because if I look at the min, mean and max of the values for the model and the guide
print(model_X_loc.min(), model_X_loc.mean(), model_X_loc.max()) print(model_X_scale_unconstrained.min(), model_X_scale_unconstrained.mean(), model_X_scale_unconstrained.max())
0.0 0.38005337 1.0 0.0 0.0 0.0
which doesn’t make sense to me because we explicitly set the prior to have a 0.1 variance.
Would anyone be able to help me and give me some more intuition or perhaps point me in the direction of some tutorials about how I can do the following:
- Set a prior distribution to the parameter X
- Constrain the mean and/or variance of the distribution of the prior for X (e.g. positive, zero_grad)
- Set a prior distribution to the parameter q (the guide)
- Constrain the mean and/or variance of the distribution of the prior for the guide (e.g. positive, zero_grad)
Thank you in advanced.
J. Emmanuel Johnson