AutoNormal documentation confusion

1

I have a minor confusion about the documentation for the AutoNormal guide. Specifically, the docstring for the guide states the following:

This implementation of AutoGuide uses Normal(0, 1) distributions to construct a guide over the entire latent space.

where I bolded my point of confusion. In terms of variational inference, I understand the guide to be the variational distribution which is optimized to fit the posterior distribution (i.e., the model in pyro).

A normal distribution with mean 0 and variance 1 can’t be fit to anything, since the parameters are already defined. I don’t believe it represents the prior because the priors should be defined in the model. I also looked at the code, and it seems that the parameter values for the normal distributions aren’t initialized to mean=0 or var=1 by default.

Is this documentation simply a typo? That is, should I read the documentation as saying “This implementation of AutoGuide uses independent normal distributions to construct a guide over the entire latent space.” Or do I have a major misunderstanding? (If it is the former, I can submit a pull request with a fix.)

Thanks for the great work on this package and the help.

2

yes, this is a typo. thanks for pointing it out! as it says elsewhere in the docstring AutoNormal is equivalent to AutoDiagonalNormal, i.e. it uses a learned mean and a learned diagonal covariance matrix to construct the guide distribution. i’ve put up a PR to clarify.

1 Like
3

Thank you for the clarification and the great work!