I have a minor confusion about the documentation for the AutoNormal guide. Specifically, the docstring for the guide states the following:
This implementation of
AutoGuideuses Normal(0, 1) distributions to construct a guide over the entire latent space.
where I bolded my point of confusion. In terms of variational inference, I understand the guide to be the variational distribution which is optimized to fit the posterior distribution (i.e., the model in pyro).
A normal distribution with mean 0 and variance 1 can’t be fit to anything, since the parameters are already defined. I don’t believe it represents the prior because the priors should be defined in the model. I also looked at the code, and it seems that the parameter values for the normal distributions aren’t initialized to mean=0 or var=1 by default.
Is this documentation simply a typo? That is, should I read the documentation as saying “This implementation of AutoGuide uses independent normal distributions to construct a guide over the entire latent space.” Or do I have a major misunderstanding? (If it is the former, I can submit a pull request with a fix.)
Thanks for the great work on this package and the help.