I am working on a latent variable model with the following form:
p is my model, with fixed global parameters
q is my mean-field guide with local variational parameters
To speed up learning, I want to perform subsampling on the indexes
i, which is very close to stochastic variational inference as presented in http://www.jmlr.org/papers/volume14/hoffman13a/hoffman13a.pdf (except that for them
theta is a random variable, corresponding to an additional global term in the guide).
My problem is that I don’t know how to tell Pyro the following: my goal is to learn the global parameter
theta, and for this learning phase I don’t care about the local variational parameters
phi_i. I only want to perform gradient ascent on
theta, and compute a batch of
phi_i in the background to get a (twice) noisy estimate of the gradient. Is this built in Pyro? Do I have to declare all of the
phi_i as parameters nonetheless?
Thanks in advance
[EDIT] Could it have something to do with this issue: https://github.com/pyro-ppl/pyro/issues/238 ?