I realize part of the problem I am having is that the posterior I get from ELBO is good enough for the variational inference even though I can clearly see that the solution is far from optimal. I am trying to fit a log-normal observed distribution with 3 latent variables and infer the mean and variance of the latent variables. I can manually step through the latent variables and create posterior that is better than what SVI yield. I have two questions.
- Is there hyperparameter search for SVI? Based on the simple toy example I posted here, I learned that learning rate, initial mean and std for the guide and the std of the model sample determines whether I can get a decent inference or not
pyro.sample("y", dist.Normal(expected_output, std), obs=y). As of now, I have no valid strategy to select these values. - as I am trying to find the latent variables that results in a distribution that has same median and std as the observed data, can I ask SVI to optimize those parameters?
Thanks in advance for your input.