Hi,
May be the question is too simple but I have a problem with ~ 20 variables and the “true” posterior contains correlations between same variables like a non-diagonal covariance matrix. Now, using a SVI with MultivariateNormal guide, the output of the variational distribution does not show this correlations. Is there a trivial reason for that? Thanks.
Are you using Pyro or NumPyro? Are you using a custom guide or an AutoGuide? What is the dimension of your latent space? Have you tried training for more steps? Increasing learning rate?
You might try Pyro dev branch, as there have been recent improvements to AutoMultivariateNormal. Also try AutoLowRankMultivariateNormal.
Hello @fritzo, thanks for your interest.
I’m using Numpyro AutoMultivariateNormal guide. The model has about 20 latent variables as well as the guide. Now, tuning the scheduling of the learning rate, I have played around, but I cannot be sure 100% that the SVI has converged.