I have been looking at Graphical Models and VAEs over the past few days, and was wondering if there are any good articles or references on designing “guides” or the variational distributions that we use for approximate inference? We usually start with a mean-field approximate distribution that is faster to compute but which does not often capture all of the dependencies of the model. Give that situation, does anyone know of any good articles on designing guides that better capture the dependence structure while also preserving faster computation? @fritzo , perhaps you might have some suggestions.
One recommendation I have is my paper: https://papers.nips.cc/paper/7570-faithful-inversion-of-generative-models-for-effective-amortized-inference.pdf
It describes an algorithm for designing an optimal structure for the guide program that captures the dependency structure of the model while preserving computation (let me know if you have any questions!)
2 Likes
@stefanwebb Oh that is like exactly what I am looking for. Thanks for pointing it out, I don’t think I would have found it otherwise. I am much more accustomed to the MCMC methods, so still just trying to explore the variational inference tool. Thanks again.
1 Like