Research papers on variational inference algorithms Pyro implements under the hood

Misc.
1

Can you please refer the research papers on variational inference algorithms and techniques that pyro uses under the hood. Without good understanding of how the actual inference happens it’s hard to debug and understand the models.
Thank you.

2

Thanks for the suggestion Aleksandr, I’ve filed Point to literature in algorithm implementations · Issue #662 · pyro-ppl/pyro · GitHub on your behalf. Are there particular places in the Pyro codebase where you would like pointers to literature? If so, I’ll add them to the task list on that issue.

1 Like
3

the references listed in the 3 svi tutorials on pyro.ai are a reasonably complete list of references for the main techniques being used:

  • Automated Variational Inference in Probabilistic Programming, David Wingate, Theo Weber
  • Black Box Variational Inference, Rajesh Ranganath, Sean Gerrish, David M. Blei
  • Auto-Encoding Variational Bayes, Diederik P Kingma, Max Welling
  • Stochastic Variational Inference, Matthew D. Hoffman, David M. Blei, Chong Wang, John Paisley
  • Gradient Estimation Using Stochastic Computation Graphs, John Schulman, Nicolas Heess, Theophane Weber, Pieter Abbeel
  • Deep Amortized Inference for Probabilistic Programs, Daniel Ritchie, Paul Horsfall, Noah D. Goodman
  • Neural Variational Inference and Learning in Belief Networks, Andriy Mnih, Karol Gregor

if there’s any particular algorithmic detail you would like a reference for, please try to be more specific.

1 Like