Hi! I'm going through the Pyro tutorials and I'm wrestling with this snippet from http://pyro.ai/examples/intro_part_ii.html
In its most general formulation, inference in a universal probabilistic programming language like Pyro is the problem of constructing this marginal distribution given an arbitrary boolean constraint so that we can perform these computations. The constraint can be a deterministic function of the return value, the internal randomness, or both.
Bayesian inference or posterior inference is an important special case of this more general formulation that admits tractable approximations. In Bayesian inference, the return value is always the values of some subset internal sample statements, and the constraint is an equality constraint on the other internal sample statements. Much of modern machine learning can be cast as approximate Bayesian inference and expressed succinctly in a language like Pyro.
I'm coming to Pyro after reading Bayesian Methods for Hackers, where the presented definition of Bayesian inference was: "updating your beliefs after considering new evidence.". I'm having trouble reconciling these two worlds now. Specifically:
- How is this "arbitrary boolean constraint" used in the construction of the marginal distribution? Boolean implies some type of binary decisionmaking process, is that the case?
- How is the "subset of internal sample statements" in the Bayesian inference separated from "the other internal sample statements"? What exactly is being compared between these two sets of samples?
Hope those make at least some sense. Any attempts at alternate explanations or pointers to helpful resources would be appreciated, thanks!