Hi,
I am New to Pyro and Probabilistic Programming Languages. I would like to know what kinds of distributions can Pyro represents. Of course, Pyro can represent the standard distributions. I would like to know what kinds of generative models, discriminative models, parameter or non-parameter models can Pyro represent. My research interest is Probabilistic Graphic Models(PGM). Can Pyro represent PGM and do the inference in PGM?
Thank you so much!
Hi, it might be better to ask whether a particular example model could have a nice implementation in Pyro rather than whether a class of models is representable in principle. You might be interested in our examples of a variational autoencoder (a deep generative model), Bayesian regression (a discriminative model), and Dirichlet process mixture model (a nonparametric model), and our ICML paper on discrete factor graph inference in Pyro. Do any of those help answer your question?
Yes! Thank you so much for your reply. I know sometimes I ask stupid questions. I am comparing different PPL like Pyro, Edward, Turing so that I am coming up some general questions. I first want to know the difference and how powerful Pyro is.
I am reading the tutorials and I am wondering how can I know it is a generative model or discriminative model. For example, for Bayesian Neural Network, I think it models p(y|x,\theta) in pyro but I can not sure.
If I have a Bayesian Network (for example a DAG) with 1000000 (very large) nodes and assume I know every CPD, how can I make a inference in Pyro? For example, can Pyro compute any p(X|E) where X are a list of nodes we are interested and E is list of nodes that are the evidence?
Thank you so much for your reply again and I really appreciate your help!!
I am reading the tutorials and I am wondering how can I know it is a generative model or discriminative model. For example, for Bayesian Neural Network, I think it models p(y|x,\theta) in pyro but I can not sure.
Bayesian neural networks include priors on the parameters, so they model p(y, \theta | x), but they are otherwise discriminative in the usual sense.
If I have a Bayesian Network (for example a DAG) with 1000000 (very large) nodes and assume I know every CPD, how can I make a inference in Pyro? For example, can Pyro compute any p(X|E) where X are a list of nodes we are interested and E is list of nodes that are the evidence?
If your model has no higher-level structure (such as plates) at all and is just a completely arbitrary low-treewidth DAG over millions of discrete variables, you can compute exact marginal probabilities by passing the conditional probability tables into a tensor contraction engine like opt_einsum or Pyro’s tensor variable elimination implementation pyro.ops.contract.einsum (which generalizes opt_einsum to support plates and multiple outputs as well as other semirings like max-add for MAP queries). This is discussed in more detail in our ICML paper “Tensor Variable Elimination for Plated Factor Graphs” that I linked to in my previous post.
When I was choosing a PPL to learn, the fact that Pyro is closely related to Pytorch made the choice a no-brainer for me 