Deep Learning vs Deep Universal Probabilistic Programming

1

What projects/research in Deep Universal Probabilistic Programming (Pyro) is more demanding compared to Deep Learning (PyTorch)?

2

Hi, I’m not sure what you mean by demanding, but see Pyro’s deep Markov model or Attend-Infer-Repeat tutorials for examples of the sort of research enabled or simplified by a PPL like Pyro. Is that what you were looking for?

1 Like
3

Thanks for the insight. I’m also asking what type of data problems are favorable for Deep Universal Probabilistic Programming compared to deep learning

4

I’m not sure I fully understand your question, but here’s a stab. If you ignore “universal” for a moment then the question is about standard deep learning vs. deep probabilistic programming. At its core, deep probabilistic programming has two characteristics: (1) making it easy to create and use probabilistic models, and (2) making it easy to integrate deep learning-based models with those probabilistic models.

The main benefit is that you get the benefits of deep learning with the advantages of probabilistic models. This is based on representing, learning from, and propagating uncertainty throughout your system. By doing so, you might be able to:

  1. Form credibility intervals for outputs and parameters
  2. Increase the robustness of your system: learning posteriors over parameters can decrease undesired variability in your system
  3. Model your problem with more specificity: there may be some structure that can be captured by your knowledge of how components of the system are distributed
  4. Use uncertainty to guide learning decisions, such as where to sample in active learning scenarios
  5. Reduce the amount of data you need to perform some useful learning task

These are benefits in and of themselves. What that means is when you see problems for which you want (a) distributions of something, (b) a way of representing variability, (c) to express probabilistic structure, (d) make decisions based on your uncertainty, and (e) do all of the above with less data than deep learning normally needs – then PPLs help. But if you also need suitably complex flexible function approximation and representation learning, then deep learning helps, which is why having a system that combines the two is advantageous over one or the other.

The Pyro team’s UAI 2018 resources give a good overview of some of these concepts and how they motivate Pyro’s development. It would help if you provided more context and motivation for your question.

4 Likes