Can I fit a "Trueskill" type model in pyro - factor graph with expectation propagation

7

@jeffmax I do not know why EP-backed PPLs are rare. I’d be interested to hear @eb8680_2’s opinion. To provide some pointers to various places Pyro currently uses message passing:

  • TraceEnum_ELBO and other discrete enumeration tools perform exact message passing, requiring only one or two passes.
  • DiscreteHMM and GaussianHMM perform exact parallel-scan message passing, requiring only one or two passes.
  • pyro.contrib.tracking.assignment provides a number of soft assignment solvers based on loopy belief propagation; empirically these converge in a handful of iterations.
  • Pyro’s future Funsor backend provides a moment_matching interpretation for low-level EP computations, but does not perform EP iteration.
  • pyro.contrib.epidemiology has a helper set_relaxed_distributions() similar to Funsor’s momement_matching interpretation that reinterprets Binomial and BetaBinomial distributions as moment-matched normals.

I’d also love to see EP implemented in Pyro. While I agree with @krishnab that PyTorch is very AD-based, PyTorch is also a good solid system for vectorized distributed mathematics, and I think EP could be a reasonable competitive inference algorithm for some models, especially now that we have a HorovodOptimizer. I’d be happy to help anyone implement EP in Pyro and to publish a TrueSkill tutorial. If anyone wants to start implementing EP, please create a feature request issue. I’ll help by populating with low level tasks and pointers into code. I’m happy to chat over zoom to get anyone started.

2 Likes