Challenges with Implementing Beta Process (Indian Buffet Process) Factor Analysis

Misc.
1

I’d like to implement Beta Process factor analysis in Pyro (or maybe NumPyro). When I previously implemented its related cousin the Dirichlet Process mixture model, I ran into multiple issues with common inference algorithms, both SVI and MCMC, (e.g. 1, 2) which led me to discover that that sampling discrete variables is difficult and that getting cluster assignments would be unreliable.

Looking forward at Beta Process factor analysis, what traps should I expect to run into? Will this even be doable?

Looking at this post on PyMC3, I’m concerned this might not be feasible at all.

Thanks in advance! I’m hoping to avoid accidentally sinking time into a futile endeavor, if possible :slight_smile:

2

@fehiepsi , you were really helpful earlier. Any suggestions of pitfalls I should look out for?

3

Hi @RylanSchaeffer, Pyro/NumPyro supports automatic marginalization over discrete latent variables so you can use it for the model in Figure 2 of your reference. But it is not practical to use enumeration for the model defined in formula (11) (yeah, they are different: in the former, z has shape (N,) while in the latter, Z has shape (N, K)). There are two workarounds I guess:

  • Using NumPyro’s DiscreteHMCGibbs: I’m not sure if it will work well (assuming z dimensions are large) because Gibbs samplers are slow to converge
  • Using Pyro’s SVI with TraceGraph_ELBO objective.