Questions about LDA example code

Hi Pyro developers,

Thank you for maintaining this amazing project and the thorough documentation.

However, I do have two questions about the example code of amortized Latent Dirichlet Allocation.

The comment at line 97 (repo tag 1.8.5) says it is using conjugate guide, but I think it is not conjugate since the posterior and prior are the same (in this case both Gamma or both Dirichlet, but neither Gamma or Dirichlet are conjugate of themselves).

From the way the neural network is constructed, it seems to take a histogram as input without normalizing it. The code certainly works but it is just not common practice for a neural net to take arbitrarily large integers as input. So is there rational around this? Is it just to keep the same as the reference?

  • What tutorial are you running?
    Amortized Latent Dirichlet Allocation

  • What version of Pyro are you using?

  • Please link or paste relevant code, and steps to reproduce.
    Linked above.

Thank you,

1 Like
  • the dirichlet is conjugate to the multinomial likelihood. i guess topic_weights isn’t quite conjugate because it uses the gamma distribution, but note the close relation between gamma and dirichlet (see e.g. “Random variate generation” here). in any case the precise conjugacy relation isn’t particularly relevant here since we’re doing stochastic variational inference anyway.
  • regarding the second question this is just a tutorial. there is no claim that anything is done optimally. there are presumably millions of neural network architectures that could work here, some better, some worse.