Enumeration in the GUIDE: Gaussian Mixture Model

Misc.
1

I have a model with discrete random variable and have been experimenting with TraceENUM_ELBO.

I understand that:

  1. enumerating a discrete variable in the model, means that the variable is marginalized out and, therefore, does not need to be sampled in the guide anymore.

  2. enumerating a discrete variable in the guide, it is just a “trick” to obtain lower variance estimator of the gradients.

My question is what is the objective which is maximized in the latter (i.e. enumeration in the guide) case? To be concrete, let’s assume that in the Model there is a c~Bernulli§ variable and that variable is enumerated in the Guide.

Is the objective being maximized ELBO_{eff} = p * ELBO(c=1) + (1-p) * ELBO(c=0)?

Thanks

2

your understanding is correct. in the simplest case, the exact expectation is arrived at by enumerating the support of the distribution weighted by its pmf. pyro does this by exploiting broadcasting.