Num_particles vs batch_size

Misc.
1

Hello,

I’m trying to get a better sense of the distinction between num_particles in Trace_ELBO vs batch_size in plates for our observation/likelihood.

From my understanding num_particles refers to the number of samples/observations taken to estimate the gradients for the ELBO and its evaluation (defualt is a single observation), whereas batch_size refers to the number of sub-samples taken when when defining the model itself.

I’m still a little confused as to how these differ, and how it would operate if different values were passed to both. I did see another thread on using different values for these parameters, but I’m left with little understanding as what their conceptual differences are. Any help/clarification would be greatly appreciated.

2

for a typical latent variable model the computation of the ELBO via monte carlo estimate involves two kinds of sampling:

  • sampling of random variables (e.g. some normal latent variarble)
  • (sub)sampling of data/observations if the model has the right conditional independence structure to allow unbiased ELBO estimates to be obtained by only using mini-batches of data/observations

in effect num_particles controls the budget that is spent on the former, and batch_size controls the budget that is spent on the latter. as such they are entirely orthogonal and can be chosen separately (although both will impact, e.g., the variance of the ELBO estimator).