I read the documentations on discrete latent variables here: Inference with Discrete Latent Variables — Pyro Tutorials 1.8.4 documentation and have some questions on NUTS enumeration strategy and MixedHMC.
- do I get a posterior samples for discrete latent variables using NUTS/TraceEnum_ELBO? looks like inference is only done for continous variables.
- do I get a posterior samples for discrete latent variables using MixedHMC/TraceGraph_ELBO? If so, is the strategy different from enumeration?
- a general question: both NUTS and MixedHMC can handle discrete and continuous variables, what are their advantages and disadvantages? Is enumeration generally better?
You can get samples for discrete variables using infer_discrete.
do I get a posterior samples for discrete latent variables using MixedHMC/TraceGraph_ELBO
yes. enumeration means that you marginalize out the discrete latent variable, so assume that you want to sample from p(z,x) where z is discrete - with enumeration, the inference provides samples for p(x) (then you can use infer_discrete to sample from p(z|x)). without enumeration, you sample from p(z,x)
Is enumeration generally better?
it depends on problems. enumeration is better if p(x) is simple to sample from. the mixedhmc paper shows some examples where nuts is not working (like multimodal densities).