Discrete hidden variables in custom guide

Misc.
1

Hello! I just wonder if it’s necessary to sample the sites of discrete hidden variables in my custom guide when I’m using TraceEnum_ELBO. I found that if the corresponding sites do not be hidden, my custom guide would not work properly.

2

Not sure I understand your question correctly but I think it is better to enumerate discrete variables in the guide if it’s possible.

3

Thanks for your reply! What I did is like the guide AutoDelta which can’t process discrete hidden variables. So I did the same way to hide these discrete hidden variables sites with poutine.block in my custom guide.

4

I find this example rather instructive for the topic of handling discrete sites in the guide.
If I dare to summarise it, in my very superficial understanding of the enumeration machinery, based on the example above, you have two options:

  1. Blocking the sites as you mention in your last message, which relies on the sites being enumerated on the model side. Perhaps, if the model structure allows, you can use the same approach in your custom guide, by omitting the discrete sites.
  2. Sampling the sites in the guide and marking them for enumeration, as @ordabayev suggested. This, I understand, is supposed to perform the enumeration on the guide side.

So to your original question I would suggest it may not be necessary in general, but if you choose to sample them, be sure to enumerate them in the guide too.

5

Thanks for your reply! I see it. Because an unknown error happened when I tried to sample the discrete sites, I have to hide these sites. I’ll try to check the reason of the error and modify my custom guide.

6

The error I got in my custom guide is as follows:

/home/ubuntu/Softwares/anaconda3/envs/deep_learning/lib/python3.7/site-packages/pyro/util.py:529: RuntimeWarning: at guide site "state_24", possibly invalid dependency.
  Expected site "state_24" to precede sites "z_c_23"
  to avoid breaking independence of plates "clicked_cards_23"
  RuntimeWarning,

I just write my guide by following mean-field assumption. Can not figure out what happened. :sleepy:

7

I don’t suppose that without seeing your model / guide much more can be suggested here, except for pointing you to this page.
Perhaps you could condense your model and guide to a minimal example reproducing the error, and share it here.