How does plate nesting work in enumeration

Misc.
1

In the Inference with Discrete Latent Variables tutorial (Inference with Discrete Latent Variables — Pyro Tutorials 1.8.6 documentation), the explanation says “there is no way to linearly order x and y such that one’s plate nesting is less than the other”.

What does that mean exactly? I tried the code within the invalid_model and it didn’t throw an error, does it mean the error will only occur if we try to use TraceEnum_ELBO? Would the invalid_model be okay if we stick with Trace_ELBO (on that note, how bad is Trace_ELBO for discrete variables, is the difference only on computational efficiency)?

plate_1 = pyro.plate("plate_1", 10, dim=-1)  # vectorized
plate_2 = pyro.plate("plate_2", 10, dim=-2)  # vectorized
with plate_1:
    x = pyro.sample("x", Bernoulli(0.5))
with plate_2:
    y = pyro.sample("y", Bernoulli(0.5))
with plate_1, plate_2:
    z = pyro.sample("z", Bernoulli((1. + x + y) / 4.))

I’m quite confused. Thank you very much for reading this! (And is there a tutorial that explains the difference between vectorized plate and sequential plate?)

image
image1081×1278 161 KB

2

Yes

The enumeration restrictions are checked by TraceEnum_ELBO and will result in an error if violated

Yes

Quote from Gaussian Mixture Model — Pyro Tutorials 1.8.6 documentation

Mathematically, guide-side enumeration simply reduces variance in a stochastic ELBO by enumerating all values, whereas model-side enumeration avoids an application of Jensen’s inequality by exactly marginalizing out a variable.

Here:
http://pyro.ai/examples/svi_part_ii.html?highlight=sequential%20plate#Marking-Conditional-Independence-in-Pyro

3

Got it! Thank you so much!!