It's a little awkward, but you can use a second
with pyro.plate("my_plate1", 2):
for i in pyro.plate("my_plate2", 2):
if i == 0:
a = pyro.sample('a', Bernoulli(0.5))
elif i == 1:
b = pyro.sample('b', Normal(0., 1.))
Ideally dependency tracking at this level of granularity will eventually be done automatically instead of making the user provide such detailed annotation. Also note that some inference algorithms in Pyro (those that exploit parallel enumeration or reparameterized gradient estimators) already have access to finer-grained dependency structure via PyTorch.