Dependency tracking in Pyro

Tutorials
3

Thank you. So I think I am starting to understand a bit better. I have a few (possibly repetitive) follow up questions below, but I think if I could confirm the answers to them it would really help solidify my understanding.

With respect to understanding how plate works - adapting the last scenario from my prior message:

  with pyro.plate("my_plate1", 2):
        a =  sample('a', Bernoulli(0.5))
        b = sample('b',  Bernoulli(0.5))
        with pyro.plate("my_plate2", 2):
            c = sample('c', Bernoulli(0.5))

  • From Pyro’s perspective:

    • Can a[0] and b[0] be treated as independent during inference? My guess is no.
    • Can a[0] and a[1] be treated as independent during inference? My guess is yes.
    • Can a[0] and c[0] independent as during inference? My guess is yes.

  • If I want to declare to Pyro that 2 Bernoulli random variables can be treated as independent during inference, are the following 3 scenarios equivalent:

for i in pyro.plate("my_plate", 2):
      sample("b_{}".format(i), Bernoulli(0.5))

with pyro.plate("my_plate", 2):
      sample("b", Bernoulli(tensor([0.5,0.5]))

with pyro.plate("my_plate1"):
      sample("b_1", Bernoulli(0.5))

     with pyro.plate("my_plate2"):
          sample("b_2", Bernoulli(0.5))

  • From an underlying implementation perspective, is there any truth to what I was saying about Pyro using the batch dimensions across sample statements as the way it tracks what it can treat as independent?

  • Is it true that the call to .independent() is a bit of a different construct than plate- it is more about reshaping a particular distribution (which could involve declaring some samples as independent), but it doesn’t handle independence across sample statements?

Thanks so much for your help!