The point you may be getting stuck on is the distinction between the underlying true conditional independence relationships in a probabilistic model and the subset of independence relationships that inference algorithms are aware of and can exploit. Tracking dependence in general probabilistic programs where some random variables may affect the existence of others is a difficult and subtle problem, so Pyro is deliberately conservative in its assumptions about independence in order to avoid introducing inference errors.
The answer to each of your examples is the same: the independence relationships you’re asking about do in fact hold for the models you’ve provided, but from Pyro’s point of view, only the independence relationships explicitly declared by pyro.plate
or implied by temporal ordering (i.e. in your final example, b[:, 0]
depends on a[0]
and not vice versa) are assumed to hold.
This is obviously suboptimal for deriving efficient inference algorithms, so a lot of our ongoing work in Pyro, especially on enumeration, involves identifying and exploiting conditional independence more aggressively.