Plate structure in model v. guide

Misc.
1

Hello there!

I am new to Pyro, and I wish to perform variational inference on a Bayesian network. Its nodes are indexed by a time j and a station i. In the code below, BN is the Networkx graph object containing my network. BN also contains the parameters defining the conditional distributions for each node n = (i, j) (they are linear gaussian).

 def BN_model():
    values = {}
    for j in pyro.markov(times):
        for i in pyro.plate("time_{}".format(j), len(stations)):
            n = (i, j)
            parents = list(BN.predecessors(n))
            loc = (
                BN.nodes[n]["mu"] +
                sum(BN.edges[p, n]["weight"] * values[str(p)] for p in parents)
            )
            scale = BN.nodes[n]["sigma"]
            values[str(n)] = pyro.sample(str(n), dist.Normal(loc=loc, scale=scale))
    return values
    
def BN_guide():
    nodes = list(BN.nodes())
    for k in pyro.plate("nodes", len(nodes)):
        n = nodes[k]
        loc = pyro.param("loc_{}".format(n), torch.tensor(0.))
        scale = pyro.param("scale_{}".format(n), torch.tensor(1.))
        pyro.sample(str(n), dist.Normal(loc=loc, scale=scale))

When using SVI on this, the inference works but I get a warning I do not understand:

UserWarning: Found plate statements in guide but not model: {'nodes'}

It seems this warning is due to the fact that I use different independence structures in my model and guide. However, that is precisely what I want to do:

  • In the model, I want all the variables in a given time to be mutually independent conditionally on the past
  • In the guide, I want all variables in the entire network to be mutually independent, which is why I have a different plate loop.

Did I do something wrong here? Or do I fail to grasp the meaning of the plate?
Thanks in advance for any piece of advice.
Giom

2

Now that I have more flight hours with Pyro, my understanding is that the plates of the guide should be present in the model so as to allow subsampling.
Sorry for the silly question, but I hope it may help someone someday :slight_smile:

1 Like
3

Is it okay though to have different plates if I only perform subsampling on those plates that are shared by model and guide?

4

My problem is that my models contains a pyro.markov for loop, which I’d like to replace with a pyro.plate in the guide, therefore the plate’s name would be unknown to the model.

On a side note, does anyone know of a trick to vectorize sampling of variables with different names (in this case the "z_{}".format(t) of my markov loop)?

1 Like