(auto) guides that return predictions

Misc.
1

Hi all,

I am using the following (anti?) pattern to define a guide that returns computations required downstream. Some background: I have a model and a guide, and in both model and guide I need a sub_model to compute some expected values (it’s a non-linear regression model). The sub_model has it’s own parameters which I would like to encapsulate from the outer model (so that I can easily swap the sub_model for something else). I want to use an AutoMultivariateNormal guide for the parameters of the sub_model. However, this guide does not return the output of the sub_model. So what I ended up doing is this (the crux is the guide method)

class SubModel(PyroModule):
    def __init__(self):
        self.auto_guide = AutoMultivariateNormal(self)

    def forward(self, x):
        # sample some parameters, compute tensor y
        # ...
        return y

    def guide(self, x):
        # call the auto_guide, and get a trace
        guide_trace = poutine.trace(self.auto_guide).get_trace(x)
        # then replay the model defined in forward with parameters sampled from auto_guide
        # However, we have to hide pyro sites from the outside world!
        y = poutine.block(poutine.replay(self, guide_trace))(x)
        return y

Now I can use sub_model(x) to compute the prediction required in model, and sub_model.guide(x) to do the same in my guide, while also sampling from the multivariate normal.

My question is: Is this good practice? Does it result in additional computational cost? For instance because the forward method is run twice?

Let me know what you think, or if there are good alternatives. Thanks,

Chris

2

i’m not sure i understand the setup exactly… can you use deterministic to compute the quantity once and share between model and guide?

3

Hi @martinjankowiak
That sounds like a much easier way to do it, but how does this work exactly? Do I write pyro.deterministic("my_result", my_result) in the model, but then how do I get access to it from the guide? According to the documentation, pyro.deterministic requires that you give a value. Also, how does this work with SVI? Isn’t the guide evaluated first, and then the model at each step?

thanks!

Chris

4

yes you’d have to do the calculation in the guide (if that’s possible) and then can retrieve the calculated value in the model with another deterministic invocation

[edit: actually i don’t think the above will work given the way deterministic is currently implemented. perhaps the solution below is best]

you may also be able to use AutoNormalMessenger to similar effect

5

Hi @chvandorp, I’d also recommend looking into AutoNormalMessenger, whose motivating use case was sharing computation between the guide and model. The default behavior is to act like the AutoNormal guide, but you can override .get_posterior() to do different computations, e.g. draw a joint multivariate normal sample.

1 Like