Working with AutoLaplaceApproximation

1

I’m struggling to work effectively with models that are using the AutoLaplaceApproximation guide. Unlike the (mutlivariate) normal approximations, you cannot just pull quantiles from the estimated posterior directly, and I am unable to find usage examples in the docs.

I did find the following approach for sampling from the posterior here:

guide = autoguides.AutoLaplaceApproximation(my_model)

<fit model>

laplace_guide = guide.laplace_approximation()
pred = pyro.infer.Predictive(laplace_guide, num_samples=1000)

but it does not work directly with my_model in my case because the model itself takes 4 tensors of data as arguments. However, when I try and pass the data to the laplace_approximation method:

laplace_guide = guide.laplace_approximation(*data)

I get a RuntimeError

RuntimeError: One of the differentiated Tensors does not require grad

Does anyone have a link to a fully worked example? That is, something that shows how to extract inference from a fitted model using the AutoLaplaceApproximation or similar guides.

2

Hi @fonnesbeck, it is hard to guess why the error happens without repro code. Here is a working example for your reference

import pyro
import pyro.distributions as dist
from pyro.infer import Trace_ELBO, SVI, autoguide
import torch

def model(x):
    a = pyro.sample("a", dist.Normal(0, 1))
    pyro.sample("obs", dist.Normal(a, 1), obs=x)

guide = autoguide.AutoLaplaceApproximation(model)
infer = SVI(model, guide, pyro.optim.Adam({"lr": 0.1}), Trace_ELBO())
infer.step(torch.tensor(1.))
laplace_guide = guide.laplace_approximation(torch.tensor(1.))
print(laplace_guide.quantiles([0.3, 0.7]))
3

Thanks. So, just to be clear the torch.tensor(1.) here represents the data, and in order to get the quantiles, you have to supply the input data again after the model is fit?

4

That’s right. We don’t store data in SVI because the input of SVI can change (e.g. when data flows into svi in batches). For Laplace approximation, we can store data in AutoLaplaceApproximation class, but it might be more flexible to let users choose which inputs needed to calculate the hessian…