SVI version of mcmc.get_samples()

Hi,

I just started to experiment with svi to get some increase in inference speed. However, what leaves confused is how I can obtain the samples for each parameter. The only params I can see is auto_loc & auto_key, which I interpret as what the loc & scale for each model parameter would be.

Seems like I’m missing a big point here :relaxed:

1 Like

You can use Predictive with the guide keyword argument, although that also runs the model. If you’re using an autoguide, you can just call it - autoguides in Pyro and NumPyro return dictionaries of samples.

1 Like

Perfect! :relaxed:

Hm, what confuses me is that using guide I get my outcome + deterministics. If I then let that feed as posterior_samples into Predictive(), I get all my parameter estimations + deterministics.

The latter one is what I think of as posterior samples. The first I’m not so sure about, posterior predictive perhaps?

Reading the docs for Predictive() I see this:
“This class is used to construct predictive distribution. The predictive distribution is obtained by running model conditioned on latent samples from posterior_samples.”

And I’m after posterior_samples

I’m not quite sure what you’re asking, but the values returned by Predictive for latent variables will be samples from the guide, i.e. posterior samples.

With Predictive(model, guide = guide, num_samples = 2000),
I’m getting posterior samples for y in the formula y = a + bx + cx^2 and I am after the posterior samples of the parameters a, b, c.

However, if I use Predictive(model, params = svi_result.param, num_samples = 2000), I get a dict with samples for a, b, c & y

You can use the argument return_sites to decide which sites will be returned. By default,

  • Predictive(model, guide=guide, num_samples=2000) will return latent/deterministic sites in model which are not appeared in guide. This is typically used to get posterior predictive distribution.
  • Predictive(guide, num_samples=2000) will return latent/deterministic sites in guide. This is used to get posterior samples.
  • Similarly, Predictive(model, num_samples=2000) will return latent/deterministic sites in model.

If you want to get both posterior samples and their deterministic transformed values in model, you can use Predictive(model, guide=guide, num_samples=2000, return_sites=list_of_latent_and_deterministic.

3 Likes

Thanks for explaining!
This is also solved my hierarchical issue, in my previous attempts, it looked like it didn’t learn individual parameters within groups :relaxed:

One question though, what is _auto_latent that’s returned in the posterior samples?

It is specific to auto guides. Because the latent variables could have constrained supports (so Normal guide does not make sense), under the hood, we transform the model to some sort of “unconstrained” model with a single “concatenated” latent variable named _auto_latent. For auto guides, instead of using Predictive, I think you can use sample_posteriors method to get the posterior. :slight_smile:

3 Likes