Sampling from fitted guide

[Jameson]

Is there a simple cookbook recipe for sampling from the fitted guide after an svi object has finished running/fitting?

I see:

Sampling from posterior predictive

Hello, first off, amazing job on Pyro! Major kudos How do I sample from the posterior predictive for an SVI-trained model efficiently? At the moment, I sample a guide trace for each desired posterior predictive sample, replay the model with the guide trace, and sample once from it, like this: ppc = [] dummy_obs = torch.zeros((1,self.D)) for sample in range(n_samples): guide_trace = pyro.poutine.trace(self.guide).get_trace(dummy_obs) posterior_predict…

Posterior sampling after model fitting

In [12]: # posterior predictive distribution we can get samples from trace_pred = TracePredictive(wrapped_model, posterior, num_samples=1000) post_pred = trace_pred.run(x_data, None) How do I generate samples with a different size? I tried changing “x_data -> x_data[:70,:]” in In [12] which seems to be reasonable to me but returns an error complaining about the shape of the input. Also, I guess there is a typo in …

http://pyro.ai/examples/bayesian_regression.html#Validating-Results
http://docs.pyro.ai/en/stable/inference_algos.html?highlight=%20TracePosterior#module-pyro.infer.svi

, and others, but they all seem to have some additional complexity, such as focusing on the posterior predictive distribution of observations, not just the fitted guide distribution of parameters.

Also: once I’ve fitted an svi object for n steps, does it contain only the latest guide parameters, or a record of all the guide parameter values it took over the course of optimization? I’d rather it was the latter but from the docs it’s unclear to me.

Sorry for the noob-like question but I figure if this is confusing to me it’s probably so to others. It would be good to have a simple cookbook answer for this question because I’m pretty sure it’s a common enough use case.

[gdalle]

Hi there!
I’m no pro, but from what I understand:

• If you code your guide function so that it returns the hidden variables after sampling them, all you would have to do after SVI terminates is to call guide(...). That will generate hidden variables according to the latest value of variational parameter phi.
• I’m not sure what the SVI object itself contains, but I think the parameters (for both model and guide) are stored in a global object called the parameter store (Parameters — Pyro documentation), which means their values are updated at each iteration. This also explains that calling guide(...) after the end of SVI works as I suggested: through the pyro.param(...) statement, your guide looks up what the latest value of the parameter in the store is (or creates the value the first time around) and uses it for simulation.
  Does that answer your questions?

[Jameson]

Yes, thanks.