Training SVI vs. Pyro model

Misc.
1

Hello,

I am sorry for asking lots of questions.
I have designed a training loop to train my SVI as below:

# set SVI
svi = SVI(myPyroModel, guide, optimizer, loss=Trace_ELBO())

# simple training loop
for i in range(num_iter):
     svi.step()

My question is, when I train my svi in this way, does the guide and myPyroModel inside the call svi = SVI(myPyroModel, guide, optimizer, loss=Trace_ELBO()) also get trained and modified appropriately?

Thank you,

2

This optimizes parameters in your guide (by working to decrease the negative ELBO which is equivalent to decreasing the KL Divergence between the guide and model). When training is completed, you can use your guide to generate data or look at the values of the parameters in the Pyro parameter store. See the last sentence in this section:

https://pyro.ai/examples/intro_part_ii.html#Parametrized-Stochastic-Functions-and-Variational-Inference

Note that optimization will update the values of the guide parameters in the parameter store, so that once we find good parameter values, we can use samples from the guide as posterior samples for downstream tasks.

1 Like
3

Thank you!

4

Thank you for this!

so after I train the guide, would I be able to calculate the prediction scores by doing infer.Predictive(myPyroModel, guide=trained_guide, num_samples = 100)?

Thank you again,