The easest way is to see numpyro
source it is much simpler. The predictive can be found here. In essence first the guide is sampled for each parameter. Then using these samples the model is evaluated.
If we look at Bayes’ the predictive distribution is the probability of observing new x if we’ve previously seen trained data \mathbf{X} or:
p(x|\mathbf{X}) = \int_z p(x|z)p(z|x) dz.
Since we don’t know the true posterior p(z|x) but the approximation through the guide q(z) then the above relation becomes:
p(x|\mathbf{X}) = \int_z p(x|z)p(z|x) dz \approx \int_z p(x|z) q(z) .
In order to solve the above integral, pyro uses monte carlo through the Predictive
class. As I wrote in the first paragraph, it first samples from the guide i.e. q(z) then runs these through the model thus evaluating p(x|z) and does integration.
So the function forward
is actually called during predictive. Check poutine.condition
on how this can be done by hand.
Regarding the second question scale
for Gaussian distribution is actually the stadard deviation. So 0.1 is 10% around the mean which is like a rule of a thumb good starting choice. You can do your initiation as you wish, you can customize this, check https://docs.pyro.ai/en/stable/infer.autoguide.html. As you can see, you can use a function that determines initial values.