The guide() is the variational distribution, which approximates the posterior distribution. When the SVI optimization is done, how do we check the final ‘optimized’ variational distribution?
To be concrete, let’s say we have the following guide
, where myNet
is an instance of a neural network:
def guide(x_data, y_data):
priors={}
for name, param in myNet.named_parameters():
if param.requires_grad:
mu = torch.randn_like(param)
priors[f'{name}'] = Normal( loc=pyro.param(f'{name}_mu', torch.randn_like(param)), \
scale=softplus( pyro.param(f'{name}_sigma', torch.randn_like(param)) ) )
lifted_module = pyro.random_module("module", myNet, priors)
return lifted_module()