I was trying to generate sampled neural networks from the guide() function by specifying the distribution for the weights and biases in the guide() without calling SVI(). However, the weights and biases of the resulting neural network did not follow the specified distribution. My understanding is that the distribution in guide() is the initial distribution and will get updated as svi.step() is called, where svi is assigned with SVI(model, guide, optim, loss=Trace_ELBO()) for example. If we do not call svi.step(), then the initial distribution in guide() should be the one used for generating the instances of neural networks. Please correct me if I am wrong. My code is as follows.
def guide(x_data, y_data):
priors={}
for name, param in myNet.named_parameters():
param1 = dic_mean[name] # dic_mean is a dictionary storing the means of weights
stdev = param1 * 0 # so that the weights in the generated instances will be the same as in dic_mean
priors[f'{name}'] = Normal( loc=pyro.param(f'{name}_mu', param1), \
scale=pyro.param(f'{name}_sigma', stdev) )
lifted_module = pyro.random_module("module", myNet, priors)
return lifted_module()
sampled_net = guide(None, None)
The values of weights in sampled_net are totally different from those stored in dic_mean.
I also tried to define model() and link model and guide via SVI(). The problem was the same. Thanks.