I have converted a part of my frequentist neural network model (
my_frequentist_model) into a Pyro model.
my_frequentist_model consists of main body and an output head, and I only applied Pyro priors to the weights of the output head, since I do not want the weights of the main body to be changed (the main body weights are pre-trained). I used the following code to achieve this:
# convert our `model_gpt2` into a pyro module. module.to_pyro_module_(my_frequentist_model) # convert the output head of `my_frequentist_model` into a # Bayesian layer. for m in my_frequentist_model.output_head.modules(): for name, value in list(m.named_parameters(recurse=False)): setattr(m, name, module.PyroSample(prior=dist.Normal(0, 1) .expand(value.shape) .to_event(value.dim())))
Now, after I execute the code above, if I do
my_frequentist_model(my_input), would Pyro give the output of the
my_frequentist_model based on the new output head weights which are drawn from their prior distribution (
dist.Normal(0,1))? or would the command
my_frequentist_model(my_input) generate the output based on the old frequentist output head weights?