Hi all,
Im looking at the Bayesian regression tutorial here
https://pyro.ai/examples/bayesian_regression.html
And trying to figure out if there is a way to assign different priors for each of the network weights in the nn.Linear layer.
Obviously, this is a bit pedantic if you are working with some huge deep neural network, but seems like a pretty reasonable request if you are just doing Bayesian linear regression like in the example, where you just have say, three weights you are trying to get the posterior of.
Can I still use this example with using nn.PyroModule and nn.PyroSample to specify my model? Or do I have to specify the model in a different way in order to specify these different priors.
cheers!
If by different priors you mean the same distribution (e.g. Normal) with different parameter values, then you can just provide tensors of parameters to Normal:
...
# here assume out_features == in_features == 2
weight_loc = torch.tensor([[0.5, 1.5,], [2, 2.5]])
weight_scale = torch.tensor([[0.1, 0.3], [0.4, 0.2]])
self.linear.weight = PyroSample(dist.Normal(weight_loc, weight_scale).to_event(2))
...
If you mean different distribution types (e.g. Normal vs Cauchy), you’ll need to write a separate sample statement for each type and express the linear operation in terms of those values in your model, as in the NumPyro Bayesian regression tutorial.
1 Like