rewriting some pyro code and saw that my belowed random_module was being phased out. Now Im trying to understand the pyroModule.
How Ive usually worked is to create a standard pytorch nn.Module and then created the two functions model() and guide() to define priors, likelihood and posteriors. With some help of net.named_parameters() its quite simple to create a mean-field model. See an example e.g. here. A new property of this combined with random_module is that I can get a posterior sample of my model by simply calling the guide functions.
So a general question is how one should build a full system using PyroModule instead of randomModule.
In specific, Im trying to modify a full network instead of a torch layer. However, this seems to break down. I suspect I just dont understand the purpose of PyroModule yet, but what is the best practice here?
TypeError: cannot assign 'pyro.nn.module.PyroSample' as parameter 'weight' (torch.nn.Parameter or None expected)
class Network(nn.Module): def __init__(self, in_features, out_features): super(Network, self).__init__() self.in_features=2 self.out_features=1 self.linear = nn.Linear(self.in_features, self.out_features) def forward(self, x, y=None): mean = self.linear(x) return mean class RandomNetwork(Network, PyroModule): def __init__(self, in_features, out_features): super().__init__(in_features, out_features) self.linear.weight = PyroSample( lambda self: dist.Normal(0, 1) .expand([self.out_features, self.in_features]) .to_event(2)) def model(x,y): pass rand_net = RandomNetwork(2,1)