Dear Pyro-Forum,
I have a follow-up question regarding the same model. I would like to compute my posterior predictive’s neg. log-likelihood, i.e. the probability of observing the data calculated under the posterior predictive’s log-density.
I understand this can be done similar to this post or this example. However, this involves annotating the prior definitions in the model using plate statements. Unfortunately I haven’t quite understood how this would work, especially in conjunction with PyroSample.
Currently, when executing:
pred = Predictive(model=model, guide=variational_density, num_samples=10)
pred.get_vectorized_trace(data)
I am getting the following error:
.local/lib/python3.6/site-packages/torch/nn/functional.py", line 1370, in linear
ret = torch.addmm(bias, input, weight.t())
RuntimeError: t() expects a tensor with <= 2 dimensions, but self is 3D
Trace Shapes:
Param Sites:
Sample Sites:
linear.weight dist 10 1 | 1 2
value 10 | 1 2
linear.bias dist 10 1 | 1
value 10 | 1
I have tried to change the prior definition to:
with pyro.plate('x_axis', size=in_features):
with pyro.plate('y_axis', size=out_features):
self.linear.weight = PyroSample(
prior=dist.Normal(weight_loc, weight_scale)
)
with pyro.plate('bias', size=out_features):
self.linear.bias = PyroSample(
prior=dist.Normal(bias_loc, bias_scale)
)
in order to get independent batch dimensions, i.e. assume all weights to be i.i.d. However, this leads to:
.local/lib/python3.6/site-packages/pyro/util.py", line 288, in check_site_shape
'- .permute() data dimensions']))
ValueError: at site "linear.weight", invalid log_prob shape
Expected [], actual [1, 2]
Try one of the following fixes:
- enclose the batched tensor in a with plate(...): context
- .to_event(...) the distribution being sampled
- .permute() data dimensions
Any help is much appreciated.
Regards!