I tried to follow Bayesian Regression tutorial architecture to implement a more complex LSTM model.
In the model, I use two LSTM layers as encoder, with hidden units 128 and 64 separately. And I use pyro.random_module to lift the weights to stochastic variables. I use independent(1) in the code, like
priors_dist[layer_name] = pyro.distributions.Normal(weights_loc, weights_scale).independent(1).
However running the code I face
RuntimeError: The size of tensor a (512) must match the size of tensor b (256) at non-singleton dimension 0.
I found the error occurred at
class Trace_ELBO -->
def _compute_log_r(model_trace, guide_trace)-->
The reason is that at first layer the log_r_term size is (512, ), but second layer the log_r_term size change to (256, ), and in the
log_r.add function it will face shape mismatch error.
It seems that I have to use
independent(2) to eliminate the log_r_term shape mismatch. Is there some better ways to solve the problem?