Edward tutorial example of Bayesian neural network in Pyro

Misc.
2

Hi, glancing at your code I can see a few issues:

  1. You don’t need any observed sites in your guide - you can remove the final loop entirely
  2. You need to make sure parameter values aren’t shared in your model and guide: qw0_loc = pyro.param("qw0_loc", lambda: params['w0_loc'].clone())
  3. You don’t need to softplus your scale parameters if you’ve already constrained them to be positive via constraints.positive
  4. Bayesian neural networks are sensitive to initialization just like regular neural networks, you might try narrowing your prior and initial variational distributions e.g. by rescaling all the initial scale parameters by 0.5
  5. Your model would run much faster if you vectorized over data with pyro.plate rather than using a for loop

See this other recent topic for pointers to BNN examples in Pyro.