I am new to pyro but want to learn more about the library as well as Bayesian Neural Networks in general. Based on the example in the docs I created a google colab notebook to play around with different toy regression datasets to get a feel for using BNNs with MCMC or VI.
I noticed that on the small sine dataset, MCMC is performing better than VI but also not “perfectly”. Therefore, I was wondering if I made a mistake in my code or if there are any other training tips especially for VI, sort of pitfalls or things to look out for (also looking ahead when going beyond toy regression datasets).
