Okay thank you very much for your answer, but I don’t see the use of having a neural network if you don’t add non linearities with your activation functions. In this case I would just be doing a linear regression of the forward of my guide (but my model, hence my prior would still be extremely complex because of all the layers and all the activation functions). Thus I would like to define a more complex guide with activation functions. I thought of an Easy Guide but here is my main problem : I am using a Pyro Module that uses PyroSample, (and not pyro.sample where you can name your parameters). Thus how should i do to have a “better” guide (here is the link the detailed question I posed i the forum Create an EasyGuide for a PyroModule model, SVI )
Have a great day,
Rémi
PS : When I look at the parameters of my guide after training, the paramX_loc seem to respect some kind of gaussian distribution but my paramX_scale all have the same value, which isn’t the case if i use a AutoLowRankMultivariateNormal and I don’t understand well why is that ? How come the fact that adding dependencies allow all the standard deviations to be different where removing dependencies between my weights gives them the same value ?