If I understand correctly what is going on in auto guides such as
AutoNormal, only the
sample sites of the numpyro model get corresponding mean and width parameters associated to them. I am currently working with a model based on both
param sites, and I would like to model the posterior distribution of all these sites.
Hence, before trying to adapt the
numpyro source code for what I am looking for, I wanted to make sure: is there some mathematical reason why auto guides do not create posterior parameters for
it doesn’t really make sense to ask for a posterior distribution for a
param site—it doesn’t have a prior associated with it.
param basically means “this is a point estimate quantity with no associated uncertainty”
In case you want to set priors for param sites, you can use
I agree with you in the general case. However I have to use
param sites as I am imposing some regularization on certain parameters in a way that cannot be expressed as a simple prior distribution (specifically, sparsity via l1-norm in the wavelet domain). But so far I don’t see why this should prevent me from sampling from these sites and model their posterior distribution within SVI.
Thanks for your answer though! The thing is that my use of these
param sites is not really the normal use, which I agree with what you say should mean “no uncertainty”. I use these sites as a way to impose my “custom prior” by hand, using the numpyro’s
Hmm that’s interesting. I completely missed this handler, but it may indeed help me. After optimizing the
param sites, I could lift them and associate some analytical prior distribution to then sample using SVI. Thanks a lot!