Hi all,
I was looking through some of the source code of the autoguides and noticed whenever a prior has constrained support (for example, values from a gamma distribution must be positive), the Diagonal/MultivariateNormal autoguides are adjusting the log prob of the posterior sample (relative to the posterior distribution) by calling the log_abs_det_jacobian method of the prior distribution.
Specifically, it’s retaining the usual log_prob from the multivariate Normal (for use in constructing the ELBO in inference). But then, since we want the model and guide to have the same sites (and the model presumably has a bunch of univariate priors instead of a multivariate), its creating pyro.samples with Delta distributions set to what was sampled from the multivariate and adding log_density adjustments if the support in the prior is constrained (if its not constrained, the log_abs_det_jacobian returns 0).
So, I was wondering… is there some easy-to-remember rule about when this type of adjustment is necessary when we’re creating custom guides? For example…
-
Do we need to use this kind of log_abs_det_jacobian adjustment when the prior and posterior support are both constrained in the same way but through different distributions? (e.g., a gamma prior and exponential posterior)
-
Do we only need to use log_abs_det_jacobian adjustments if the prior and posterior are of different distributions, so we never use them if they’re of the same distribution?
-
Are log_abs_det_jacobian adjustments even something I should be thinking about when building custom guides? Or am I thinking about this incorrectly?
Thanks for your help!!