Custom Guides and log_abs_det_jacobian Question

Hi all,

I was looking through some of the source code of the autoguides and noticed whenever a prior has constrained support (for example, values from a gamma distribution must be positive), the Diagonal/MultivariateNormal autoguides are adjusting the log prob of the posterior sample (relative to the posterior distribution) by calling the log_abs_det_jacobian method of the prior distribution.

Specifically, it’s retaining the usual log_prob from the multivariate Normal (for use in constructing the ELBO in inference). But then, since we want the model and guide to have the same sites (and the model presumably has a bunch of univariate priors instead of a multivariate), its creating pyro.samples with Delta distributions set to what was sampled from the multivariate and adding log_density adjustments if the support in the prior is constrained (if its not constrained, the log_abs_det_jacobian returns 0).

So, I was wondering… is there some easy-to-remember rule about when this type of adjustment is necessary when we’re creating custom guides? For example…

  1. Do we need to use this kind of log_abs_det_jacobian adjustment when the prior and posterior support are both constrained in the same way but through different distributions? (e.g., a gamma prior and exponential posterior)

  2. Do we only need to use log_abs_det_jacobian adjustments if the prior and posterior are of different distributions, so we never use them if they’re of the same distribution?

  3. Are log_abs_det_jacobian adjustments even something I should be thinking about when building custom guides? Or am I thinking about this incorrectly?

Thanks for your help!!

After looking at the code a bit closer, I think I might understand why its making the log_abs_det_jacobian adjustments now…

It seems like a multivariate sample in the autoguide is actually meant to represent an unconstrained value of the latents… this is in contrast to most custom guides that sample true/constrained values (and then convert them to unconstrained in the ParamStore). So, if I’m reading the implementation correctly, the log_abs_det_jacobian adjustments should be unnecessary in most custom guides unless the guide specifies a variational distribution that is meant to sample an unconstrained value of a constrained variable.

yes log det jacobians are used to account for transformations between unconstrained and constrained spaces (which changes the density by a jacobian factor)