Hi,
I looking at the descriptions of the auto guide module. But I’m not so sure that I understand what the difference is between the auto Laplace approximation guide and the auto multivariate normal guide.
Could someone maybe explain that to me or point me to some references?
Best,
R
the latter is used in the context of variational inference: the parameters are fit by maximizing the variational objective (the evidence-lower bound or ELBO).
the former is fit by finding the maximum a posteriori (MAP) estimate and then using the local curvature at the MAP estimate to define a normal covariance matrix
Thank you for your reply!
But what if I use the AutoLaplaceApproximation guide in the variational inference context? Is it not fitted by the maximization of ELBO even though I specified it this way:
guide = pyro.infer.autoguide.AutoLaplaceApproximation(model)
optim = pyro.optim.Adam({“lr”: 0.01})
svi = pyro.infer.SVI(model, guide, optim, loss=pyro.infer.Trace_ELBO(), num_samples=1000)
MAP can be seen as degenerate limit of the ELBO so you can still fit it with SVI.
note however that you needn’t specify num_samples here—the latent distribution is a dirac delta distribution.
see the usage example.