Confusion about reparametrizations

Hi there,

I was looking at Reparameterizers — Pyro documentation, especially the LocScaleReparam. Are reparams only useful for MCMC and not SVI. I was under the impression that for SVI, the distributions are already reparametrized for gradients to flow. If so, is there any benefits to doing reparam for SVI?

reparameterization is an overloaded term. it can refer to pathwise gradients (torch.distributions.Normal.rsample). it can refer to changing geometry via coordinate transforms. the latter is the use in the link above. and yes it can be used for SVI. mean field in different coordinates results in different variational approximations.


I got a further question on guide writing for manual reparams. Supposedly I want to reparam the following code

def model:

rate = pyro.sample(“rate”, dist.Normal(rate_loc, rate_scale))

def guide:

rate = pyro.sample(“rate”, dist.Normal(rate_loc, rate_scale))

If I change the rate(in the model) to

  • rate = pyro.sample(“rate”, dist.Normal(0, 1)) * rate_scale + rate_loc

do I need to change my guide for this?

you probably don’t need to do anything but you’ll get a difference guide after ELBO optimization since the coordinate system is now different