I used LocScaleReparam, which is based on Automatic Reparameterisation of Probabilistic Programs*, Maria I. Gorinova, Dave Moore, Matthew D. Hoffman (2019). The documentation for LocScaleReparam in case of centeredness mentions that: " If None (default) learn a per-site per-element centering parameter in [0,1]", but upon looking through the implementation of the algorithm, I did not find anything relevant to finding the correct centeredness (as far as I understand the implementation by default takes the centeredness to be 0.5 and does not optimize it). Is there some way to get the optimized centeredness for a given model, as mentioned in the paper using the current implementation?
The foo_centered parameter would be optimized during SVI run. Please file a bug with repro code otherwise.
So, does that mean I must run an SVI run explicitly apart from the LocScaleReparam to actually get the best centeredness?
Yes, currently in numpyro, only SVI supports optimizing parameters. If you want to sample (rather than optimizing), you might want to use lift handler.
When SVI runs on the model where the LocScaleReparam is set to None, we obtain auto_loc, and auto_scale along with theta_centered. In the reparameterization theta_centered is used. But, then what is the use of auto_loc and auto_scale?
As an example, with Normal(0, 1), loc is 0 and scale is 1. auto_loc is loc of your Normal posterior, same for auto_scale
1 Like
Oh, ok do you mean that the auto_loc and auto_scale represent the mean and scale of the surrogate posterior used in VI, whereas theta_centered is associated with parameterization?
yes, auto loc scale refers to posterior of the decentered noise. The formula can be found in the continuous relaxation section 5.2 of the paper.
1 Like