I am going through the Bayesian Linear Regression Tutorial. While reparameterising the model the centeredness has been directly taken to be 0, I wish to use SVI to find the best centeredness in this case. How can I do it?
try the docs? i believe you want centered=None
Yes, I want centered = None, but then how should I use SVI to get the required (optimized) centeredness for the different parameters in the hierarchical model?
if you reparameterize a latent variable named mylatent
and you get the params after training an SVI
instance:
svi = SVI(...)
svi_result = svi.run(...)
params = svi_result.params
the dictionary params
should contain a parameter keyed by mylatent_centered
Thank you @martinjankowiak. I tried out the way you told me but for some reason it does not seem to work. I am trying to implement the Radon gas example. I am further trying to implement partial centering using LocScaleReparam. The model code is as follows:
def model_reparam(county_idx = county, log_radon = log_radon, floor = floor_measure, J=J, N=N):
sigma_y = numpyro.sample("sigma_y", dist.Normal(0.0, 1.0))
sigma_beta = numpyro.sample("sigma_beta", dist.Normal(0.0, 1.0))
sigma_alpha = numpyro.sample("sigma_alpha", dist.Normal(0.0, 1.0))
mu_alpha = numpyro.sample("mu_alpha", dist.Normal(0.0, 10))
mu_beta = numpyro.sample("mu_beta", dist.Normal(0.0, 10))
with numpyro.handlers.reparam(numpyro.plate('J', J), config={'alpha': LocScaleReparam(None)}):
alpha = numpyro.sample("alpha", dist.Normal(mu_alpha, sigma_alpha))
with numpyro.handlers.reparam(numpyro.plate('J', J), config={'beta': LocScaleReparam(None)}):
beta = numpyro.sample("beta", dist.Normal(mu_beta, sigma_beta))
with numpyro.plate("N", N):
mu = alpha[county_idx] + beta[county_idx] * floor
numpyro.sample("obs", dist.Normal(mu, sigma_y), obs=log_radon)
Then to use SVI to get the required centeredness I am using SVI as follows:
guide = AutoDiagonalNormal(model_reparam)
svi = SVI(model, guide, numpyro.optim.Adam(3e-3), Trace_ELBO(5))
svi_results = svi.run(rng_key, 10000)
loss = svi_results.losses[-1]
return loss
Upon running this, I am getting the following error:
IndexError: Too many indices for array: 1 non-None/Ellipsis indices for dim 0.
Please help me
since you haven’t provided a full error traceback nor a complete runnable script it’s hard to help.
i just confirmed on a simple model that a *_centered
parameter should appear in the params
dict after SVI training.
I am not entirely sure, but it seems like the model you passed for creating the guide and the model you used in SVI are different (based on the variable name). Ideally, both of them should be the same.
It seems like there is no tutorial or example on partial centering of hierarchical models. If you guys agree, I’d like to contribute an example/tutorial on the same. Please let me know.