How to apply SVI with Hierarchical model in case of LocScaleReparam

numpyro
1

I am going through the Bayesian Linear Regression Tutorial. While reparameterising the model the centeredness has been directly taken to be 0, I wish to use SVI to find the best centeredness in this case. How can I do it?

2

try the docs? i believe you want centered=None

https://docs.pyro.ai/en/stable/infer.reparam.html?highlight=LocScaleReparam#pyro.infer.reparam.loc_scale.LocScaleReparam

1 Like
3

Yes, I want centered = None, but then how should I use SVI to get the required (optimized) centeredness for the different parameters in the hierarchical model?

4

if you reparameterize a latent variable named mylatent and you get the params after training an SVI instance:

svi = SVI(...)
svi_result = svi.run(...)
params = svi_result.params

the dictionary params should contain a parameter keyed by mylatent_centered

1 Like
5

Thank you @martinjankowiak. I tried out the way you told me but for some reason it does not seem to work. I am trying to implement the Radon gas example. I am further trying to implement partial centering using LocScaleReparam. The model code is as follows:

def model_reparam(county_idx = county, log_radon = log_radon, floor = floor_measure, J=J, N=N):
    sigma_y = numpyro.sample("sigma_y", dist.Normal(0.0, 1.0))
    sigma_beta = numpyro.sample("sigma_beta", dist.Normal(0.0, 1.0))
    sigma_alpha = numpyro.sample("sigma_alpha", dist.Normal(0.0, 1.0))

    mu_alpha = numpyro.sample("mu_alpha", dist.Normal(0.0, 10))
    mu_beta = numpyro.sample("mu_beta", dist.Normal(0.0, 10))

    with numpyro.handlers.reparam(numpyro.plate('J', J), config={'alpha': LocScaleReparam(None)}):
        alpha = numpyro.sample("alpha", dist.Normal(mu_alpha, sigma_alpha))

    with numpyro.handlers.reparam(numpyro.plate('J', J), config={'beta': LocScaleReparam(None)}): 
        beta = numpyro.sample("beta", dist.Normal(mu_beta, sigma_beta))

    with numpyro.plate("N", N):
        mu = alpha[county_idx] + beta[county_idx] * floor
        numpyro.sample("obs", dist.Normal(mu, sigma_y), obs=log_radon)

Then to use SVI to get the required centeredness I am using SVI as follows:

guide = AutoDiagonalNormal(model_reparam)
svi = SVI(model, guide, numpyro.optim.Adam(3e-3), Trace_ELBO(5))
svi_results = svi.run(rng_key, 10000)
loss = svi_results.losses[-1]
return loss

Upon running this, I am getting the following error:

IndexError: Too many indices for array: 1 non-None/Ellipsis indices for dim 0.

Please help me

6

@martinjankowiak, @fehiepsi could you please suggest the possible solution for this problem?

7

since you haven’t provided a full error traceback nor a complete runnable script it’s hard to help.

i just confirmed on a simple model that a *_centered parameter should appear in the params dict after SVI training.

1 Like
8

I am not entirely sure, but it seems like the model you passed for creating the guide and the model you used in SVI are different (based on the variable name). Ideally, both of them should be the same.

1 Like
9

It seems like there is no tutorial or example on partial centering of hierarchical models. If you guys agree, I’d like to contribute an example/tutorial on the same. Please let me know.