Put more weightage on likelihood than the prior

numpyro
1

Hello team, I am trying to use SVI in numpyro to train a neural network. Since I do not know too much about the weights and biases of the neural network a-priori, I want SVI to focus more on the likelihood than on the prior. Is there a way to achieve that? I read somewhere that:

  1. I can increase the spread of the likelihood distribution
  2. I can use numpyro.factor to scale my likelihood by a factor which adjusts its weightage in the loss function.

Is any of this correct? At the moment, my data is not fitting well to the model’s predictions and I have a strong suspicion that the choice of prior may very well be the reason behind it. Thanks for the help.

2

You can scale likelihood using scale handler.

1 Like
3

@fehiepsi Thanks a lot for the response. Is there some example where the scale handler is used? If I get it correctly, I can use it with a context manager like this:

with numpyro.scale(scale=2.0):
numpyro.sample(“y”, dist.Normal(y_pred, sigma), obs=y_obs)

And this would mean that I am weighting likelihood twice (scale value of 2) relative to the prior? Is it?

Thanks again :slight_smile:

4

Almost correct, it weights “log” likelihood twice. :slight_smile:

1 Like