Prior selection

Misc.
1

Hi everyone,

I’m relatively new to Bayesian methods and I’m looking for a prior for parameters that should generally be strongly discouraged from becoming negative (but it should not be impossible) and the most probable value should be around 0.1 or so and remain about as probable until value 1.0 or so.

I’m picturing a kind of truncated sigmoid function. Actually the CDF of a normal distribution seems like a reasonable option to me but that would be an improper prior, which I guess would not be a good idea?

Any suggestions for a good prior? I’ve also been looking into the inverse gamma distribution but I’m wondering if that would be a good option. I don’t really have a theoretical reason to prefer 0.1 over 0.5 for instance. Mostly I just know that anything approaching and below zero should become increasingly less probable.

2

Inverse Gamma can’t be negative so probably not a good choice. I would think a Normal with a small standard deviation, or a Cauchy with a small scale, or a t-distribution.

3

Thanks for your response!

The reason I was thinking about inverse gamma is because I could shift it to the left (making negative values possible) but it can have a heavy tail to the right.

The problem with normal, cauchy and t-distributions, I think, is that they are symmetric whereas my prior knowledge suggests a non-symmetric distribution with a very heavy positive skew.

I don’t want my prior to discourage a value of 0.5 versus 0.1, for instance, while I want negative values to be strongly discouraged but not impossible. The latter I can achieve with a normal distribution with small sigma, e.g., Normal(0.1, 0.1) or something like that, but that would mean I also make 0.5 much less probable in my prior.

4

That’s fair. Have you tried going through the distributions in Distributions — NumPyro documentation? I usually use Wikipedia or WolframAlpha to get a sense of distribution shapes and then sample from that distribution using numpyro and plot it using matplotlib to narrow down my prior.

5

Thanks for thinking along! Perhaps this is a silly question but how do you sample from the distribution outside of the model definition? I get an assertion error when I do something like:

numpyro.sample(
                "theta",        
                dist.LogNormal(
                loc=jnp.array([-0.02] * num_params), 
                scale=jnp.array([0.2] * num_params), 
            ),)

And the effect of the loc and scale parameter seems to be a bit different in the numpyro distribution than the scipy lognorm distribution (which I usually use to assess distributions)

6

n_samples of a distribution:

dist.Beta(2,2).sample(random.PRNGKey(0), (n_samples,))

So e.g. this is n_samples independent draws of a Beta(2,2).

1 Like
7

Very nice! That’s incredibly helpful. Thank you!

8

By the way, in scipy.stats the distributions have the option to change the location of distributions that are only defined on a positive scale (like lognormal). I use this to create a distribution that starts at negative values but is highly skewed to the right.

How would one approach this in Numpyro? Simply add some negative value, e.g. -0.1, to all the parameters? Or is there a way to do this within the distribution functions? (there doesn’t seem to be)