Hi!
I have in my model several variables like
numpyro.sample('A', dist.Uniform(A_min, A_max))
and got some divergence points. I wander if internally Numpyro reparametrizes the sampling to get unbounded variable? Thanks.
Hi!
I have in my model several variables like
numpyro.sample('A', dist.Uniform(A_min, A_max))
and got some divergence points. I wander if internally Numpyro reparametrizes the sampling to get unbounded variable? Thanks.
Yes, it does the transform. If A_min and A_max are not constants, it is better to reparamterize it using Reparam handler. But I guess they are constants in your case?
Hello @fehiepsi, houw do you perform the transformation if A_min and A_max are simple constant?
It is composed of Sigmoid and AffineTransform. You can see the implementation here.
Heeu can you elaborate a bit more please
A is in [a_min, a_max] then how do the code proceed? Sorry for the details.
sigmoid(x) = 1/(1+e^{-x}) x\in\mathbb{R}
and
Affine(x) = a_min + x *(a_max-a_min) x\in[0;1]
Yes, that’s exactly what it does. The transform is x -> Affine(sigmoid(x))
.
Ok, Thanks
@fehiepsi Thinking a bit more I have a question
if A ~ Uniform(a_min, a_max)
, I understand that the transformation Affine(sigmoid(x))
with x
unbounded gives a “A” value in the range [a_min, a_max]. But what about the probability distribution of x
should follow to gives the uniform distribution of A
? Is it the Logistic Distribution?
I am asking because, I have a model with 8 variables with Uniform Bounded priors and the NUTS chains are clearly not Ok, while if I use Normal priors with (mu, sigma) defined as mu=(min+max)/2 and sigma such that mu-5sigma ~ min, then I get quite good results. Okay this may be quite linked to my model, but is there a workaround when defining UniformBounded prior is not working? Thanks.
Is it the Logistic Distribution?
Yes, I think so. In the implementation, the unconstrained value x
has distribution TransformedDistribution(Uniform(a_min, a_max), [Affine.inv, sigmoid.inv])
, i.e. Transformed(Uniform(0, 1), logit)
which is equivalent to a Logistic distribution.
is there a workaround when defining UniformBounded prior is not working?
For tricky posteriors, you can either try to change some hyperparameters like target_accept_prob
or apply some sophisticated methods like neutra transportation. But I would prefer changing priors if it works.