Hello,

I am trying to implement a variation of ELBO called GECO (see paper https://arxiv.org/abs/1810.00597).

It seems that pyro has all the machinery necessary to implement this fairly easily. I just need to know the details of how a pyro updates constrained parameters.

For example assume that your loss function is:

Loss = f + mu *g

where mu was declared as a pyro.param with constraint=constraint.unit_interval.

What is the update rule for mu?

For an unconstrained parameter it is mu -= learning_rate * dLoss / d(mu)

What is the update rule if mu is constrained to be in the unit interval?

What is the update rule if mu is constrained to be positive?

Thanks!