I am trying to implement a variation of ELBO called GECO (see paper https://arxiv.org/abs/1810.00597).
It seems that pyro has all the machinery necessary to implement this fairly easily. I just need to know the details of how a pyro updates constrained parameters.
For example assume that your loss function is:
Loss = f + mu *g
where mu was declared as a pyro.param with constraint=constraint.unit_interval.
What is the update rule for mu?
For an unconstrained parameter it is mu -= learning_rate * dLoss / d(mu)
What is the update rule if mu is constrained to be in the unit interval?
What is the update rule if mu is constrained to be positive?