Reparam vs reinforce

Tutorials
1

Hi, I’ve just read the series of “SVI Pat I-IV” articles at pyro.ai/examples. Part III in particular talks about the reparam and reinforce options of computing the gradients wrt to the posterior’s params. I’d like to ask:

  1. How does pyro (and numpyro) know in practice when to reparametrize?

  2. I’ve noticed that the inbuilt distributions in numpyro have the `reparametrized_params` attribute (not so in pyro?) But still, it would seem to me that in order to reparametrize, it is not enough to just know that a given distribution is reparametrizable (wrt to a given parameter), but that one would also need an explicit transformation, the `g` function using the notation from the article along with the new posterior `q_\epsilon`?

  3. If I implement a custom distribution (with logprob and sample methods) how do I make it reparametrizable in pyro/numpyro (assuming I have a concrete reparametrization in mind)?

2

Did I missunderstand things so terribly that my questions don’t make any sense? :face_with_peeking_eye: If that’s the case, please let me know as well :joy:

3

Hi @Pawel, reparametrized distributions are determined via has_rsample attribute. You can implement rsample method. You might also be interested in using transformed distribution.