IWAE ELBO in Pyro/NumPyro?

Misc.
1

Hi, Pyro devs,

Is the IWAE EBLO in Pyro or NumPyro? I couldn’t find anything from a quick search, but I figured I should ask before I started coding.

I need it to reproduce figure 2 from Rainforth et al.. If it doesn’t already exist (and if it’s of interest), I’ll gladly submit a PR.

2

for pyro RenyiELBO with alpha=0 is equivalent to IWAE (although i guess the docs could be more helpful here…)

similarly for numpyro: RenyiELBO

1 Like
3

Awesome, thanks!

4

@martinjankowiak does Pyro’s RenyiELBO implementation use a doubly reparametrized gradient estimator (DReG) trick like the incomplete TraceIWAE_ELBO PR, or does it use some other trick?

5

@fritzo i don’t believe DReG or any comparable variance reduction technique is used anywhere within the repo. the variance reduction that is implemented is focused on discrete latent variables

6

@martinjankowiak @fritzo, is DReG of interest in Pyro? I’m working on a DReG implementation for the RenyiELBO in NumPyro as I have reason to believe it will improve convergence for elbo-within-stein.

1 Like
7

yes i think so. in general i think those elbo implementations could probably be revamped in various ways. @OlaRonning please put up a PR or make a github issue for discussion : )

8

Added a discussion issue on Pyro ( #3104).

1 Like