Hi, Pyro devs,
Is the IWAE EBLO in Pyro or NumPyro? I couldn’t find anything from a quick search, but I figured I should ask before I started coding.
I need it to reproduce figure 2 from Rainforth et al.. If it doesn’t already exist (and if it’s of interest), I’ll gladly submit a PR.
for pyro RenyiELBO with alpha=0 is equivalent to IWAE (although i guess the docs could be more helpful here…)
similarly for numpyro: RenyiELBO
Awesome, thanks!
@martinjankowiak does Pyro’s RenyiELBO implementation use a doubly reparametrized gradient estimator (DReG) trick like the incomplete TraceIWAE_ELBO PR, or does it use some other trick?
@fritzo i don’t believe DReG or any comparable variance reduction technique is used anywhere within the repo. the variance reduction that is implemented is focused on discrete latent variables
@martinjankowiak @fritzo, is DReG of interest in Pyro? I’m working on a DReG implementation for the RenyiELBO in NumPyro as I have reason to believe it will improve convergence for elbo-within-stein.
yes i think so. in general i think those elbo implementations could probably be revamped in various ways. @OlaRonning please put up a PR or make a github issue for discussion : )