Normalizing flows with non-standard base Gaussian distributions

Hi there,

I am a new user who discovered Pyro after looking for example implementations of normalizing flows. I am impressed at what I have seen so far. Congratulations on the huge efforts into putting this together! I am looking forward to making contributions in the future.

I had a quick look at your normalizing flow tutorial, and I noticed that all of your examples make use of standard Gaussian base distributions. On the other hand, Eqs. (15) and (16) of Papamakarios et al. (2021) describe a more general maximum likelihood estimation procedure which also updates the Gaussian parameters (which can be done in closed-form). Before delving further into your examples, I was hoping you can confirm that this more general updating procedure can be easily incorporated within your libraries. If so, I would appreciate if you provide a minimal reproducible example.

Thank you in advance!

3 Likes

It’s been a while since I first posted this question, so I am writing again in a hope to foster some discussion.

cc @stefanwebb who has moved maintenance effort to https://flowtorch.ai

@daaiml I’m unsure whether Pyro allows learning the base distribution, but I suspect that should be pretty easy.

1 Like

@fritzo, thank you for responding back. I’m quite new to these libraries and would greatly appreciate if someone writes a small example of learning base distribution parameters. After that, I will be hopefully able to customize to more complex settings. Appreciate the help of you and @stefanwebb in advance!

It would be great if someone knew how to do this. I’m doing a very odd application with Normalizing Flows and defining my own distribution class would make some of what I’m doing a whole lot easier.