I’m trying to use normalizing flow in a model-guide pair with compiled inference (CSIS) and SVI, but I cannot find a comprehensive tutorial for it. I read the examples in documentations, but I am not sure if they can be directly used for CSIS or SVI.
For some conditional transforms like ConditionalPlanar, I will get error “ConditionedPlanar object expected to find key in intermediates cache but didn’t” when using in the guide in a way like the doc string does.
Also, it seems the inverse operation for NeuralAutoregressiveFlow is not implemented, is there any plan for including it in the future?
Hi @fshipy, some flows in Pyro do not implement inverse methods for arbitrary values, either because they are expensive or because they have not been necessary; instead they may only implement Jacobian determinant methods given both input and output values. This is presumably the cause of the error you are seeing from ConditionedPlanar - the “inverse” assumes its argument was computed by the flow from a known input and fails when it cannot find that input in its internal cache.
Note that CSIS requires guide distributions to implement log_prob for arbitrary values (because the guide is trained on samples from the model), whereas SVI only requires sample and log_prob of a sampled value, so a flow that only implements a forward method can be used in a guide in SVI but not in CSIS.
We are no longer actively developing the normalizing flows library in Pyro ourselves and will switch to using FlowTorch at some point, but in the meantime we welcome externally contributed bugfixes and enhancements.
Thank you @eb8680_2! I’m wondering is it possible to utilize FlowTorch with SVI or CSIS? If it’s possible, would you mind sharing an example or tutorial?
I’m not aware of any examples or tutorials that combine Pyro and FlowTorch as of this post, but FlowTorch was only released a few weeks ago. If you or any readers manage to make one, please consider sharing it with the community!