Variational ecological inference


A collaborator and I are working on coding a model/guide for ecological inference. I’m experienced with python, but am having some initial questions about how to do this in pyro:

My first question has to do with the fact that the natural parameter space for the guide is not the same as that of the model. In fact, because the likelihood is an indicator function (do the latents agree with observations?), I’m building a guide conditional on the observations, and not using observations at all on the model side. Thus, the natural parameter space of the guide is lower-dimensional than that of the model. I know that this is not the normal way to do VSI, but I am confident that it works mathematically; my question is about how to program this in practice. I can write the modelspace-to-guidespace mapping function in either direction, and I can calculate its pseudo-jacobian (the absolute value of the product of its nonzero eigenvalues). Which side should I apply it on? How do I ensure that pyro uses the correct pseudo-jacobian, rather than the actual jacobian which is 0?

  1. When I have a bunch of latent parameters of the same type, is it always the right move to index them over a plate, or is there a case for sometimes sampling a single tensor with all of them? If the latter is better, how would I do it?

I realize these are both somewhat beginner questions — that if I were more familiar with this package, I’d probably know what to do more obviously. So, I’m especially grateful for any helpful responses.



On the first question: you might want to implement your model and guide components as torch.distributions.TransformedDistributions with custom Transforms. Have a look at the source code of some of the built-in Transforms in torch.distributions (e.g. AbsTransform) for examples.

On the second question: plates provide extra information about conditional independence to inference algorithms. Without access to this information, inference may sometimes be significantly slower computationally or statistically, so there’s no reason to withhold it if it’s correct.

Note that the more narrow and detailed you can make your questions, ideally including runnable code or math illustrating the issue, the more helpful we can be.