Thanks for the quick response!
Hi, you've pretty much hit on the reason we don't provide a generic MH implementation. Here's a (very) old PR where we considered introducing a generic version and ultimately decided against it; the MH code is here.
Ah, ok, thanks. That clears things up for the first question.
Basically, your second point is a theoretical show-stopper without additional constraints on the model or proposal. Coming up with a completely general-purpose reversible jump correction scheme is a subject of active research, as is modifying the language, algorithm or theoretical justification to avoid the need for such a thing.
Ok, this is what I was coming to think. It's quite a fascinating problem, though.
I'm not completely up to date with those lines of research, but perhaps one promising idea that could be adapted to this context is to build static analysis tools for distinguishing between "structural" random choices that influence control flow and "non-structural" ones that don't, as in e.g. this paper.
Thanks for the reference! Looks super interesting. I'll have a read through this weekend.
Personally, I think it would still be OK to have MH with user-specified proposals, but we don't have any use cases to justify implementing the algorithm and the attendant RJ corrections and/or warning+validation logic ourselves.
Another thought that I had (again, this is a common use case for me but I realize not for very many others) was an RJMC sampler that uses something like NUTS or HMC for within-model sampling (same caveat as above, no stochastic introduction of new variables within models), but then jumps between a user-specified list of models with either user-specified cross-model proposals or proposals from the prior of the new variables (which would probably be terrible, but maybe sometimes would work). I just wasn't sure what the right way to go about this is. I imagine that passing a list of models to a sampler is not really idiomatic Pyro, but short of solving the problem mentioned above, I'm not sure what the right way to do this would be.
I've been busy with some other stuff, but I can try taking a stab at some of things things in a more restricted setting this weekend to fix ideas and see if there's anything useful for you and the rest of the community.