I’ve discussed a bit with fellow scientists that argued about Rhat not being usable in the case of ensemble samplers due to the intrinsic correlation between the walkers. I am having a hard time finding materials/second opinion on this topic on the internet, what are your opinions on this question ?
perhaps we might ask the person who implemented aies in numpyro => @amifalk
I’m not aware of any work in that area. Dan Simpson, one of the authors of the r-hat paper does make the point that “R-hat is a diagnostic not a hypothesis test,” so I’m not sure how to respond to the claim that it’s an inadmissible estimator in the case of ensemble methods. You could probably run a simulation study and investigate that empirically if you want a better sense.
For what it’s worth, people tend to use some version of an autocorrelation statistic to assess convergence of ensemble mcmc methods.
2 Likes
Some additional resources that might be of interest - this paper from Margossian et al. on \hat{R} for many short chains might be useful as something to consider. The lead author from Consensus Clustering for Bayesian mixture models also talks about testing convergence in ensembles beyond the original suggestions of that paper in a blog post. It’s not rigorous, but it might be a start point.
TY for these insights, it helps very much. I’ll keep investigating this topic