Thanks @fehiepsi that’s very interesting but not my current use case. I think my loss landscape is very wiggly but also steep so NUTS is taking very small steps and stuck near the init_value guess. I was instead thinking something like run SVI (or adam) first to get a MAP estimate of parameters, and then use the result of that to initialize the starting point for NUTS at the MAP from SVI/adam, kind of like here:
- https://forum.pyro.ai/t/initialize-mcmc-with-svi/7799
- https://forum.pyro.ai/t/using-svi-multivariatenormal-to-estimate-nuts-inverse-mass-matrix/5188
- https://forum.pyro.ai/t/initialice-trace-of-mcmc-nuts-with-map-estimate/1408
is this sequential 2-stage SVI/adam then NUTS usually a good strategy for complicated posteriors? Is it common to do?