I am looking for a way to save data from a numpyro chain to disk in a way that minimizes memory overhead.
I saw this post but it appears the API has changed substantially since 2019, and the MCMC.fit
method as well as the __init__
methods for NUTS
and initialize_model
are quite complicated and difficult (at least for me) to understand.
Ideally I would use a mmap or something so that data can be committed to disk and removed from memory at the operating system’s leisure.
Do you have any advice?