Training on single GPU

numpyro
1

I have access to the machine with only single GPU.
What is the most efficient way to use multiple chains for MCMC sampling. Is it possible at all to use num_chains > 1 with only single GPU?

Thanks a lot.

2

Yes, if you run, you will see a message say that sequential method will be used. You can also use vectorized method, which might be good if you draw a lot of chains.

3

Thank you.
Maybe not related to this question, but what is the recommended way to use MCMC numpyro on large dataset (e.g. ~300K samples).

  • Is there any promising consequence of using single chain GPU instead of several chains on multiple CPU.
  • Is there at all possible to use concept of batch in neural networks with MCMC. Unless I am missing some new trends in Bayesian statistics, it should no be possible conceptually.
    Thanks again.
4

if your latent variables are all continuous you might try the approach described here

5

Thanks for the link.