Spiking fluctuations in ELBO loss

Hi there! I’m using SVI for posterior inference, however, after a few thousand steps there seem to be unstable behaviors in the ELBO loss. I’m not sure whether this is related with optimization, as i’m using Adam which adapts the learning rate by it self, or is this related to anything else?