VAE parameters gradients stuck to zeros

Tutorials
1

Hello,

It’s my first post here.

I get the code for the VAE on pyro-ppl github.

I just add some lign to plot the gradients of parameters.

I see that de loss is decreasing but all gradients stuck to zero.

It’s take a lot of time but i still don’t get why.

Here is my code to plot gradients histograms with tensorboard.

        for name, param in vae.named_parameters():
            if param.requires_grad:
                writer.add_histogram(
                    f"{name}_grad", param.grad, epoch
                )
2

you probably want to use register_hook as shown e.g. here