Hi, I’m trying to adapt the SS-VAE example for my own use, and in particular I’m replacing the one-hot categorical latent “y” with an array of Bernoulli latent variables.
The problem is that the unsupervised loss (when y is not observed) is always 0. So, I imagine the model is not learning from the unsupervised examples.
After some debugging, I think the issue could be related to my Bernoulli latents, and the “enum_discrete” mode. If I enter the “step” function, and follow the debugger down to Trace_ELBO.loss_and_grads(), it’s computing “weight” values of 0 for all examples in the batch. Further down, the “scale” returned by iter_discrete_traces() is all zeros for some reason…
Any clues or suggestions would be much appreciated.
I don’t know if its related, but I’ve noticed that every time this latent is sampled in the model or guide function (using pyro.sample), it always returns the same value, for every value in the array. For example, I can call
y= pyro.sample("y", dist.bernoulli, alpha_y) where alpha_y is all 0.5s, and every value of the returned y is either 0 or 1. I’d expect a mixture of 0s and 1s.