Variational Autoencoder Bernoulli sample

In the VAE tutorial, the observed images in the mini-batch x are scored against the Bernoulli likelihood parametrized by loc_img:

pyro.sample("obs", dist.Bernoulli(loc_img).to_event(1), obs=x.reshape(-1, 784))

However, X is not either 0 or 1. X is not discrete; it is continuous. It could have any value in [0, 1] interval. Isn’t it an error to make this assumption? Shouldn’t the sample happen from a continuous distribution in the [0, 1] interval centered in loc_img?

From D. Kingma, M. Welling (2014):
“We let pθ(x|z) be a multivariate Gaussian (in case of real-valued data) or Bernoulli (in case of binary data)…”. … however, in this case the data is not binary. Why then the tutorial uses Bernoulli sample?


this is common practice. empirically bernoulli tends to outperform gaussian in this setting. see e.g. the discussion at the beginning of this paper (which proposes what it claims to be a more principled solution)

1 Like