Variational Autoencoder Bernoulli sample

Tutorials
1

In the VAE tutorial, the observed images in the mini-batch x are scored against the Bernoulli likelihood parametrized by loc_img:

pyro.sample("obs", dist.Bernoulli(loc_img).to_event(1), obs=x.reshape(-1, 784))

However, X is not either 0 or 1. X is not discrete; it is continuous. It could have any value in [0, 1] interval. Isn’t it an error to make this assumption? Shouldn’t the sample happen from a continuous distribution in the [0, 1] interval centered in loc_img?

From D. Kingma, M. Welling (2014):
“We let pθ(x|z) be a multivariate Gaussian (in case of real-valued data) or Bernoulli (in case of binary data)…”. … however, in this case the data is not binary. Why then the tutorial uses Bernoulli sample?

Thanks!

2

this is common practice. empirically bernoulli tends to outperform gaussian in this setting. see e.g. the discussion at the beginning of this paper (which proposes what it claims to be a more principled solution)

1 Like