Looking at the tutorial on VAEs (http://pyro.ai/examples/vae.html) it appears to me strange why the data is not binarized, even though the calculation of the likelihood of the image is supposed to be with respect to generation by a bernoulli distribution.
# define the model p(x|z)p(z)
def model(self, x):
# register PyTorch module `decoder` with Pyro
with pyro.iarange("data", x.size(0)):
# setup hyperparameters for prior p(z)
z_loc = x.new_zeros(torch.Size((x.size(0), self.z_dim)))
z_scale = x.new_ones(torch.Size((x.size(0), self.z_dim)))
# sample from prior (value will be sampled by guide when computing the ELBO)
z = pyro.sample("latent", dist.Normal(z_loc, z_scale).independent(1))
# decode the latent code z
loc_img = self.decoder.forward(z)
# score against actual images
pyro.sample("obs", dist.Bernoulli(loc_img).independent(1), obs=x.reshape(-1, 784))
How is the likelihood calculated when the observation is not 0 or 1?, what is it actually calculating?.
I also noticed that results are better at digit generation when binarization is not done, even though the elbo is better when doing binarization.