SVI.step and dtype property


#1

Hi ! :slight_smile:

Following the release of Pyro 0.2/PyTorch 0.4 I tried to update a project to be in line with the latest practices. The tutorial works fine but doesn’t use :
device = torch.device(“cuda” if torch.cuda.is_available() else “cpu”)

Which I thought would be great in order to remove all :
if args.cuda:
x = x.cuda()

First, I rewrote :
vae = VAE().to(device)

Then, for the training loop :
for _, x in enumerate(train_loader):
x = x.to(device)
# do ELBO gradient and accumulate loss
epoch_loss += svi.step(x)

At this point, it throws an error :
File “PycharmProjects/VAE/vae_custom.py”, line 183, in main
epoch_loss += svi.step(x)
File “PycharmProjects/VAE/venv/lib/python3.6/site-packages/pyro/infer/svi.py”, line 75, in step
loss = self.loss_and_grads(self.model, self.guide, *args, **kwargs)
File “PycharmProjects/VAE/venv/lib/python3.6/site-packages/pyro/infer/svi.py”, line 47, in _loss_and_grads
loss_val = loss(*args, **kwargs)
TypeError: ‘str’ object is not callable

After further analysis between the tutorial and this try/fail experiment, it appears that the “custom” tensors I have present an property dtype :
tensor([[-5.5055e-01, -3.4432e-01, -1.4731e-01, …, 1.3304e+00,
-1.8560e-01, 3.0320e-01],
…,
[ 1.4803e+00, 1.3554e+00, -1.5034e-01, …, -1.4670e+00,
-1.0415e+00, -1.4008e+00]], dtype=torch.float64)

Whereas the tutorial offers tensors which haven’t that property, e.g. :
tensor([[ 0.0000, 0.0000, 0.0000, …, 0.0000, 0.0000, 0.0000],
…,
[ 0.0000, 0.0000, 0.0000, …, 0.0000, 0.0000, 0.0000]])

My “custom” tensor is just
train_loader = torch.utils.data.DataLoader(train_data, batch_size=batch_size, shuffle=True)

And train_data inherits the torch.utils.data.dataset.TensorDataset.
Prior to update, I didn’t have this issue.

Is there a known workaround to make it work ?
Thanks ! :slight_smile:


#2

Your issue is that when you create an SVI instance you need to pass in an ELBO instance rather than the string "ELBO".

- svi = SVI(model, guide, optim, "ELBO")
+ from pyro.infer import Trace_ELBO
+ svi = SVI(model, guide, optim, Trace_ELBO())