PPCA in Pyro

1

Hi. I’m building this very simple Probabilistic PCA model. I’m pretty confident that I did everything correctly. But I’m getting this error:

RuntimeError: The size of tensor a (4) must match the size of tensor b (2) at non-singleton dimension 0

The dimensions perfectly match.

Here’s my simple code

import torch
import pyro
import matplotlib.pyplot as plt
import numpy as np
import pyro.infer
import pyro.optim
import pyro.distributions as dist
from torch.distributions import constraints
from sklearn import datasets

pyro.enable_validation(True)  # <---- This is always a good idea!

pyro.set_rng_seed(101)
d, D = 2, 4  # small dimension d, large dimension D.
iris = datasets.load_iris()
X = torch.tensor(iris.data, dtype=torch.float32)
y = iris.target


def ppca(data):
    A = pyro.param("A", torch.zeros((D, d)))
    mu = pyro.param("mu", torch.zeros(D))

    for i in pyro.plate("data", len(data)):
        z = pyro.sample("latent_{}".format(i), dist.Normal(torch.zeros(d), 1.0).to_event(1))
        pyro.sample("observed_{}".format(i), dist.Normal(A @ z + mu, 1.0).to_event(1), obs=data[i])


def guide(data):
    A_ = pyro.param("A_", torch.zeros((D, d)).T)
    for i in pyro.plate("data", len(data)):
        pyro.sample("latent_{}".format(i), dist.Normal(A_ @ data[i], 1.0).to_event(1))


pyro.clear_param_store()
svi = pyro.infer.SVI(model=ppca,
                     guide=guide,
                     optim=pyro.optim.SGD({"lr": 0.001, "momentum": 0.1}),
                     loss=pyro.infer.Trace_ELBO())

losses, a, b = [], [], []
num_steps = 2500
for t in range(num_steps):
    losses.append(svi.step(X))

plt.plot(losses)
plt.title("ELBO")
plt.xlabel("step")
plt.ylabel("loss")
2

looks like you’re missing {} in some of your format()s

1 Like
3

Hi, thanks for that. It was a dumb mistake on my side. I fixed it, but now it complains about dimensions.

RuntimeError: The size of tensor a (4) must match the size of tensor b (2) at non-singleton dimension 0

The dimensions perfectly match. It is simple matrix vector multplication. When I run ppca(X) it works perfectly. It only throws the error of dimensions when optimizing.

4

when i run your code i don’t see that error. instead i see an error due to bad parameter values being fed into distributions. that goes away, however, if i use Adam instead of SGD. (SGD generally doesn’t perform well for these kinds of stochastic optimization problems)

1 Like
5

Thanks a lot. Now it is working!