Some parameters not converging in simple model

Misc.
1

Hi everyone,
I am trying to learn Pyro to use in my modeling work and am currently following the introductory documentation.
I am trying to understand inference following the steps here:
Inference in Pyro: From Stochastic Functions to Marginal Distributions

I built this simple model of linear dependence + noise:

import numpy as np
import matplotlib.pyplot as plt
import torch
import pyro
import pyro.infer
import pyro.optim
import pyro.distributions as dist
torch.manual_seed(101);

def linear(x):
    noise = pyro.sample('noise', dist.Normal(0, 0.5))
    return 2.0 * x + noise

And this guide:

def linear_guide(x):
    a = pyro.param("a", torch.tensor(3.0, requires_grad=True))
    b = pyro.param("b", torch.tensor(1.0, requires_grad=True))
    return a * x + pyro.sample('noise', dist.Normal(0, b))

Here is the inference code:

pyro.clear_param_store()
svi = pyro.infer.SVI(model=linear,
                     guide=linear_guide,
                     optim=pyro.optim.SGD({"lr": 0.001}),
                     loss=pyro.infer.Trace_ELBO())

losses = []
xs = np.random.choice(np.arange(100), 10000)
for x in xs:
    losses.append((svi.step(float(x))))
plt.plot(losses)
plt.title("ELBO")
plt.xlabel("step")
plt.ylabel("loss")

Only one of the parameters seem to be converging:

pyro.param('b').item()
#0.529, real value 0.5

pyro.param('a').item()
#3.0, real value 2.0

I would appreciate any guidance as to what I am missing here.
Thanks in advance!
Yosi

2

@yosihammer You need to define observation variable in your model. I guess what you want is following

def linear(x, y):
    a = pyro.param(...)
    noise = pyro.sample(...)
    return pyro.sample("y", dist.Normal(a * x, noise), obs=y)

def guide(x, y):
    b = pyro.param(...)
    noise  = pyro.sample(..., dist.Normal(0, b))

xs = ...
y = 2 * xs
svi.step(xs, y)