What is 'obs' and what is 'mu'

1

I’ve ran a bayesian regression with 12 variables. One of of those variables is an intercept.(I derive the dataframe from a patsy formula. When I run the Predictive module, is ‘mean’ the same as the intercept or should be interpreted as the intercept? Is ‘obs’ the predicted values?

2

Hi, can you please provide a code snippet to clarify your question? It’s not clear what ‘mean’ and ‘obs’ are referring to.

3

When I put:

class BayesianRegression(PyroModule):
    def __init__(self, in_features, out_features):
        super().__init__()
        self.linear = PyroModule[nn.Linear](in_features, out_features)
        self.linear.weight = PyroSample(dist.Normal(0., 1.).expand([out_features, in_features]).to_event(2))
        self.linear.bias = PyroSample(dist.Normal(0., 10.).expand([out_features]).to_event(1))

    def forward(self, x, y=None):
        sigma = pyro.sample("sigma", dist.Uniform(0., 10.))
        mean = self.linear(x).squeeze(-1)
        with pyro.plate("data", x.shape[0]):
            rate = self.linear(x).squeeze(-1).exp()
            obs = pyro.sample("obs", dist.Poisson(rate), obs=y)
        return mean
    
model = BayesianRegression(x.shape[1], 1)    
guide = AutoDiagonalNormal(model)    

adam = pyro.optim.Adam({"lr": 0.03})
svi = SVI(model, guide, adam, loss=Trace_ELBO())

num_iterations = 5000
pyro.clear_param_store()
for j in range(num_iterations):
    # calculate the loss and take a gradient step
    loss = svi.step(x, y)
    if j % 100 == 0:
        print("[iteration %04d] loss: %.4f" % (j + 1, loss / len(x)))

guide.quantiles([0.25, 0.5, 0.75])

def summary(samples):
    site_stats = {}
    for k, v in samples.items():
        site_stats[k] = {
            "mean": torch.mean(v, 0),
            "std": torch.std(v, 0),
            "5%": v.kthvalue(int(len(v) * 0.05), dim=0)[0],
            "95%": v.kthvalue(int(len(v) * 0.95), dim=0)[0],
        }
    return site_stats
predictive = Predictive(model, guide=guide, num_samples=1000,
                        return_sites=("linear.weight", "obs", "_RETURN"))
samples = predictive(x)
pred_summary = summary(samples)

When I look at pred_summary, there is a dictionary of tensors. The obs come from return_sites in the Predictive function. I’m not sure at this point what return_sites is though.

4

The return_sites argument to Predictive specifies the set of sample sites for which to compute the posterior or posterior predictive distribution.

"obs" in your code refers to the site "obs" in the model, and "_RETURN" is a special symbol for the return value of model, which corresponds in this case to the variable mean in BayesianRegression.forward.

5

So ‘obs’ is my actual y variable and "_Return" is the predicted distribution for each observation?

6

The "obs" field in the output of Predictive contains samples from the posterior predictive distribution of that random variable. This is what you would use to make predictions given a new datapoint x.

"_RETURN" indicates the value returned by the model function BayesianRegression.forward, which in your model code above happens to be the intermediate Python variable mean. Thus the "_RETURN" field in the output of Predictive contains samples from the posterior predictive distribution of that value. If you changed the return statement in your model to something else, like return mean + 1, the "_RETURN" field would contain the posterior predictive distribution of that new value instead.