Hello I was trying to understand how I could get posterior distributions for my parameters from a trained model, when I observe new data points for both X and y values. Shouldn’t the model parameters adapt to new observations? I am attaching my code on simulated data, where I am trying to model a simple linear regression (y = mu*X
).
import random
import numpy as np
import torch
import pyro
import pyro.distributions as dist
import pyro.optim as optim
from pyro.infer import Predictive
from pyro.infer.autoguide.guides import AutoDiagonalNormal
from pyro.infer import SVI, TraceEnum_ELBO
# %% Functions to create trajectories
np.random.seed(100)
samp_t = np.linspace(0, 10, 10)
samp_traj = 5 * samp_t
X = torch.tensor(samp_t, dtype=torch.float)
y = torch.tensor(samp_traj, dtype=torch.float)
# %% Training
def model(X, y):
mu = pyro.sample("mu", dist.Normal(0., 5.))
sigma = pyro.sample("sigma", dist.Uniform(0., 10.))
mean = mu * X
T = 10
with pyro.plate("data", T):
pyro.sample("obs", dist.Normal(mean, sigma), obs=y)
# %% Using SVI
pyro.set_rng_seed(1524)
guide = AutoDiagonalNormal(model)
svi = SVI(model,
guide,
optim.Adam({"lr": .01}),
loss=TraceEnum_ELBO())
pyro.clear_param_store()
num_iters = 1000
losses = []
j = -1
for i in range(num_iters):
elbo = svi.step(X, y)
losses.append(elbo)
print(elbo)
# %% Predictive
samp_t_new = np.linspace(0, 10, 10)
samp_traj_new = 9 * samp_t_new
X_new = torch.tensor(samp_t_new, dtype=torch.float)
y_new = torch.tensor(samp_traj_new, dtype=torch.float)
num_samples = 1000
predictive = Predictive(model, guide=guide, num_samples=num_samples)
data = predictive(X_new, y_new)
mu = data.get('mu').detach().numpy()
mu_mean = np.mean(mu, 0)
During training the data was generated using y = 5*X
. Then during the predictive phase, I changed the observation data such that y_new = 9*X_new
. I was expecting the Predictive
function to adapt to this new data observation and predict the slope parameter closer to 9
or something in between. However, the model still predicts the same value for parameter as it was trained on, that is the result was close to 5
instead of being closer to 9
.
This is just a simulated code where I was trying to understand if the Predictive
functionality would be able to adapt to new observations of both X and y values. Is this the correct approach? Please help me understand how this would be possible using Pyro.
Thanks