Hi, I’m using the pyro.contrib.gp to train my data, but the trace of ELBO seems irregular like the above picture shows, it dropped to -30,000.
The input and output data are all in [-1, 1], I have tried different values of variance, lengthscale and learning rate, but the situation has not improved, and the predict value of Xnew is also very poor, so I came here for help.
Below is my code:
X = data_1[‘params’]
Y = data_2[‘Cp_M_neg’]
X = torch.tensor(X, dtype=torch.float)
Y = torch.tensor(Y, dtype=torch.float)
kernel = gp.kernels.RBF(input_dim=1, variance=torch.tensor(1.), lengthscale=torch.tensor(1.))
gpr = gp.models.GPRegression(X, Y.t(), kernel, noise=torch.tensor(10.))
gpr.kernel.lengthscale = pyro.nn.PyroSample(dist.LogNormal(0.0, 1.0))
gpr.kernel.variance = pyro.nn.PyroSample(dist.LogNormal(0.0, 1.0))
optimizer = torch.optim.Adam(gpr.parameters(), lr=0.001)
loss_fn = pyro.infer.Trace_ELBO().differentiable_loss
losses =
num_steps = 8000 if not smoke_test else 2
for i in range(num_steps):
optimizer.zero_grad()
loss = loss_fn(gpr.model, gpr.guide)
loss.backward()
optimizer.step()
losses.append(loss.item())
Hope someone can give me some suggestions, thanks in advance.