Pyro in sklearn.gaussian_process

Hi, I’m using pyro to optimize my agent model, below is my code:

X = data_1[‘params’] #130×3
Cp_E = data_1[‘Cp_E’] #the obs, which is 400×1
Y = data_1[‘Cp_M’] #130×400
X = torch.tensor(X, dtype=torch.float)
Y = torch.tensor(Y, dtype=torch.float)
Cp_E = torch.tensor(Cp_E, dtype=torch.float)
gaussian=GaussianProcessRegressor()
fiting=gaussian.fit(X, Y)

r1 = 0.
r2 = 0.
r3 = 0.
def model(r1, r2, r3):
   beta2 = pyro.sample(“beta2”, dist.Uniform(r1, 0.5))
   betaStar = pyro.sample(“betaStar”, dist.Uniform(r2, 0.5))
   a1 = pyro.sample(“a1”, dist.Uniform(r3, 1))
   Z = np.array([beta2, betaStar, a1])
   YX=gaussian.predict(Z.reshape(1, -1))
   #the YX has 400 numbers, which is 400×1
   YX = pyro.param(“YX”, torch.tensor(YX))
   return YX
conditioned_model = pyro.condition(model, data={“YX”: Cp_E})
#the Cp_E is the observed value, which is 400×1

def guide(r1, r2, r3):
   mean1 = pyro.param(“mean1”, torch.tensor(0.08))
   mean2 = pyro.param(“mean2”, torch.tensor(0.09))
   mean3 = pyro.param(“mean3”, torch.tensor(0.31))
   va1 = pyro.param(“va1”, torch.tensor(1.), constraint=constraints.positive)
   va2 = pyro.param(“va2”, torch.tensor(1.), constraint=constraints.positive)
   va3 = pyro.param(“va3”, torch.tensor(1.), constraint=constraints.positive)
   beta2 = pyro.sample(“beta2”, dist.Normal(mean1, va1))
   betaStar = pyro.sample(“betaStar”, dist.Normal(mean2, va2))
   a1 = pyro.sample(“a1”, dist.Normal(mean3, va3))

from pyro.infer import SVI, Trace_ELBO
svi = SVI(model=conditioned_model, guide=guide, optim=Adam({“lr”: .001, “betas”: (0.95, 0.999)}), loss=Trace_ELBO())
losses, beta2, betaStar, a1 = [], [], [], []
num_steps = 5000
for t in range(num_steps):
   losses.append(svi.step(r1, r2, r3))
   beta2.append(pyro.param(“mean1”).item())
   betaStar.append(pyro.param(“mean2”).item())
   a1.append(pyro.param(“mean3”).item())

The problem is the “mean1”, “mean2” and “mean3” have the same value in these 5000 steps, and the losses are all inf.
But when I make the model as:

r1 = 0.0828
r2 = 0.09
r3 = 0.31
def model(r1, r2, r3):
   beta2 = pyro.sample(“beta2”, dist.Normal(r1, 1))
   betaStar = pyro.sample(“betaStar”, dist.Normal(r2, 1))
   a1 = pyro.sample(“a1”, dist.Normal(r3, 1))
   Z = np.array([beta2, betaStar, a1])
   YX=gaussian.predict(Z.reshape(1, -1))
   YX = pyro.param(“YX”, torch.tensor(YX))
   return YX

the losses are falling, the value of “mean” also changed, but did not achieve the optimized effect.
Actually I also want to real-time monitor the changes of “beta2”, “betaStar” and “a1” at each step, but I can’t get them by pyro.get_param_store().items(), so I just monitor the mean value of these parameters.

My problem has not been solved yet, I would like to ask any suggestions, thanks in advance.

What is GaussianProcessRegressor()? Is that a scikit-learn object? If so, you can’t use it in a Pyro model because PyTorch cannot propagate gradients through NumPy/SciPy functions.

Please see GPyTorch or pyro.contrib.gp for PyTorch/Pyro-compatible Gaussian process models.

Yes, it’s a scikit-learn object. I’m so sad to hear that.
I have tried the pyro.contrib.gp, but I did not realize its high-dimensional input and output, I will try again, thank you for your prompt reply.