Gaussian Process - multiple observations

Misc.
1

Hi,

I have been reading through the documentation and I did not find a way to deal with multiple observations of the same GP. Is there such a feature in pyro?

Best,
Luis

2

Hi @luisroque, I’m not sure which model that you are using. You can take a look at GPR class, the requirement for the observations y is its last dimension is the number of data point. You can use observations with arbitrary number of dimensions as long as they satisfy that condition.

3

Hi @fehiepsi when I say multiple observations I am talking about y being, in reality, a matrix (n, m), where n is the number of points observed and m is the number of dimensions observed. You can think of it as observing several processes (e.g. time series) and we wanted to learn the latent process that generates them (in this case a GP). My interest is in adding several of these latent GPs indexed by the time series that belong to different groups. Example: GP1 applies to time series [0, 0, 1, 1] and GP2 applies to time series [1, 0, 1, 0]. This means that for time series 3 I would like to add both “latent” GPs: GP1+GP2 as the mean of Gaussian Likelihood for instance. Is this doable in pyro? Can you point me in the right direction?

Thanks,
Luis

4

If your observations have shape (num_points x num_dims), then you can transpose it to have shape (num_dims x num_points). You might also want to try variational models where you can specify latent_shape to the number of output GPs. Note that currently, all Pyro GP models only support 1 set of parameters of the kernel - i.e. they do not support batch GP to train different GPs on different output dimensions.

for time series 3 I would like to add both “latent” GPs: GP1+GP2 as the mean of Gaussian Likelihood for instance. Is this doable in pyro?

I’m not sure if I understand correctly. I think your code will look like

def get_likelihood(ts1, ts2, sigma)):
    return dist.Normal(gp1(ts1) + gp2(ts2), sigma)
5

That did the trick to handle the multiple observations @fehiepsi and generate a single latent dimension!

As a way to index and then sum VariationalSparseGP GPs I was thinking about something along these lines. I will write code but it is more like pseudocode as I am starting with pyro, sorry if I miss anything obvious. I will follow the example that I gave above: GP1 applies to time series [0, 0, 1, 1] and GP2 applies to time series [1, 0, 1, 0].

vsgp1 = gp.models.VariationalSparseGP(X, y, kernel1, latent_shape=(1,), Xu=Xu, whiten=True)
vsgp2 = gp.models.VariationalSparseGP(X, y, kernel2, latent_shape=(1,), Xu=Xu, whiten=True)

def model(X, y):
    f1 = vsgp1.model()
    f2 = vsgp2.model()
    f1_m = f1.reshape((1,-1)) * np.array([1, 0, 1, 0]).reshape((1, -1))
    f2_m = f2.reshape((1,-1)) * np.array([0, 0, 1, 1]).reshape((1, -1))
    pyro.sample("y", dist.Normal(f1_m+f2_m, sigma), obs=y)

As I wrote before, I would expect this way that series 2 (indexing starting at 0) would have the contribution of GP1 and GP2 (f1+f2), while series 3 would have only the contribution of f2. Does it make sense?

6

Hi @fehiepsi , I can see that you specifically recommend using either the GPRegression or VariationalSparseGP models. I was wondering if the SparseGPRegression class could also be suitable for this purpose? (I’m asking mainly because I encountered this issue recently.) Thanks a lot for your answer :slight_smile: