Hybrid MCMC + SVI for point-prediction of specific parameters

1

Hey guys,

Is there a way to do a from of mcmc + svi where you want to get point estimates of a subset of the parameters (maybe this is mcmc + MAP), and so you take a gradient with respect to the parameter-subset, given the posterior-distribution of the other parameters (which are sample sites)?

For example, with the sparse variational gaussian process, there is an mcmc version (referenced in this paper https://arxiv.org/pdf/1506.04000) where it would be nice to actually take a gradient with respect to the inducing points, while performing mcmc sampling on the remaining parameters.

2

not really. MAP isn’t even really a well-defined inference algorithm insofar as it’s coordinate-dependent (i.e. you get a difference answer if you change coordinate systems in latent space).

though generically if you have a model of the form p({\rm data}|x,y)p(x,y) you can e.g. do mixed MAP/variational inference w.r.t. x and y–say you get a point estimate of x–and then plug x into the model and treat it as known, and then do MCMC w.r.t. y using the new model