I am wondering if it is possible to specify a guide that is a high-dimensional MVG with a sparse precision matrix. I am dealing with a high-dimensional regression problem in which the number of parameters is ~ 1 million and so there is no shot at modeling a full covariance matrix. However, I have a strong prior on which parameters are correlated with each other.
Due to the enormous size of the precision / covariance matrices, Pyro would need to have some way of representing the sparse matrix efficiently.
specifying the precision matrix would probably be the least of your problems, since you would also need to be able to sample from the distribution (which involves the inverse square root of the precision) and compute the log_prob (which would involve e.g. computing the log determinant of the precision). all of this is possible but it wouldn’t be a small lift. your best bet is probably to use AutoLowRankMultivariateNormal