Hi all - I was curious whether the “QR trick” (see e.g. The QR Decomposition For Regression Models) could be used to speed up/improve fitting of regression models in pyro when the covariates are correlated. I’m doing this using a custom guide which I think is correct. However, I get much worse convergence and final error (I know the true posterior in the linear model case). If anyone is interested my code is here:
Google Colab
Maybe this approach just isn’t helpful for SVI, or I’m doing something wrong in the setup (I don’t think pyro is doing anything wrong).
i think you’re just seeing suboptimal optimization driven by initialization choices.
e.g. if i instead use
Q = Q * math.sqrt(N)
R = R / math.sqrt(N)
i get
qr_mv_guide-----------
Error in E[beta] 0.041163329035043716
Error in diag cov[beta] 0.008744467981159687
Error in cov[beta] 0.0009263675310648978
1 Like