Hi, I am having trouble trying to use AutoDelta guide with svi for a mixed effects model. Is it an issue to use the AutoDelta guide for this model architecture? The MAP estimation of the variance of the random effects seems to be way off (and highly variable on different runs even though it seems that elbo has converged)
It is hard to know what’s going on without details. Could you elaborate your question?
Sure so basically I am fitting a basic mixed effects model. We have 2 groups called A and B. I want effects for each n_a and n_b levels of each group. I am modeling the group A standard deviation as sigma_A and group B standard deviation as sigma_B. And then the random effects as Normal(mu_A, sigma_A) and Normal(mu_B, sigma_B)
Then I model the response as mu = A[i] + B[i] + fixed effects
The priors for sigma_A and sigma_B come from a relatively uninformative prior (folded Student T dist). When I fit this model with an AutoNormal guide, I get reasonable estimates for the random effects (and sigma_A and sigma_B). However, I get very wonky estimates for the ranefs (nonsensical stds) when I use AutoDelta even though elbo loss seems to have converged.
The elbo loss is empirically lower when using the AutoDelta guide (extracting from svi results), but the fit is actually way worse on the training and test data.
Trying to provide as much detail as I can. Let me know what else would be helpful but I’m confused as to why the std are so poorly estimated using MAP
I would test things on a toy dataset and a simple generative model. You might want to use MCMC to see what the posterior looks like.
Yeah so I have but still not seeing why auto delta performs so much worse than auto normal. Still having the error where it underestimates sigma_A and sigma_B. Are there reasons for why this could be the case generally speaking? I do not think my model is misspecified as it works fine with mcmc and svi with autonormal guide