From what I gather, delta guide assigns all probabilities associated with the guide’s parameter to a single value, so that there is no uncertainty in the parameter. If this is the case, then how is the delta-based SVI different from the normal frequentist inference?
In Bayesian, whether we use Delta guide or not, we put some prior knowledge on parameters while in frequentist, there is no such prior notion. If there is no prior, it is similar to frequentist inference where we use log-likelihood as the objective function.