Thank you for your help. 
Are you proposing all of the training first and then all of the validation with a separate model afterwards. Just asking whether I understand you correctly.
Is there no way of getting per-iteration training data, as described in the following blog post: https://forum.pyro.ai/t/how-to-get-validation-loss-in-svi/4037
I tried learning more about amortized inference. Do you have a beginner-friendly tutorial that is out there for amortized inference using pyro?