I have been using Pyro for some time and recently looked at the code base to understand how it’s working. I signed up on this forum just to ask this question. I noticed that pyro computes everything sequentially maintaining a trace and then back/forward propogates through this trace. Is there a reason for this choice? I am not sure but intuitively I feel like building a computational graph might be more efficient since we can also run minor optimizations on the graph and remove unnecessary computations (for e.g. sampling a variable that is not used anywhere in the graph).
What does the community think of this design choice and are there concrete reasons behind why it is implemented the way it is now?