I have been working on some models for the past week or so and am using the pyro.ops.stats.waic helper function to help with the model comparison. One of the inputs is of course the pointwise log likelihood of multiple posterior samples. Right now the way I am computing that is by tracing the conditional model, then diving into the trace and extracting the log_prob for each observed node. It’s kind of a pain and before I put more effort into improving/streamlining my implementation I figured I should ask whether pyro already has some kind of helper utility that will compute log likelihoods for you automatically? I’m thinking something along the lines of log_likelihood(conditional_model, guide, num_samples, data)… does something like this already exist? If not, do you think it’d make a good addition?
@physicswizard Currently, there is no such utility. I think it is a good addition to Pyro.
Btw, it seems to me that arviz supports extracting log likelihood for multiple observed sites here.
Ahh, that arviz implementation looks pretty similar to what I’m doing now, except they use vectorization. Might be nice to have a stand-alone utility in pyro though. I can take a stab at it 