Loss increases with TraceEnum_ELBO, decreases with Trace_ELBO

1

I have a quick question.

I ran my model using three classes:

  1. TraceEnum_ELBO
  2. JitTrace_ELBO
  3. Trace_ELBO

when I ran the model using 1 and 2, the loss increased but when I used 3, it decreased(which should be the case) I am I
finding this behavior happen?
the left column is epoch number and right column is the loss after each epoch in the TraceEnum_ELBO and Trace_ELBO cases

2

Please post your code (ideally a minimum reproducible example), otherwise there’s not much we can say about what’s going wrong.

3

model.JPG1004×672 120 KB
guide.JPG710×740 117 KB
infer.JPG1068×473 68.9 KB

The model and guide are initialized via the constructor of the class.

4

@eb8680_2 Do you why this is happening ? When I use TraceEnum_ELBO() and JitTraceEnum_ELBO(), the loss is decreasing. But If I use Trace_ELBO() andJitTrace_ELBO(), the loss is actually increasing. However, the accuracies on the test set in both cases above is not affected (surprisingly!) Is no training actually happening? Actually my starting priors are sophisticated

5

It’s probably related to sample sites enumerated in your model that are also present in the guide. If you enumerate a variable in your model, you don’t need a corresponding distribution in the guide - it’s already been integrated out.

1 Like