Stop updating some params during svi.step

1

I have registered the params of two neural nets in a guide with

pyro.module("net1", net1)
pyro.module("net2", net2)

I’d like to do some svi.steps with the weights in net1 fixed. And then do the same with the weights in net2 fixed. How do I do this?

I tried to do

svi_1 = pyro.infer.SVI(model=model, 
                     guide=guide_with_net1on_net2off, 
                     ...)
# svi_1 steps...
svi_2 = pyro.infer.SVI(model=model, 
                     guide=guide_with_net1off_net2on, 
                     ...)
# svi_2 steps...

But the weights in net1 changed by doing svi_2.step

2

Nevermind, I just had to add pyro.clear_param_store() before svi_2 = pyro.infer.SVI(...)

1 Like
3

Oh right, I bet the names collided between your two nets, so without clearing the param store, one net’s parameters were aliased to the other. I guess this is an “unintended weight tying” bug :laughing:

4

Why do you think I had to stop the freeze the weights for the net that was predicting the weights on one of the latents, and then turn it on again? Could it be related to the step size being on a different scale for the two distributional parameters? One is a gaussian, the other a ProjectedNormal. I needed to freeze the weights on the encoder for the ProjectedNormal.