Hello, first post on the forum, hope it is in the right place.
Basically I have two agents, the parameters of which are being fitted using SVI and Adam as an optimizer. The two agents do exactly the same thing in a slightly different way. The model for both looks like this:
def model():
param_1 = pyro.sample('param_1', dist.Beta(alpha_1, beta_1))
....
param_n = pyro.sample('param_n', dist.Beta(alpha_n, beta_n))
agent = initialize_agent(param_1,...., param_n)
for trial in ntrials:
for t in nt:
observation = data['observation'][tau,t]
rewards = data['reward'][tau,t]
posterior_actions = agent.update_beliefs(reward,observation)
response = self.data["actions"][tau, t]
pyro.sample('response_{}_{}'.format(tau, t), Categorical(posterior_actions), obs=response)
So you initialize an agent, pass it the simulated data (observations, rewards, responses) and try and find the parameters that best explain the simulated behaviour. The two agents differ in how agent.update_beliefs() is implemented but ultimately compute the same thing.
My problem is that for one agent the ELBO converges and for the other it doesn’t, even though everything is the same (model, guide, parameters, etc). To debug this I set the same seed for the fitting and compared output for the two agents line be line.
As expected the initial parameter guesses are the same, the calculated posterior_actions is the same, the calculated ELBO is the same at the first iteration but then I was surprised that I got different parameter estimates at the next iteration. So my question is, shouldn’t setting the seed at the beginning ensure the same behavior for the ADAM optimizer if the calculated sample probabilities and the ELBO passed are the same between the two agents? What happens under the hood in pyro which causes the difference?