Hello,
I want to define a custom guide (without EasyGuide) for my model.
Here is a brief description of my model:
- I have two static parameters (“mu” and “phi”) which for now I assumed them to be unconstrained and Normal.
- I have also latent variables of “h_1” to “h_T” following a Markov sequence.
def model(data):
mu = pyro.sample('mu', dist.Normal(0.0, 0.1))
phi = pyro.sample('phi', dist.Normal(0.0, 0.05))
sigma = torch.tensor([0.05])
N = 1
with pyro.plate('Latent params', N) as n:
h_t = pyro.sample('h_0', dist.Normal(mu, sigma/(torch.sqrt(1-phi)**2)))
for t in range(1, T):
h_t = pyro.sample(f"h_{t}", dist.Normal(mu+phi*(h_t-mu), sigma))
y_t = pyro.sample(f"y_{t}", dist.Normal(0.0, torch.exp(h_t/2)), obs=data[t]) # observations
At previous step, I implemented the “AutoNormal” guide for my model and it worked fine. Now, I want to have the most nearest (possible) guide to the “AutoNormal” by a manual derivation. Following the tutorial, I wrote the below guide. But I receive an error for running the SVI.
I assume the error that I receive is directly related to the parameters definitions such as “h_loc = pyro.param(‘AutoNormal.locs.h_0’, mu_loc)” which I have a parameter which I would like to define its posterior according to one another parameter “mu_loc”. I do not know whether I am allowed to do so, or I should just put a constant tensor instead of “mu_loc”? This is also my concern for defining the guide for the loop section that I have in my model, which relates the mean of each step to the mean of previous step. Do I have the degree of freedom to initialize their locs and scales as a sequence with just initialization for the first of them, or I should put constant values for each of them?
I also tried the constant values for the initialization “h” s and it runs without error. But it has really unreasonable almost constant parameters along the convergence plots. I wonder if there is a methodology to set the locs and scales values in a reasonable bounds, and is this the procedure that the AutoNormal follows? (I read the source code, but confused there as I am not expert…)
def guide(data):
mu_loc = pyro.param('AutoNormal.locs.mu', torch.zeros(1))
mu_scale = pyro.param('AutoNormal.scales.mu', 0.1*torch.ones(1), constraint=constraints.positive)
mu = pyro.sample('mu', dist.Normal(mu_loc, mu_scale))
phi_loc = pyro.param('AutoNormal.locs.phi', torch.zeros(1))
phi_scale = pyro.param('AutoNormal.scales.phi', 0.05*torch.ones(1), constraint=constraints.positive)
phi = pyro.sample('phi', dist.Normal(phi_loc, phi_scale))
sigma = torch.tensor([0.6])
## LATENTS ##
h_loc = pyro.param('AutoNormal.locs.h_0', mu_loc)
h_scale = pyro.param('AutoNormal.scales.h_0', sigma, constraint=constraints.positive)
h_t = pyro.sample('h_0', dist.Normal(h_loc, h_scale))
for t in range(1, samps):
h_loc = pyro.param(f"AutoNormal.locs.h_{t}", mu_loc+phi_loc*(h_loc-mu_loc))
h_scale = pyro.param(f"AutoNormal.scales.h_{t}", sigma, constraint=constraints.positive)
h_t = pyro.sample(f"h_{t}", dist.Normal(h_loc, h_scale))