Guide function for inferring posterior on Dirichlet distribution

1

Hi, sorry for another dumb question - I’m trying to use SVI to find the posterior over the parameters of a Dirichlet distribution in a hierarchical model. The parameters are a scalar Alpha and a vector Beta with an exponential prior and a uniform prior respectively.

I’m getting stuck on how to specify the guide function such that I can end up with the posterior over Alpha and Beta, given my data. I know I’m doing something wrong because during inference Alpha increases, whereas the data is best explained by a very small value. Thanks again for the help!

Revelant chunk of code is:

lmbda = 1
bag_samples = 10
colors = 2

data = torch.Tensor([[0., 10.],[10., 0.],[0., 10.],[10., 0.],[0., 10.],[10., 0.],[0., 10.],[10., 0.]])

def model(lmbda=lmbda):
    alpha = 1.0
    beta = torch.ones(colors)/2
    with pyro.plate("thetas", len(data):
        theta = pyro.sample("theta", dist.Dirichlet(torch.FloatTensor(alpha*beta)))
    with pyro.plate("obs", len(data)):
        return pyro.sample("draws", dist.Multinomial(bag_samples, theta), obs = data)    
    
def guide(lmbda=lmbda):
    q_alpha = pyro.param("alpha", dist.Exponential(1/lmbda))
    q_beta = pyro.param("beta", dist.Dirichlet(torch.ones(colors))) 
    with pyro.plate("thetas", len(data)):
        q_theta = pyro.sample("theta", dist.Dirichlet(torch.FloatTensor(q_alpha*q_beta)))
    with pyro.plate("obs", len(data)):
        return pyro.sample("draws", dist.Multinomial(bag_samples, q_theta))
2

You need to specify your prior in the model object. If alpha has a exponential prior you need to specify it here (not in the guide). i.e. (not sure if this is 100% correct syntax):

alpha = pyro.sample("alpha", dist.Exponential(1/lmbda))

Also some more stuff to this code. I propose you go through the tutorials and implement those first :slight_smile:

1 Like