MCMC and Marginal Likelihood

Misc.
1

I’m following the tutorial at https://docs.pymc.io/notebooks/Bayes_factor.html using Pyro and would like to clarify something for my own understanding.
From the following cell in PyMC3

import pymc3 as pm
import numpy as np

y = np.repeat([1, 0], [50, 50])  # 50 "heads" and 50 "tails"
priors = ((1, 1), (30, 30))
n_chains = 1000

models = []
traces = []
for alpha, beta in priors:
    with pm.Model() as model:
        a = pm.Beta('a', alpha, beta)
        yl = pm.Bernoulli('yl', a, observed=y)
        trace = pm.sample(1000, step=pm.SMC(), random_seed=42)
        models.append(model)
        traces.append(trace)

BF_smc = models[1].marginal_likelihood / models[0].marginal_likelihood
print(round(BF_smc))

I’ve converted the above tutorial code to the following in Pyro:

import numpy as np
import pyro
import pyro.distributions as dist
from pyro.infer import MCMC, NUTS
import torch

def model(y, alpha, beta):
    a = pyro.sample('a', dist.Beta(alpha, beta))
    yl = pyro.sample('yl', dist.Bernoulli(a), obs=y)
    return yl

y = np.repeat([1, 0], [50, 50])  # 50 "heads" and 50 "tails"
priors = ((1, 1), (30, 30))
y_tensor = torch.from_numpy(y)

models = []
traces = []
for alpha, beta in priors:
    nuts_kernel = NUTS(model)
    mcmc = MCMC(nuts_kernel, num_samples=1000, warmup_steps=200)
    mcmc.run(y_tensor, alpha, beta)
    models.append(mcmc)
    mcmc_samples = mcmc.get_samples()
    traces.append(mcmc_samples)

My question is: What is the parallel in pyro for models[1].marginal_likelihood? My ideas so far are to use something in poutine.trace or perhaps some functionality in arviz? I feel that I’m missing something obvious here but extended searches of the docs and examples haven’t revealed anything to me so far. Thanks for the help!

2 Likes