Assume that I have a model like: `f ~ Normal(a, b), y ~ Bernoulli(f).`

How to write a model in Pyro to get likelihood estimate `E_{p(f)} log p(y|f)`

using MC with 10 samples of `f`

? For 1 sample we can write like this

```
def model(y):
f = Normal(a,b)()
return pyro.sample("y", Bernoulli(f), obs=y)
```

Now, I build a guide `q(f)`

for `f`

and want to use an objective like `KL(q(f)||p(f)) + alpha E_{q(f)} log p(y|f)`

where `alpha`

is a scale factor. Is there a way to add a scale factor without rewrite ELBO? This is useful for training models with mini-batch.

For those who interested in this, we can obtain these targets by using `pyro.poutine.scale`

decorator.

2 Likes

Sorry to dig up this old post. But does `pyro.poutine.scale`

solves the first problem in the original post? If I understand correctly `f`

here is not a latent variable and the likelihood is the `E_{p(f)} log p(y|f)`

which is itself an intractable intergral and can be esimated as you suggested by a MC estimator. Can `pyro.poutine.scale`

actually do this, or maybe we need `pyro.poutine.mc_extend`

to do the job?

Side question: Shouldn’t `f`

be constrained to be a fractional? It wouldn’t be a valid Bernoulli parameter otherwise, right?

We can draw a batch of f (say 10) and then scale likelihood to 1/10.

Yes, you are right. A response function (inverse link function) should be used here

Thanks. So if I understand correctly, for the unscaled model, when writing `pyro.sample`

statement for observables, we need to *expand* each data point with the number of samples used to estimate the likelihood (10 in this case). Correspondingly we need to sample 10 samples of non-latent variable `f`

in the model using `torch.distributions`

instead of `pyro.sample`

. Is this the correct way to do it? Or is there a more *poutine* way of doing it? Thanks.

I’ll do that way because I have some specific reason. I don’t know if there is a *poutine* way to do it.

I guess you can set `num_particles=10`

in ELBO to get what you want.

Thanks. Setting num_particles=10 is probably not what I want since if I understand correctly this means how many do you sample from the guide. In my case (or this case) the likelihood itself is an integral and needs multiple samples to estimate. At least using `torch.distribution`

is one correct way to do it.