Hi All,

I am looking for a way to penalize different posterior distributions arising from different training sets.

To be more precise, assume we are given the Pyro (shortened) tutorial example:

```
def model(is_cont_africa, log_gdp=None):
a = pyro.sample("a", dist.Normal(0.0, 10.0))
b_a = pyro.sample("b_a", dist.Normal(0.0, 1.0))
sigma = pyro.sample("sigma", dist.Uniform(0.0, 10.0))
mean = a + b_a * is_cont_africa
with pyro.plate("data", len(is_cont_africa)):
pyro.sample("obs", dist.Normal(mean, sigma), obs=log_gdp)
```

Now, in each epoch I want to penalize a difference in the posterior distributions over `b_a`

given two different input data sets, `x`

and `y`

, where `y`

includes additional samples compared to `x`

. `x`

is used for ELBO gradient estimate, though.

Hence, instead of only maximizing the ELBO only I want to optimize the following objective:

```
max ELBO - KL[p(b_a | x) || p(b_a | y)]
```

Is there any way to achieve something like this in pyro?

If this is not implemented yet, but intended, I am happy to contribute!