Weighted SVI?

1

Hi All,

I’m looking a way how to do a weighted SVI. I mean, I have weights for every data point and I would like to use it in the my pyro model.

2

you can use poutine.scale for that purpose. see for example here (this example scales by a scalar but you can also use a vector or any tensor with correct shape)

2 Likes
3

thank you!

if it’s possible, could you please point me to the math description of this approach? I would like to understand it and add it to my paper.

4

the scale simply scales log probability terms in the enclosed context.

so for example if your ELBO has a likelihood term that looks like

sum_i log_likelihood_i(data_i)

and you enclose the corresponding sample statement in a scale context using a scale vector scale_i then the corresponding term in the ELBO becomes

sum_i scale_i log_likelihood_i(data_i)

note that care needs to be taken wrt infs/nans since e.g. 0 * nan != 0

1 Like
5

thank you again, it really helps.