Example for custom unnormalized likelihood and custom guid

kayhan · August 23, 2018, 4:27pm

Hi,

I recently started using pyro. I am working on a problem where I only have un-normalized joint distribution and custom q (ie guide). I would like to maximize ELBO. Here is my question:

Is it possible to provide an example where we have only the unnormalized joint probability of the model? This can be done pymc3 by defining a distribution object and passing a function that computes logp. However, defining custom q is a bit restrictive. Defining custom guide in pyro is straightforward but according to this issue defining a custom probability for the model is not advised. Am I missing something:

github.com/pyro-ppl/pyro

User-defined distributions

opened 08:10AM - 22 Sep 17 UTC

closed 04:22AM - 07 Oct 17 UTC

jpchen

discussion

We should allow users to define their own distributions. Came across this probl…em when specifying priors over parameters, where the prior needs to take the shape of the parameter to sample properly. This prior needs to have the same interface as a pyro distribution because the trace methods assumes certain characteristics of a distribution object that a function does not have. I was imagining something like: ```py def sample_fn(*args, **kwargs): //sample dist = pyro.distributions.customDistribution({ 'init': init_fn, 'sample': sample_fn, 'log_pdf': log_pdf_fn, 'batch_log_pdf' (optional): ..., 'support' (optional): ..., }) ``` or have setter methods to do this. Something along those lines

If possible, could you please provide an example that one can pass logp and define a custom distribution?

Thanks

neerajprad · August 24, 2018, 7:07pm

That’s an old discussion thread more related to making it easier for users to define/test custom distribution classes. Since then, we have a much wider support for different distributions most of which have been moved upstream to torch.distributions. It should be straightforward to extend from the base distribution class (take a look at these pyro distributions, for example). I think that for as long as you have a valid log_prob (unnormalized is fine) method for the RVs in your model, and a valid sample method for the RVs in your guide, SVI should work fine.

jpchen · August 27, 2018, 7:35am

as an example, we’ve done exactly this for the von mises distribution. notice that it only has a score method, so it only can be used in the model.