How to integrate pyro model to c++

Is there any c++ API in pyro? There is a model (several .p files) trained in pyro. I want to use it for prediction in c++. What should I do?


Hi @sj2019, great question!

tl;dr your trained model from Python, then torch::jit::load() it into C++.

My team is also exploring ways to serve Pyro models in C++. While I haven’t used the ; torch::jit::load() path myself, I’ve used torch.jit.trace() to speed up models. If you do this, I’d recommend using PyTorch modules and pyro.module() rather than directly storing params in the param store via pyro.param(), because PyTorch modules are a bit easier to I believe a pattern that should work is (again YMMV as I haven’t tried this):

class Model(nn.Module):
    def forward(self, ...):

class Guide(nn.Module):
    def forward(self, ...):

class Predictor(nn.Module):
    def __init__(self, model, guide):
        self.model = model = guide

    # This might need to be torch.jit.traced, I'm not sure.
    def forward(self, data):
        tr = poutine.trace(
        return poutine.replay(self.model, tr)(data)

...train using SVI...

# Finally save the trained predictor.
predictor = Predictor(...), "predictor.torch")

Then I believe you can load your model with something like:

auto predictor = torch::jit::load("predictor.torch")

Let me know how this goes. We could definitely use a tutorial on how to do this, so we welcome feedback about anything you learn. I’ve added this question as a request for tutorial: