I am trying to make predictions with a Pyro Bayesian neural network, and while the validation errors from the Stochastic Variational Inference during the training phase are very small (meaning the Bayesian network is very well trained), when I try to make predictions based on this Pyro neural network model, I am getting the accuracy rate (% of correct predictions from the test set) of 0%. I don’t know why this is happening.
My likelihood function is a multinomial likelihood, which I specified in my model statement as below:
# define likelihood function for our Bayesian layer. class MyModel(PyroModule): def __init__(self, model, name=""): super( ).__init__( ) self.model = model def forward(self, input, true_answer = None, y=None): if true_answer != None: classification_scores = model(input) softmax_tensor = nn.Softmax(dim=-1)(classification_scores) if true_answer == torch.tensor(): true_answer_tensor = torch.tensor([[1.,0.,0.,0.]]) elif true_answer == torch.tensor(): true_answer_tensor = torch.tensor([[0.,1.,0.,0.]]) elif true_answer == torch.tensor(): true_answer_tensor = torch.tensor([[0.,0.,1.,0.]]) elif true_answer == torch.tensor(): true_answer_tensor = torch.tensor([[0.,0.,0.,1.]]) else: classification_scores = model(input) softmax_tensor = nn.Softmax(dim=-1)(classification_scores) true_answer_tensor = None # specify the multinomial likelihood for my `y` return pyro.sample("y", dist.Multinomial(1, probs = softmax_tensor), obs = true_answer_tensor)
and then, I am trying to make predictions like below:
# turn on the evaluation mode. model = MyModel(original_model) model.eval() pred_obj = Predictive(model, guide=my_guide, num_samples = 100) prediction = pred_obj.forward(input) prediction_tensor = prediction["y"].squeeze(1) prediction_tensor = torch.sum(prediction_tensor, dim=0) my_prediction = torch.argmax(prediction_tensor).tolist()
I am thinking either my model definition and/or the way I use the
Predictive function is wrong.
Is there any error in my code? It might have to do with the fact that I am not explicitly returning
my_guide in my training function. Do I need to explicitly return the trained
my_guide at the end of my training function, in order to make my
Predictive() statement to properly refer to the trained
my_guide (as opposed to
my_guide before the training)?