Hi,
I would be really glad if I could get some help.
I am using the Tyxe: Pyro model, according to which I have converted the fc layer to probabilistic layer. [Reasoning about Shapes and Probability Distributions - Eric J. Ma's Personal Site] I also read this link for reference but still I am not able to solve the broadcastable error :
Value is not broadcastable with batch_shape+event_shape: {actual_shape} vs {expected_shape}
net.fc.2.bias dist | 4
value | 4
log_prob |
likelihood.data dist 8 |
value 8 4 |
I am using one-hot encoded label format. example:
tensor([[0., 0., 1., 0.],
[1., 0., 0., 0.],
[1., 0., 0., 0.],
[1., 0., 0., 0.]])
Below Is the code in which I am facing the above error:
training_Set = CustomDataset(images=mono_images[:640], labels=mono_probs[:640], transform=data_transform)
train_loader = DataLoader(training_Set, batch_size=8, shuffle=True)
# model
model = models.resnet18(pretrained=True)
model.conv1 = nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False)
model.fc = nn.Sequential(
nn.Linear(512, 50),
nn.Tanh(),
nn.Linear(50, 4)
)
#guide
prior = tyxe.priors.IIDPrior(dist.Normal(0, 1))
likelihood = tyxe.likelihoods.Categorical(len(training_Set))
inference = tyxe.guides.AutoNormal
bnn = tyxe.VariationalBNN(model, prior, likelihood, inference)
#training
lr = 1e-3
optimizer = pyro.optim.Adam({"lr": lr})
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
training_10epoch = []
pred_params_num_predict_1 = []
with tyxe.poutine.local_reparameterization():
training_10epoch = bnn.fit(train_loader, optim=optimizer, num_epochs=5, callback=None, num_particles=1, closed_form_kl=True, device=device)
I am to certain extent sure that the error I am facing is because of Likelihood but not understanding where exactly in implementation I am making a mistake. [GitHub - TyXe-BDL/TyXe] I also read this article for implementation.
Thank you in advance!