The pytorch nll loss documents how this aggregation is supposed to happen but as far as I can tell my implementation matches that so I'm at a loss how to fix it.Įdit: I'm going to add that I think it's overwhelmingly likely that the variables wt and logpt that I use to calculate the loss on line A are correct, given that the non-aggregated losses match the output of nn.CrossEntropyLoss. I'm sort of ok with the results of experiment 2, the sum is only off by a few parts per hundred thousand, but I'd like to fix it to match the official implementation if possible.Įxperiment 3 however shows that my mean aggregation is just incorrect. At this point I'm matching the reference implementation to full precision. Note that the equals function checks exact equality. We can tell from experiment 1 that line A computes the correct weighted losses. Print(torch.equal(my_loss, official_loss), my_loss, official_loss) False tensor(3.5066) tensor(3.5072) My_cel = M圜rossEntropyLoss(weight=weights, ignore_index=-1, reduction='mean')Ĭel = nn.CrossEntropyLoss(weight=weights, ignore_index=-1, reduction='mean') My_cel = M圜rossEntropyLoss(weight=weights, ignore_index=-1, reduction='sum')Ĭel = nn.CrossEntropyLoss(weight=weights, ignore_index=-1, reduction='sum') Print(torch.equal(my_cel(logits, truth), cel(logits, truth))) # True Experiment 2: My_cel = M圜rossEntropyLoss(weight=weights, ignore_index=-1, reduction='none')Ĭel = nn.CrossEntropyLoss(weight=weights, ignore_index=-1, reduction='none') Return loss Simulate a semantic segmentation minibatch with 8 images, 32 classes and 128x128 pixels # No aggregation, just return the raw values # Some code that I don't have questions about:Ī loss = - wt * logpt # mb, d1, d2. Self.reduction = reduction def forward(self, input_, target): Can anyone tell me how to fix my loss aggregation to match the pytorch implementation? Here's my code.ĭef init(self, weight=None, ignore index=-100, reduction='mean'): Currently I get the same loss values as nn.CrossEntropyLoss when I don't aggregate the loss but when I do aggregate the loss then the result starts to diverge from nn.CrossEntropyLoss. Hey all - I'm trying to implement a CrossEntropyLoss layer that reproduces the behavior of the standard torch.nn.CrossEntropyLoss behavior. They prioritize commercial interests over intellectual ones.Ĭhatrooms Official Discord Server Wiki Getting Started with Machine Learning Resources Related Subreddits /r/MachineLearning /r/MLQuestions /r/datascience /r/computervision Machine Learning Multireddit /m/machine_learning Do not share referral links and other purely marketing content.Keep our subreddit fresh by posting your YouTube series or blog at most once a week. Do share your works and achievements, but do not spam.We want to encourage everyone to feel welcomed and not be afraid to participate. Foster positive learning environment by being respectful to others.Feel free to share any educational resources of machine learning.Īlso, we are a beginner-friendly sub-reddit, so don't be afraid to ask questions! This can include questions that are non-technical, but still highly relevant to learning machine learning such as a systematic approach to a machine learning problem. A subreddit dedicated for learning machine learning.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |