kaiyangzhou / pytorch-center-loss Goto Github PK
View Code? Open in Web Editor NEWPytorch implementation of Center Loss
License: MIT License
Pytorch implementation of Center Loss
License: MIT License
Hi, I tried to use your package directly in my network. However, the param.grad is NoneType and param.grad.data cannot be got.. Could you help me to explain it? Thanks!
Hi, thanks for your source code. But I notice in the line:
https://github.com/KaiyangZhou/pytorch-center-loss/blob/master/center_loss.py#L47
you wrote the center loss as the mean of dis, however, in the paper, the center loss is just the sum of the dis, so could you explain the reason? Thanks a lot
I'm afraid that u're writing the wrong code for the classification loss. In pytorch, the softmax cross entropy loss should be
NLL + LogSoftmax
or
nn.CrossEntropyLoss + logits
Hi, Kaiyang. The feature embedding is 2-dim in this repository and it is straightforward to visualize.
But if the feature is in a high-dimension space (e.g 2048, ResNet50 GAP feature). How I visualize
it in 2-d feature space, should I utilize the approaches such as PCA or TSNE ?
A concise and easy to understand version
class CenterLoss(nn.Module):
def __init__(self, num_class=10, num_feature=2):
super(CenterLoss, self).__init__()
self.num_class = num_class
self.num_feature = num_feature
self.centers = nn.Parameter(torch.randn(self.num_class, self.num_feature))
def forward(self, x, labels):
center = self.centers[labels]
dist = (x-center).pow(2).sum(dim=-1)
loss = torch.clamp(dist, min=1e-12, max=1e+12).mean(dim=-1)
return loss
Thanks the auther about the implementaion of center loss.But i have a question about will the feat_dim in center_loss have an negative effect on the result ? My experiment was on the res_net18 and the feat_dim is 2048,after using the center_loss,the accuracy has a sharp decrease from 80% to 60%.
请问,可以单独使用center loss不加softmax loss吗,效果怎么样
请问 优化算法可以不用SGD用Adam吗
What are the axis in the plots shown in the README file?
How can we achieve it?
Hi, would it be possible for you to share your pre-learned model? Thank you.
Hi~
May I ask whether the implementation in this repo will be equivalent to the original implementation in paper?
# by doing so, weight_cent would not impact on the learning of centers
for param in criterion_cent.parameters():
param.grad.data *= (1. / args.weight_cent)
Simply let alpha=alpha/lambda
, (alpha is lr_cent, lambda is weight_cent), will it equivalent to previous implementation?
It seems the author do not adopt gradient w.r.t. c_j, and instead, use the delta rule shown below.
Hello authors and contributors of center loss, thanks for the impressive work. I got a question as I noticed the center update is based on each mini-batch, and if I had a small batch and many classes (way more than batch size), the center update may become tricky, and I wonder if using momentum or other optimizers is necessary for training. Thanks.
Packages:
torch 1.5.0
torchvision 0.6.0
Code:
distmat.addmm_(1, -2, x, self.centers.t())
Warning:
..\torch\csrc\utils\python_arg_parser.cpp:756: UserWarning: This overload of addmm_ is deprecated:
addmm_(Number beta, Number alpha, Tensor mat1, Tensor mat2)
Consider using one of the following signatures instead:
addmm_(Tensor mat1, Tensor mat2, *, Number beta, Number alpha)
Fix:
distmat.addmm_(x, self.centers.t(), beta=1, alpha=-2)
In my project experiment, the center loss does not decrease with the number of iterations.It seemingly irregular changes. I don't quite understand what's going on. I'd like to ask you about it.Thanks.
Hello!
I am trying to train a model with central loss. My dataset has 901 classes. I am creating mini batches after shuffling the training data with batch size of 128 such that:
feats_dim = [128,512] per batch
labels_dim =[128] per batch
Then:
center_loss = CenterLoss(num_classes = ????, feat_dim = 512, use_gpu=False)
What should I pass in num_classes?
901: actual number of classes
128: classes in current batch
I don't know how Xi changes with depth.
请问,公式里说Xi是随深度变化的特征维度,代码里是怎么体现的,没看出来?feat_dim的设置是输入的维度吗,可以保证优化的是最后的输出维度吗?我用于的数据不是图片数据集。请大佬回答我的疑惑,谢谢。
Hi @KaiyangZhou,
Could you please kindly provide the code used for the visualization of the feature learning process?
I want to use ResNet50 as my model and change Fully Connection layer to classify images.
#Create model:
model = models.resnet50(pretrained=True)
fc_inputs = model.fc.in_features
class FClayer(nn.Module):
def init(self):
super(FClayer, self).init()
self.fc1 = nn.Linear(fc_inputs, 2)
self.relu1 = nn.ReLU()
self.fc2 = nn.Linear(2, num_classes)
def forward(self, x):
x = self.relu1(self.fc1(x))
y = self.fc2(x)
return x, y
model.fc = FClayer()
...
#In train function:
features, outputs = model(inputs)
alpha = 1
loss = loss_criterion(outputs, labels) + (loss_criterion_cent(features, labels) * alpha)
optimizer.zero_grad()
optimizer_cent.zero_grad()
loss.backward()
optimizer.step()
for param in loss_criterion_cent.parameters():
param.grad.data *= (1./alpha)
optimizer_cent.step()
I dont know why when i was training, at the very first epochs, the accuracy had increased, but soon later, nothing happended. Here is my accuracy plot curve chart:
Do I have something wrong?
I use your center loss in my own vgg19. Although my loss decreasing slowly, but the acc on test set never changed, I'm sure I update both opimizer's params, what's wrong with my code?
when i put centerloss's parameters into optimizer, raise valueError("optimizer got an empty parameter list")
optimizer_centloss = torch.optim.SGD(criterion_cent.parameters(), lr=args.lr_cent)
Congratuations! You have done a great work. But I have a question about the center loss.
Can you explain why you wrote there two lines?
Line 128 in 082ffa2
Gradient exploding when I set ‘weight-cent’ to 200+,then I got a loss value(Nan). why?
At the beginning of training, the center loss is very large, for example, 520, although the use of pre-trained model.
PS. I use the arcface as pre-train before adding center loss
The center loss is not a regular change.
in the center_loss.py, the shape of ground truth label should be (batch_size), not num_classes
and I think it can be good, if you decrease using of cuda() in center_loss.py
Hello,thanks for your projects,when i use my own database something is wrong,the problems is:
Traceback (most recent call last):
File "main.py", line 213, in
main()
File "main.py", line 96, in main
train(model, criterion_xent, criterion_cent,optimizer_model, optimizer_centloss, train_loader, use_gpu, 2, epoch)
File "main.py", line 120, in train
for batch_idx, (data, labels) in enumerate(trainloader):
ValueError: too many values to unpack (expected 2)
I don't know how to fix it.thank you very much.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.