Comments (2)
感谢您对我们工作的认可。
对比学习损失的权重设置为 0.1 的原因主要在于它的数量级较大。由于对比学习中温度超参设置为 0.5,在最好的情况下,正样本特征之间 cosine_similarity = 1,256 个负样本之间的 cosine_similarity = -1,此时经过 softmax 后,其概率分别为 0.1758 和 0.0032,导致它的损失为 1.74.
换言之,一个 “完美” 的模型,对比学习提供的 loss 将达到 1.74,对比 supervised loss 的量级 (后期可下降至 0.1),我们设定权重为 0.1
from u2pl.
您可通过以下代码进行数量级的估算
import torch
import torch.nn.functional as F
a = -torch.ones((1, 257))
a[0, 0] = 1
b = F.softmax(a / 0.5, dim=0)
labels = torch.zeros((1,)).long()
print(b, F.cross_entropy(a / 0.5, labels))
from u2pl.
Related Issues (20)
- 对compute_unsupervised_loss计算thresh的疑惑 HOT 1
- DDP loss计算疑问 HOT 2
- 关于在val里使用center crop 的问题 HOT 2
- AEL分支中model_teacher.eval()疑惑 HOT 1
- 对比损失的变化 HOT 4
- Unsupervised loss is 0 HOT 4
- 方法同样适用于二分类任务吗? HOT 1
- sup loss逐渐为0,con loss一直为0
- GPU数量必须按你们的建议来吗? HOT 2
- train_semi.py报错,找不到一个方法 HOT 1
- 数据集疑问 HOT 6
- windows上运行
- 数据集问题 HOT 3
- 数据集 HOT 11
- 损失函数缺乏定义 HOT 2
- 关于两种数据集训练的差异问题 HOT 4
- 关于DPA与threshold的区别
- How to split the datasets (cityscapes) into labeled and unlabeled? HOT 2
- 请问 训练中带标签数据必须和无标签数据数量相等吗? HOT 1
- compute_binary_memobank_loss HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from u2pl.