Comments (2)
I used 4 tesla v100(32G) gpus, batch_size=14(16 is OOM) to reproduce Dog and kept the same config with your paper, but the acc is only 90.5, a big difference from the paper 92.3.
from transfg.
I used 4 tesla v100(32G) gpus, batch_size=14(16 is OOM) to reproduce Dog and kept the same config with your paper, but the acc is only 90.5, a big difference from the paper 92.3.
Hi Oliver, just wondering if you can share your pretrained model with me? Thanks in advance!
Thanks for your reply, i use the pretrained model "VIT_B16" downloaded from your link. By the way, i removed the "part_select" and "part_layer"(like pure vit), the performance is similar with TransFG which i reproduced 90.5.
`
class Encoder(nn.Module):
def init(self, config):
super(Encoder, self).init()
self.layer = nn.ModuleList()
for _ in range(config.transformer["num_layers"] - 1):
layer = Block(config)
self.layer.append(copy.deepcopy(layer))
# self.part_select = Part_Attention()
# self.part_layer = Block(config)
self.part_norm = LayerNorm(config.hidden_size, eps=1e-6)
def forward(self, hidden_states):
# attn_weights = []
for layer in self.layer:
hidden_states, _ = layer(hidden_states)
# attn_weights.append(weights)
# part_num, part_inx = self.part_select(attn_weights)
# part_inx = part_inx + 1
# parts = []
# B, num = part_inx.shape
# for i in range(B):
# parts.append(hidden_states[i, part_inx[i,:]])
# parts = torch.stack(parts).squeeze(1)
# concat = torch.cat((hidden_states[:,0].unsqueeze(1), parts), dim=1)
# part_states, part_weights = self.part_layer(concat)
# part_encoded = self.part_norm(part_states)
part_encoded = self.part_norm(hidden_states)
return part_encoded
`
from transfg.
Related Issues (20)
- How to visualize the attention map in TransFG
- How to fix the RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasGemmEx HOT 3
- How to visualize the attention map in TransFG ?
- Normalization parameters HOT 2
- [minor error]The linear layer self.out of Attention in file modeling
- ImportError: cannot import name 'Di1stributedDataParallel' from 'apex.parallel' HOT 2
- Pip won't find requirements HOT 1
- memory error
- NAbirds dataset
- About visualization
- About trainint details HOT 4
- Would you like to open source the implementation based on [DeiT] pretrained on ImageNet-1K with distillation fine-tuning. HOT 1
- About the training details HOT 1
- How to pretrain TransFG on my own dataset HOT 1
- about Part Selection Module
- About INatureList17 acc HOT 2
- About valid accuracy
- dataset help
- About PSM module
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from transfg.