Super Vision Transformer (Link)
Pytorch implementation of our paper "Super Vision Transformer"
We present a novel training paradigm that trains only one ViT model at a time, but is capable of providing improved image recognition performance with various computational costs. Here, the trained ViT model, termed super vision transformer (SuperViT), is empowered with the versatile ability of solving incoming patches of multiple sizes as well as preserving informative tokens with multiple keeping rates (the probability of keeping tokens) to achieve a good hardware efficiency for inference.
- checkpoints:Google Drive
- log: Google Drive
| token number\keep ratio | 0.5 | 0.7 | 1 |
|---|---|---|---|
| 8x8 | 73.75 | 75.28 | 75.79 |
| 10x10 | 77.32 | 78.31 | 78.53 |
| 12x12 | 78.91 | 79.62 | 79.87 |
| 14x14 | 79.95 | 80.50 | 80.62 |
- checkpoints:Google Drive
- log: Google Drive
| token number\keep ratio | 0.5 | 0.7 | 1 |
|---|---|---|---|
| 8x8 | 76.64 | 79.82 | 80.71 |
| 10x10 | 79.81 | 81.73 | 82.24 |
| 12x12 | 81.12 | 82.59 | 82.94 |
| 14x14 | 82.11 | 83.15 | 83.47 |
- python 3.9.7
- pytorch 1.10.1
- torchvision 0.11.2
- The ImageNet dataset should be prepared as follows:
ImageNet
├── train
│ ├── folder 1 (class 1)
│ ├── folder 2 (class 1)
│ ├── ...
├── val
│ ├── folder 1 (class 1)
│ ├── folder 2 (class 1)
│ ├── ...
- Evaluate SuperViT(DeiT-S) on ImageNet
python -m torch.distributed.launch --nproc_per_node=4 main_deit.py --model super_deit_s --batch-size 256 --data-path PATH_TO_IMAGENET --dist-eval --output PATH_TO_LOG --eval --resume PATH_TO_CHECKPOINTS
- Evaluate SuperViT(LV-ViT-S) on ImageNet
python -m torch.distributed.launch --nproc_per_node=4 main_lvvit.py PATH_TO_IMAGENET --model super_lvvit_s -b 256 --native-amp --drop-path 0.1 --token-label --token-label-data PATH_TO_TOKENLABEL --token-label-size 14 --model-ema --output PATH_TO_LOG --eval --resume PATH_TO_CHECKPOINTS
- Train SuperViT(DeiT-S) on ImageNet
python -m torch.distributed.launch --nproc_per_node=4 main_deit.py --model super_deit_s --batch-size 256 --data-path PATH_TO_IMAGENET --dist-eval --output PATH_TO_LOG
- Train SuperViT(LV-ViT-S) on ImageNet
python -m torch.distributed.launch --nproc_per_node=4 main_lvvit.py PATH_TO_IMAGENET --model super_lvvit_s -b 256 --native-amp --drop-path 0.1 --token-label --token-label-data PATH_TO_TOKENLABEL --token-label-size 14 --model-ema --output PATH_TO_LOG
Our code of LV-ViT is from here. Our code of DeiT is from here.
First author: [email protected]; Second author: [email protected]
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent