pre-trained and fine-tuned models about droppos HOT 4 OPEN

Due to limited storage, we could only upload ViT-B checkpoints using google drive. Here is the link: https://drive.google.com/drive/folders/1gDsfYoCllMHa7sYXEsrdOKk74WoT-XQR?usp=drive_link

from droppos.

It is trained for 800 epochs.

from droppos.

Hi, thanks for releasing them. Is DropPos_pretrain_vit_base_patch16.pth trained for 800 epochs or 200 epochs?

from droppos.

Are you also fine-tuning 800 epochs model for 100 epochs? I can only achieve ~83.2% with this setup

--batch_size 1024 \
    --accum_iter 1 \
    --model vit_base_patch16 \
    --finetune DropPos_pretrain_vit_base_patch16.pth \
    \
    --epochs 100 \
    --warmup_epochs 5 \
    --blr 1e-3 --layer_decay 0.8 --weight_decay 0.05 \
    --drop_path 0.1 --reprob 0.25 --mixup 0.8 --cutmix 1.0 \
    --dist_eval \
    --data_path /path \
    --nb_classes 1000 \

But If I train DropPos_mae_vit_base_patch16_dec512d2b for 200 epochs and fine-tune for 100 I get 82.91% with the same setup, so it this really close to the paper. What could be the problem?

from droppos.