mlfoundations / patching Goto Github PK

Hi loved the work!

I am trying to apply patching to a different dataset, is there any documentation regarding the structure of the input patching dataset?
If not could you please describe it?

Thanks!

I hope this email finds you well. My name is [Your Name], and I am currently working on [Your Project/Research Topic]. I am very interested in using [Your Model/Software Name] for my work.

While attempting to fine-tune the model on the SUN397 dataset, I encountered the following error: AssertionError: Unsupported dataset: SUN397. Supported datasets: ['CIFAR10', 'CIFAR100', 'Cars', 'DTD', 'EuroSAT', 'EuroSATVal', 'FashionMNIST', 'GTSRB', 'ImageNet', 'KITTI', 'KITTIVal', 'MNIST', 'MTSD', 'RESISC45', 'STL10', 'SVHN']

I was wondering if there is a way to resolve this issue or if you have plans to include SUN397 in the list of supported datasets in the future.

Any guidance or advice on this matter would be greatly appreciated.

Thank you for your time and consideration. I am looking forward to your response.

Hello, thank you for sharing the great research!

My question is, is this code version easily compatible to the work https://github.com/mlfoundations/wise-ft ??
(since that work is from same research group)
Or, is the overall code structure modified?

Thank you.

Hello,
I tried to check the CIFAR100 dataset fine-tune performance.

When I finetuned and evaluated both with CIFAR100 dataset,
(Without any code modification, I used below command.
I followed the command written in Readme for MNIST, except changing #epochs and #batch_size)
python src/patch.py --train-dataset=CIFAR100 --epochs=1 --lr=0.00001 --batch-size=16 --model=ViT-L/14 --eval-datasets=CIFAR100 --results-db=results.jsonl --save=models/patch/ViTL14 --data-location=~/data --alpha 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

The result I obtained was
Classification head for ViT-L/14 on CIFAR100 exists at models/patch/ViTL14/head_CIFAR100.pt
Loading classification head from models/patch/ViTL14/head_CIFAR100.pt
Files already downloaded and verified
Files already downloaded and verified
CIFAR100 Top-1 accuracy: 0.922

When I see the paper, in Figure 14, it seems that the best fine-tune performance on CIFAR100 is below 80%.
So I am little surprised the high score. Is it due to the train set difference? Or did I do any mistake?

Thank you.