Resnet相比VIT的效果怎么样呢 about vand-april-gan HOT 5 OPEN

Hi! What's your intuition on why auroc_sp is much lower on image vs pixel? And what could be done to improve?

Hi! In the zero-shot setting, the anomaly scores used for classification are obtained by calculating the similarity between class tokens and text features. In this case, the accuracy of classification and segmentation is not significantly relevant. The accuracy of classification primarily depends on the pre-trained model and the prompts. Designing appropriate prompts manually is quite challenging. Perhaps you could try prompt learning to further enhance the classification performance.

Besides, due to the sufficiently high predictive accuracy of anomaly maps, you can also attempt to take their maximum values as anomaly scores. Adding these two parts together can further enhance performance.

from vand-april-gan.

Hi, I'm sorry for replying late.

In our approach, we focused on exploring the CLIP model based on ViTs without conducting extensive experiments on the model based on ResNets. I can provide you with a reference result obtained by training the RN50x16 model for 5 epochs at a resolution of 384 (the standard resolution for this model).

The training command is:

python train.py --dataset mvtec --train_data_path ./data/mvtec \ 
--save_path ./exps/visa/RN50x16 --config_path ./open_clip/model_configs/RN50x16.json --model RN50x16 \ 
--features_list 1 2 3 4 --pretrained openai --image_size 384  --batch_size 8 --aug_rate 0.2 --print_freq 1 \ 
--epoch 5 --save_freq 1 --learning_rate 0.0001

In our experiments, we found that training the linear layers with ResNets is more challenging compared to using ViTs, and it may require more suitable hyperparameters and training strategies. The code now supports changing resolutions, so you can continue your exploration using these code modifications.

Hope that my answer can be helpful to you. :)

from vand-april-gan.

Hi! What's your intuition on why auroc_sp is much lower on image vs pixel? And what could be done to improve?

from vand-april-gan.

@ByChelsea

Do you have any reference paper for prompt learning?

from vand-april-gan.

@ByChelsea

Do you have any reference paper for prompt learning?

I'm sorry, I just noticed this question. You can refer to CoOp and AnomalyCLIP.

from vand-april-gan.