alexrame / diwa Goto Github PK

Hi,

Thank you for sharing the code.
When I was using your code, I think I found some problems. In the code here, you try to get all the env{i}_out_Accuracies/acc_net. It was called here.

I think there is no such metric in the results, and the logic here is not right. If my test_env=0, then after first iteration, the for loop will break, and fails to collect the rest env_i_acc.

Can you take a look at it?

Best regards.

In datasets files datasets classes define enviroment as a list by in the script file it is an integer.
also i used test_env =0 , trial_seed =0 with the default training code on OfficeHome and got accuracy arrpython -m scripts.diwa --data_dir=/input/domainbed/ --output_dir=sweep_output --dataset OfficeHome --test_env 0 --weight_selection uniform --trial_seed 0ound 67% .
I used the following commands:
python -m scripts.train --algorithm ERM --dataset OfficeHome --test_env 0 --init_step --path_for_init init_checkpoint.pth
follown by:
python -m scripts.sweep launch --data_dir /input/domainbed/ --output_dir sweep_output --command_launcher multi_gpu --datasets OfficeHome --test_env 0 --path_for_init init_checkpoint.pth --algorithms ERM --n_hparams 20 --n_trials 3
and
python -m scripts.diwa --data_dir=/input/domainbed/ --output_dir=sweep_output --dataset OfficeHome --test_env 0 --weight_selection uniform --trial_seed 0

Maybe that is due to my computer, but i am wondering if i am using the exact setting in the right way.
What is the role of the variable --test_env ${test_env}
what are the possible values of test_env and their meaning?

thank you

Hello.
In
python3 -m domainbed.scripts.train\ --data_dir=/my/data/dir/\ --algorithm ERM\ --dataset OfficeHome\ --test_env ${test_env}\ --init_step\ --path_for_init ${path_for_init}\ --steps ${steps}\
you said that if
random initialization, set steps to -1: there will be no training.
Linear Probing, ICLR2022, set steps to 0: only the classifier will be trained.

However, i could not find that part in the code.
what i saw is if init_step=0 train everithing elif int_step=-1 training classifier head only in algorithm.py from line 109 to 113.

also if step=-1 in line 201 n_steps will be set to -1 in (n_steps = args.steps or dataset.N_STEPS)
could you correct me

EOFError: Compressed file ended before the end-of-stream marker was reached