somedaywilldo / bm-nas Goto Github PK

A

May I ask when will the code and datasets be released？

Dear BM-NAS authors,

In the code the seed is fixed with lines:

    np.random.seed(args.seed)
    cudnn.benchmark = True
    torch.manual_seed(args.seed)
    cudnn.enabled=True
    torch.cuda.manual_seed(args.seed)

However, every time I run the code, I get different results. Particularly with MM-IMDB, with the same hyper-parameters and the same seed, I got 62.43%, 62.55% and 62.36% Weighted F-1 scores. I believe, that fixing the score must make sure that we will get the exact same results on each run, can you please let me know where else I shall fix the seed? Furthermore, in the the weighted F1 score on MM-IMDB is reported as 62.92% +- 0.03, can you please explain whether this range is because of the different results after fixing the seed?

Thanks in advance,
Best regards,
Grigor

I would like to ask the author to help me solve the problem. After completing the search, the error occurs when I am preparing to train the searched network.Thank you！

Thanks for providing code and insightful research in this area. However, when I try to execute mmimdb, I meet with several problems.

I meet the problems when I'm trying to execute: " python main_darts_searchable_mmimdb.py " as shown in the experiment section.

Firstly, on line 26 in main_darts_searchable_mmimdb.py, the argument should be "--parallel"? Or it will report a mistake :

And after changing this, I find an error of matrix shape:

I used the provided "python datasets/prepare_mmimdb.py" command to process the multimodal_imdb.hdf5 file.

This problem is solved by changing matrix size to 768.

However, when I'm trying to execute main_darts_found_ntu.py, I got the error mentioned in closed issue on ego and I'm not able to solve it:

Can you help fix this?
Thanks a lot for your kind help!

Hi,

I am interested in using your code for depth completion using two modalities as the ntu dataset. In the readme it mentions it can be done by using main_darts_searchable_ntu.py but I did not see it anywhere in the repository. Will it be published soon? Thank you

Hi:)

First of all, I thank you so much for the codes and the insightful research. Im doing a study on time series using Neural Architecture Search and your repository is helping greatly .

I have some questions regarding phases and status in your code though:(

Firstly,
Where it says, (in BM-NAS/models/search/train_searchable/ego.py)

phases = []
if status == 'search':
phases = ['train', 'dev']
else:
phases = ['train', 'test']
# model.load_state_dict(best_test_model_sd)

-- shouldnt the phases be just ['test'] for 'else'? (=status being 'Eval')
I couldnt understand why 'train' phase was included.:(

Another similar point that confused me was that in the scripts training the found architectures,

(For example, in the arguments of https://github.com/Somedaywilldo/BM-NAS/blob/master/main_darts_found_mmimd.py)

# loading searching experiment, if not None, perform evalution
parser.add_argument('--search_exp_dir', type=str, help='evaluate which search exp', default=None)
# loading evaluation experiment, if not None, perform test
parser.add_argument('--eval_exp_dir', type=str, help='test which eval exp', default=None)

What do 'perform evaluation' and 'perform test' mean here, respectively?

'Evaluation' seemed to actually mean 'train' (thay is, training the found architecture) if I look at the codes, so i was confused again with the use of the words.:(

Thank you again for the great model, and Id be very grateful for your answer.:)