vinairesearch / misca Goto Github PK

Hi I tried to train MISCA from scratch but there seems to be missing files. May I know if anyone has managed to resolve this n how?

Hello, I am very interested in your research work.

May I ask if the Overall (ACC) in the paper is the semantics_frame_acc in the experiment？

when I train the MISCA model, the following command is
python main.py --token_level word-level \ --model_type roberta \ --model_dir misca \ --task mixatis \ --data_dir data \ --attention_mode label \ --do_train \ --do_eval \ --num_train_epochs 100 \ --intent_loss_coef 0.5 \ --learning_rate 1e-5 \ --num_intent_detection \ --use_crf \ --base_model dir_base \ --intent_slot_attn_type coattention
Finally, I got a low expected f1 score. What's wrong with it?

The reproduction of the results on Overall is not very good. I ran it on V100, and here are my parameter settings and experimental results. May I ask what the reason is, or how should I reproduce it correctly? Thank you!
python main.py --token_level word-level
--model_type roberta
--model_dir dir_base
--task mixatis
--data_dir data
--attention_mode label
--do_train
--do_eval
--num_train_epochs 100
--intent_loss_coef 0.5
--learning_rate 1e-5
--train_batch_size 32
--num_intent_detection
--use_crf

python main.py --token_level word-level
--model_type roberta
--model_dir misca
--task mixatis
--data_dir data
--attention_mode label
--do_train
--do_eval
--num_train_epochs 100
--intent_loss_coef 0.5
--learning_rate 1e-5
--num_intent_detection
--use_crf \
--base_model dir_base
--intent_slot_attn_type coattention

I first train the base model using bert backbone,the following command is
python main.py --token_level word-level
--model_type bert
--model_dir dir_base
--task my dataset
--data_dir data
--attention_mode label
--do_train
--do_eval
--num_intent_detection
--use_crf,
and then loads dir_base model,the following command is
python main.py --token_level word-level
--model_type bert
--model_dir misca
--task my dataset
--data_dir data
--attention_mode label
--do_train
--do_eval
--num_intent_detection
--use_crf
--base_model dir_base
--intent_slot_attn_type coattention,
however, the result still low.

while loading model its not able to load model due to no config.json file present in the generated model directory.

OSError: misca does not appear to have a file named config.json. Checkout 'https://huggingface.co/misca/main' for available files.
also after loading model while prediction its asking for sequence_length and heads which is not present in the inputs dictionary.

Hi, can I check if line 29 of predict.py has a missing attribute?

Also, when i try to run predict.py with this command
python predict.py --input_file ./data/sample_pred_in.txt --output_file ./data/sample_pred_out.txt --model_dir ./dir_newmodel_3

i get this error
TypeError: JointRoberta.forward() missing 2 required positional arguments: 'heads' and 'seq_lens'

Hello, i tried to reproduce model performance by train from scratch and from given base model checkpoint. However, none of these way produce performance claimed in paper. Could you give me more details to reproduce?

I download given checkpoint in best_model folder and run evaluate :
** evaluate scrips **
python main.py --token_level word-level
--model_type roberta
--model_dir misca
--task mixatis
--data_dir data
--attention_mode label
--do_train
--do_eval
--model_dir best_model
--num_train_epochs 100
--learning_rate 1e-5
--num_intent_detection
--use_crf
--intent_slot_attn_type coattention

However, i got much lower performance.