yg211 / bert_nli Goto Github PK

In this code, you have provided .dict files for BERT_base, BERT_large, and albert_large. I want to download the .dict file for SciBERT. How can I download that file?

你好，我大致看了一下您的代码，您的方法是，取的pooled_out向量, warm_up + Adamw ,模型参数只有一个线性层，在bert上微调，能够达到约86%的准确率。

我有点关于训练细节的东西想请教一下，取pooled_out 效果会比取-2层好吗，我取-2层，Adam优化器，固定bert参数，只有一个线性层，学习率2e-5，基本只能达到70准确率，我试了微调bert结果基本没有变化，是因为没有warm up 的原因吗，还是说Adamw优化器的原因，您对于这类调参有什么建议吗？

I have trained distilbert model on my custom data. It has completed the training but after completion, it is giving the error. When I checked the .py file I am not getting the cause. Can I get the solution for this?

We are running the code on a different dataset than yours and are getting this error.

[...]
evaluate: 100% 723/724 [02:51<00:00,  5.96it/s]
evaluate: 100% 724/724 [02:52<00:00,  4.20it/s]
All predcit:  7176
All labels:  7237
training:   9% 1999/22916 [11:37<2:01:35,  2.87it/s]
Traceback (most recent call last):
  File "./train.py", line 195, in <module>
    model_dic = train(model, optimizer, scheduler, train_data, dev_data, batch_size, fp16, checkpoint, gpu, max_grad_norm, best_acc)
  File "./train.py", line 90, in train
    acc = evaluate(model,dev_data,checkpoint,mute=True)
  File "/content/drive/MyDrive/bert_nli-master/test_trained_model.py", line 25, in evaluate
    assert len(all_predict) == len(all_labels)
AssertionError

We have not changed anything else. Printing the two values all_predict and all_labels actually gives us different lengths.
We don't understand why, could you help us?

there is no output_hidden_states attribute for module

Sorry to distrupt you.
In my attempt, I found that "transformers" has been upgraded, and in that case the "warmup" function cannot be used. Maybe you could consider renew the code in train_test.py.
For example, WarmupLinearSchedule → get_linear_schedule_with_warmup.

The question is that when I used your datasets and tried to train a new model. An error has been raised. It's in utils.py Line 14, tokenized_text[i] is out of range.

torch==1.0.1.post2 in requirements.txt not work for this experiment, I use torch==1.5.0 instead as you suggest in the readme and it works.

Thanks for your work~

I was trying to run the example script, but it seems to give this error Missing key(s) in state_dict: "bert.embeddings.position_ids".
I tried instantiating model = BertNLIModel('output/bert-base.state_dict') but it gives the same error.
Any idea why?

Hey I am finetuning bert base on custom nli dataset. But while starting a training process it is getting stuck at the error stating that torch.nn.modules.module.ModuleAttributeError: 'BertEncoder' object has no attribute 'output_hidden_states'. Can you solve this as as I want to get this done as soon as possible?

2020-12-11 19:57:14 - Lock 140474735309264 acquired on /Users/mac/.cache/torch/transformers/aa1ef1aede4482d0dbcd4d52baad8ae300e60902e88fcb0bebdec09afd232066.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157.lock
2020-12-11 19:57:14 - https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-pytorch_model.bin not found in cache or force_download set to True, downloading to /Users/mac/.cache/torch/transformers/tmp_34x9lwo
Downloading: 100%|██████████| 440M/440M [3:20:47<00:00, 36.6kB/s]
Traceback (most recent call last):
File "/Users/mac/Downloads/bert_nli-master/train.py", line 181, in
model = BertNLIModel(gpu=gpu, batch_size=batch_size, bert_type=bert_type, model_path=trained_model, reinit_num=reinit_layers, freeze_layers=freeze_layers)
2020-12-11 23:18:04 - storing https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-pytorch_model.bin in cache at /Users/mac/.cache/torch/transformers/aa1ef1aede4482d0dbcd4d52baad8ae300e60902e88fcb0bebdec09afd232066.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157
File "/Users/mac/Downloads/bert_nli-master/bert_nli.py", line 21, in init
self.bert = BertModel.from_pretrained('bert-base-uncased')
2020-12-11 23:18:04 - creating metadata file for /Users/mac/.cache/torch/transformers/aa1ef1aede4482d0dbcd4d52baad8ae300e60902e88fcb0bebdec09afd232066.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157
File "/Users/mac/miniconda3/envs/test_py3/lib/python3.6/site-packages/transformers/modeling_utils.py", line 536, in from_pretrained
model = load_tf2_checkpoint_in_pytorch_model(model, resolved_archive_file, allow_missing_keys=True)
2020-12-11 23:18:04 - Lock 140474735309264 released on /Users/mac/.cache/torch/transformers/aa1ef1aede4482d0dbcd4d52baad8ae300e60902e88fcb0bebdec09afd232066.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157.lock
File "/Users/mac/miniconda3/envs/test_py3/lib/python3.6/site-packages/transformers/modeling_tf_pytorch_utils.py", line 227, in load_tf2_checkpoint_in_pytorch_model
tf_model = tf_model_class(pt_model.config)
NameError: name 'tf_model_class' is not defined
2020-12-11 23:18:04 - loading weights file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-pytorch_model.bin from cache at /Users/mac/.cache/torch/transformers/aa1ef1aede4482d0dbcd4d52baad8ae300e60902e88fcb0bebdec09afd232066.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157
2020-12-11 23:18:05 - Loading TensorFlow weights from /Users/mac/.cache/torch/transformers/aa1ef1aede4482d0dbcd4d52baad8ae300e60902e88fcb0bebdec09afd232066.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157

Process finished with exit code 1
I wanna know how can I solve this problem? I have been downloaded the pre-trained model,but name tf_model_class is not defined.Please help me!