- NLP Engineer, contributing on Korean NLP with Open Source!
monologg / goemotions-pytorch Goto Github PK
View Code? Open in Web Editor NEWPytorch Implementation of GoEmotions ππ’π±
License: Apache License 2.0
Pytorch Implementation of GoEmotions ππ’π±
License: Apache License 2.0
Shouldn't sigmoid(multilabel classification) or softmax(multiclass classification) be applied here to get the predictions? Otherwise, later the thresholds are being applied on the logits directly.
GoEmotions-pytorch/run_goemotions.py
Line 183 in 47f5afb
my input text is too long, so there's an error: RuntimeError: index out of range: Tried to access index 512 out of table with 511 rows. at ../aten/src/TH/generic/THTensorEvenMoreMath.cpp:418. how to solve it?
μλ
νμΈμ μ’μ λͺ¨λ λ§λ€μ΄μ£Όμ
μ κ°μ¬ν©λλ€.
μ κ° cuda νκ²½μμ λͺ¨λΈμ λΆλ¬μμ
pprint(goemotions(text))
λ₯Ό μ€ννλ©΄ μλμμ²λΌ μλ¬κ° λ¨λ€μ.
multilabel_pipeline μμ deviceλ₯Ό -1μμ 0μΌλ‘λ λ°κΏλ³΄κ³ ,
import torch
device = torch.device('cuda:0')
model.to(device)
μ μ²λΌ μΆκ° μ€μ λ ν΄λ³΄μλλ° λμΌν μλ¬κ° λ μ μ¬κΈ°μ μ¬μ€λ³΄κ² λμ΅λλ€.
μ΄λ»κ² κ³ μΉ μ μμκΉμ?
RuntimeError Traceback (most recent call last)
in
----> 1 pprint(goemotions(df['sentences'][0]))
~/ProjComment/IMDB/GoEmotions-pytorch/multilabel_pipeline.py in call(self, *args, **kwargs)
37
38 def call(self, *args, **kwargs):
---> 39 outputs = super().call(*args, **kwargs)
40 scores = 1 / (1 + np.exp(-outputs)) # Sigmoid
41 results = []
/home/ubuntu/anaconda3/envs/GoEmotions-pytorch/lib/python3.8/site-packages/transformers/pipelines.py in call(self, *args, **kwargs)
472 def call(self, *args, **kwargs):
473 inputs = self._parse_and_tokenize(*args, **kwargs)
--> 474 return self._forward(inputs)
475
476 def _forward(self, inputs, return_tensors=False):
/home/ubuntu/anaconda3/envs/GoEmotions-pytorch/lib/python3.8/site-packages/transformers/pipelines.py in _forward(self, inputs, return_tensors)
491 with torch.no_grad():
492 inputs = self.ensure_tensor_on_device(**inputs)
--> 493 predictions = self.model(**inputs)[0].cpu()
494
495 if return_tensors:
/home/ubuntu/anaconda3/envs/GoEmotions-pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
530 result = self._slow_forward(*input, **kwargs)
531 else:
--> 532 result = self.forward(*input, **kwargs)
533 for hook in self._forward_hooks.values():
534 hook_result = hook(self, input, result)
~/ProjComment/IMDB/GoEmotions-pytorch/model.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, labels)
25 labels=None,
26 ):
---> 27 outputs = self.bert(
28 input_ids,
29 attention_mask=attention_mask,
/home/ubuntu/anaconda3/envs/GoEmotions-pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
530 result = self._slow_forward(*input, **kwargs)
531 else:
--> 532 result = self.forward(*input, **kwargs)
533 for hook in self._forward_hooks.values():
534 hook_result = hook(self, input, result)
/home/ubuntu/anaconda3/envs/GoEmotions-pytorch/lib/python3.8/site-packages/transformers/modeling_bert.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask)
724 head_mask = self.get_head_mask(head_mask, self.config.num_hidden_layers)
725
--> 726 embedding_output = self.embeddings(
727 input_ids=input_ids, position_ids=position_ids, token_type_ids=token_type_ids, inputs_embeds=inputs_embeds
728 )
/home/ubuntu/anaconda3/envs/GoEmotions-pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
530 result = self._slow_forward(*input, **kwargs)
531 else:
--> 532 result = self.forward(*input, **kwargs)
533 for hook in self._forward_hooks.values():
534 hook_result = hook(self, input, result)
/home/ubuntu/anaconda3/envs/GoEmotions-pytorch/lib/python3.8/site-packages/transformers/modeling_bert.py in forward(self, input_ids, token_type_ids, position_ids, inputs_embeds)
172
173 if inputs_embeds is None:
--> 174 inputs_embeds = self.word_embeddings(input_ids)
175 position_embeddings = self.position_embeddings(position_ids)
176 token_type_embeddings = self.token_type_embeddings(token_type_ids)
/home/ubuntu/anaconda3/envs/GoEmotions-pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
530 result = self._slow_forward(*input, **kwargs)
531 else:
--> 532 result = self.forward(*input, **kwargs)
533 for hook in self._forward_hooks.values():
534 hook_result = hook(self, input, result)
/home/ubuntu/anaconda3/envs/GoEmotions-pytorch/lib/python3.8/site-packages/torch/nn/modules/sparse.py in forward(self, input)
110
111 def forward(self, input):
--> 112 return F.embedding(
113 input, self.weight, self.padding_idx, self.max_norm,
114 self.norm_type, self.scale_grad_by_freq, self.sparse)
/home/ubuntu/anaconda3/envs/GoEmotions-pytorch/lib/python3.8/site-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
1482 # remove once script supports set_grad_enabled
1483 no_grad_embedding_renorm(weight, input, max_norm, norm_type)
-> 1484 return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
1485
1486
RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'index' in call to _th_index_select
Hi,
Thank you for your great work on GoEmotions-pytorch!
I am trying to use your code to generate models using either bert-mini, bert-small or bert-tiny for faster predictions.
I changed the file original.json by setting model_name_or_path to prajjwal1/bert-mini for example and I run python3 run_goemotions.py --taxonomy original
It works and the new model is a bit faster than the one using bert-base. However, I was wondering if I need to also change the tokenizer_name_or_path to a different value. The original value is "monologg/bert-base-cased-goemotions-original". Any thoughts on how to get a tokenizer based on bert-mini?
Many thanks!
Chedia
Hi!
Would you be able to help me and provide me with the www.syndicai.co file to deploy the project there?
That would really help me :) Thanksa. lot!
your work is very interesting and valuable.
I would like to know how to get the final hidden representations of sentence to get the vector.
Thanks
Hi,
Thanks so much for this repo and please forgive me if this is trivial, I've been trying for a little while now to run the model on Google Colab. I'm running into two separate issues, which I think may be linked. The first, is that if I load the model in a GPU runtime the model defaults to the 'cpu'.
After running:
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') model.to(device)
There's an error when trying to run goemotions(texts):
"RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'index' in call to _th_index_select".
Second, when trying to run goemotions over more than a few thousand rows on Colab in a high-RAM runtime environment, I run into an out of memory error. I'm wondering if this is a problem with batching in the data loader? I'll be looking for solutions & hope to close this issue myself, but in the meantime any help is much appreciated, thanks!
I am looking into loading the model and the tokenizer after it has been trained on a custom dataset. After training, I am able to produce pytorch_model.bin
, config.json
, tokenizer_config.json
, special_tokens_map.json
, training_args.bin
, and vocab.txt
for every checkpoint saved.
Is there any script where I can know how to load the saved checkpoints along with the tokenizer just like the example that you have provided here for your pre-trained model
tokenizer = BertTokenizer.from_pretrained("monologg/bert-base-cased-goemotions-ekman")
model = BertForMultiLabelClassification.from_pretrained("monologg/bert-base-cased-goemotions-ekman")
goemotions = MultiLabelPipeline(
model=model,
tokenizer=tokenizer,
threshold=0.3
)
Thanks for the awesome repo!
Let's say we want to fine-tune the model (any of the taxonomies - ekman/original) on another dataset having a different number of classes for e.g. only positive and negative. Then what's the correct procedure to do it?
Currently, If I prepare the data in the same format (.tsv files with data and labels), and put the labels.txt
as having only two classes, the training seems to run. But is this way correct? Or any other changes need to be made inside the model training?
Please provide the aws link for downloading your finetuned model. Thanks.
Thanks for this great work.
I am using transformer v4
I know this is not transformer v2 as requested in the readme file but I cannot install v2.11.0 anymore because there are dependency errors in that version. And I tried v2.4.1 it raise other errors.
In transformer v4, it raises:
Traceback (most recent call last):
goemotions = MultiLabelPipeline(
TypeError: Can't instantiate abstract class MultiLabelPipeline with abstract methods _forward, _sanitize_parameters, postprocess, preprocess
Do you think you can update your code so it can work with the latest Hugginface transformer (v4)?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.