aniskywalker / sarcasmdetection Goto Github PK

View Code? Open in Web Editor NEW

128.0 3.0 38.0 11.91 MB

Sarcasm detection on tweets using neural network

Home Page: https://www.aclweb.org/anthology/W/W16/W16-0425.pdf

License: GNU Lesser General Public License v3.0

Python 100.00%

sarcasm neural-networks lstm cnn keras twitter sarcasm-detection

sarcasmdetection's People

Contributors

Stargazers

Watchers

sarcasmdetection's Issues

is the dataset the original one in the paper?

May I know is the dataset here the original one in the paper? - Fracking sarcasm using neural network.

FileNotFoundError: [Errno 2] No such file or directory: SarcasmDetection/resource/text_model/weights/model_wv.json'

Can you please upload this file as well?


[jalal@goku src]$ python sarcasm_detection_model_CNN_LSTM_DNN_word2vec.py 
Using TensorFlow backend.
2018-03-03 20:58:10.558623: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-03-03 20:58:10.558655: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-03-03 20:58:10.558664: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-03-03 20:58:10.558671: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-03-03 20:58:10.558678: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2018-03-03 20:58:10.860328: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties: 
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.6705
pciBusID 0000:05:00.0
Total memory: 10.92GiB
Free memory: 10.17GiB
2018-03-03 20:58:11.117002: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x55a328acfc60 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
2018-03-03 20:58:11.117900: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 1 with properties: 
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.6705
pciBusID 0000:06:00.0
Total memory: 10.92GiB
Free memory: 10.76GiB
2018-03-03 20:58:11.118705: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 1 
2018-03-03 20:58:11.118723: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y Y 
2018-03-03 20:58:11.118730: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 1:   Y Y 
2018-03-03 20:58:11.118744: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:05:00.0)
2018-03-03 20:58:11.118752: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:06:00.0)
Loading resource...
initializing...
test_maxlen 30
Traceback (most recent call last):
  File "sarcasm_detection_model_CNN_LSTM_DNN_word2vec.py", line 285, in <module>
    t.load_trained_model()
  File "sarcasm_detection_model_CNN_LSTM_DNN_word2vec.py", line 187, in load_trained_model
    self.__load_model(self._model_file + 'model_wv.json', self._model_file + weight_file)
  File "sarcasm_detection_model_CNN_LSTM_DNN_word2vec.py", line 192, in __load_model
    self.model = model_from_json(open(model_path).read())
FileNotFoundError: [Errno 2] No such file or directory: 'SarcasmDetection/resource/text_model/weights/model_wv.json'
[jalal@goku src]$ ls ../resource/text_model
text_model/    text_model_2D/ 
[jalal@goku src]$ ls ../resource/text_model/
TestResults.txt.analysis  vocab_list.txt  weights
[jalal@goku src]$ ls ../resource/text_model/weights/
model.json       weights.01__.hdf5  weights.03__.hdf5  weights.05__.hdf5  weights.07__.hdf5  weights.09__.hdf5
model.json.hdf5  weights.02__.hdf5  weights.04__.hdf5  weights.06__.hdf5  weights.08__.hdf5  weights.10__.hdf5

Issue with loading model: "You are trying to load a weight file containing 7 layers into a model with 5 layers."

Hi there,

Thank you to the authors for making your research, code, and results public. I'm trying to run the model from sarcasm_detection_model_CNN_LSTM_DNN.py but keep coming across this error:

model loaded from file...
Traceback (most recent call last):
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 314, in
t.load_trained_model(model_file='model.json', weight_file='model.json.hdf5')
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 208, in load_trained_model
self.__load_model(self._model_file_path + model_file, self._model_file_path + weight_file)
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 215, in __load_model
self.model.load_weights(model_weight_path)
File "C:\Users\Me\Anaconda3\envs\sarcasmpy\lib\site-packages\keras\models.py", line 768, in load_weights
topology.load_weights_from_hdf5_group(f, layers, reshape=reshape)
File "C:\Users\Me\Anaconda3\envs\sarcasmpy\lib\site-packages\keras\engine\topology.py", line 3365, in load_weights_from_hdf5_group
str(len(filtered_layers)) + ' layers.')
ValueError: You are trying to load a weight file containing 7 layers into a model with 5 layers.

The model.json says that it uses Keras v 2.1.6, which I'm using. I'm also working with Python 3.7.

Has anyone come across this? Thanks for your help.

How to run this on my own data to get sarcasm result?

Two questions:

I have a CSV with one column that contains tweets.

how can I run this using your pre-trained weights?
how can I train this using my own training data? Does having 800 ground truth tweets work in this case? Is there a number of minimum ground truth tweets you would suggest?

https://pastebin.com/wxwbmD16 this took about 1 hour for me. I was wondering how I could use a saved model and don't start training from scratch.

Sarcasm detection on tweets using neural networks pdf is missing plz upload as soon as possible.

Does it work well for text with misspelling??

Just open an issue and wanna discuss about it.
Now I have encountered this phenomenon. I have a bunch of data from daily communication, well, as you may be realized, it is completely different from what we have witnessed in the training set.

So I am wondering what should I do in order to enhance the performance. Anyone could give some ideas??

Error when checking : expected embedding_1_input to have shape (None, 30) but got array with shape (2000, 50)

I ran the following:
[jalal@goku src]$ python sarcasm_detection_model_attention.py
(commented the training part).
And got this error:


#_L_S
#_s_t
#_S_h
#_s_w
#_f_u
#_n_o
#_B_u
#_t_o
#_n_e
#_t_r
#_I_J
#_l_i

vocab loaded...
Token coverage: 0.4024455818214407
Word coverage: 0.006309334276785187
Error: Error when checking : expected embedding_1_input to have shape (None, 30) but got array with shape (2000, 50)

Do you know how should I fix it?

Data files for the dataset with context information are missing

Thank you so much for making your code and dataset available on Github for fellow researchers.

But I see that the data files for the dataset with context information are missing from the repository. Can you please share them with me or upload them to the GitHub repo?

train_file = basepath + '/resource/train/Train_context_moods_v1.txt’
validation_file = basepath + '/resource/dev/Dev_context_moods.txt’
test_file = basepath + '/resource/test/Test_context_AW.txt'

Resource path (basepath) incorrectly determined

I am running into an error that's caused by line 286 of sarcasm_detection_model_CNN_LSTM_DNN.py. As you can see the string slice, to get the directory above the working directory, with index os.getcwd().rfind('/') doesn't work with a path that contains backslashes instead of forward slashes (as is the case in Windows).

Traceback (most recent call last):
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 303, in
t.load_trained_model(weight_file='weights.05__.hdf5')
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 201, in load_trained_model
self.__load_model(self._model_file_path + model_file, self._model_file_path + weight_file)
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 206, in __load_model
self.model = model_from_json(open(model_path).read())
FileNotFoundError: [Errno 2] No such file or directory: 'c:\Users\Niels\Documents\Maaksels\SarcasmDetection\sr\resource/text_model/weights/model.json'

Fix

basepath = os.path.abspath(os.path.join(os.getcwd(), '..'))

Should work, and achieve the desired result, for both Windows and Unix style paths.

I'll be committing this fix to the pull request that I've already made.

aniskywalker / sarcasmdetection Goto Github PK

sarcasmdetection's People

Contributors

Stargazers

Watchers

Forkers

sarcasmdetection's Issues

is the dataset the original one in the paper?

FileNotFoundError: [Errno 2] No such file or directory: SarcasmDetection/resource/text_model/weights/model_wv.json'

Issue with loading model: "You are trying to load a weight file containing 7 layers into a model with 5 layers."

How to run this on my own data to get sarcasm result?

Sarcasm detection on tweets using neural networks pdf is missing plz upload as soon as possible.

Does it work well for text with misspelling??

Error when checking : expected embedding_1_input to have shape (None, 30) but got array with shape (2000, 50)

Data files for the dataset with context information are missing

Resource path (basepath) incorrectly determined

Fix

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent