vanzytay / pytorch_sentiment_rnn Goto Github PK

python train.py --batch-size 20 --rnn_type GRU --cuda --gpu 1 --lr 0.0001 --mdl ATT-RNN --clip_norm 1 --opt Adam --epochs 50/scratch/sjn-p2/anaconda/anaconda2/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.
There are 2 CUDA devices
Setting torch GPU to 1
Using device:1 
Stored Environment:['term_len', 'word_index', 'glove', 'max_len', 'train', 'dev', 'test', 'index_word']
Loaded environment
Creating Model...
Setting Pretrained Embeddings
Initialized GRU model
Starting training
Namespace(aggregation='mean', attention_width=5, batch_size=20, clip_norm=1, cuda=True, dataset='Restaurants', dev=1, dropout_prob=0.5, embedding_size=300, epochs=50, eval=1, gpu=1, hidden_layer_size=300, l2_reg=0.0, learn_rate=0.0001, log=1, maxlen=0, mode='term', model_type='ATT-RNN', opt='Adam', pretrained=1, rnn_direction='uni', rnn_layers=1, rnn_size=300, rnn_type='GRU', seed=1111, term_model='mean', toy=False, trainable=1)
========================================================================
/scratch2/debate_tweets/sentiment/pytorch_sentiment_rnn/models/attention.py:50: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  a = self.softmax(a)
Traceback (most recent call last):
  File "train.py", line 345, in <module>
    exp.train()
  File "train.py", line 328, in train
    loss = self.train_batch(i)
  File "train.py", line 300, in train_batch
    output, hidden = self.mdl(sentence, hidden)
  File "/scratch/sjn-p2/anaconda/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/scratch2/debate_tweets/sentiment/pytorch_sentiment_rnn/models/rnn.py", line 44, in forward
    output = self.AttentionLayer(output, attention_width=self.args.attention_width)
  File "/scratch/sjn-p2/anaconda/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/scratch2/debate_tweets/sentiment/pytorch_sentiment_rnn/models/attention.py", line 67, in forward
    results = torch.cat((results,output),0)
RuntimeError: invalid argument 0: Tensors must have same number of dimensions: got 2 and 3 at /opt/conda/conda-bld/pytorch_1518238409320/work/torch/lib/THC/generic/THCTensorMath.cu:102

[jalal@goku pytorch_sentiment_rnn]$ /scratch/sjn-p2/anaconda/anaconda2/bin/python train.py --batch-size 20 --rnn_type GRU --cuda --gpu 1 --lr 0.0001 --mdl RNN --clip_norm 1 --opt Adam
Using TensorFlow backend.
RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
ERROR; return code from pthread_create() is 11
	Error detail: Resource temporarily unavailable

[jalal@goku pytorch_sentiment_rnn]$ /scratch/sjn-p2/anaconda/anaconda2/bin/python train.py --batch-size 20 --rnn_type GRU --cuda --gpu 1 --lr 0.0001 --mdl RNN --clip_norm 1 --opt Adam
Using TensorFlow backend.
RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
ERROR; return code from pthread_create() is 11
	Error detail: Resource temporarily unavailable
[jalal@goku pytorch_sentiment_rnn]$ /scratch/sjn-p2/anaconda/anaconda2/bin/python train.py --batch-size 20 --rnn_type GRU --cuda --gpu 1 --lr 0.0001 --mdl RNN --clip_norm 1 --opt Adam
Using TensorFlow backend.
RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
There are 2 CUDA devices
Setting torch GPU to 1
Using device:1 
Stored Environment:['term_len', 'word_index', 'glove', 'max_len', 'train', 'dev', 'test', 'index_word']
Loaded environment
Creating Model...
Setting Pretrained Embeddings
Initialized GRU model
Starting training
Namespace(aggregation='mean', attention_width=5, batch_size=20, clip_norm=1, cuda=True, dataset='Restaurants', dev=1, dropout_prob=0.5, embedding_size=300, epochs=50, eval=1, gpu=1, hidden_layer_size=300, l2_reg=0.0, learn_rate=0.0001, log=1, maxlen=0, mode='term', model_type='RNN', opt='Adam', pretrained=1, rnn_direction='uni', rnn_layers=1, rnn_size=300, rnn_type='GRU', seed=1111, term_model='mean', toy=False, trainable=1)
========================================================================
/scratch2/debate_tweets/sentiment/pytorch_sentiment_rnn/models/rnn.py:51: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  decoded = self.softmax(decoded)
Traceback (most recent call last):
  File "train.py", line 343, in <module>
    exp.train()
  File "train.py", line 326, in train
    loss = self.train_batch(i)
  File "train.py", line 303, in train_batch
    coeff = clip_gradient(self.mdl, self.args.clip_norm)
  File "train.py", line 35, in clip_gradient
    modulenorm = p.grad.data.norm()
AttributeError: 'NoneType' object has no attribute 'data'
[jalal@goku pytorch_sentiment_rnn]$ vi train.py 

Output:
---
[Epoch 1] Train Loss=0.9544535914837327 T=207.711814s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 2] Train Loss=0.9511685806084733 T=223.582632s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 3] Train Loss=0.9511685740223247 T=225.45853200000005s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 4] Train Loss=0.9486824010617166 T=219.52165200000002s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 5] Train Loss=0.9511685766567841 T=222.91000699999995s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 6] Train Loss=0.9486823934876458 T=221.76340299999993s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 7] Train Loss=0.9486823994151795 T=219.41352699999993s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 8] Train Loss=0.9511685839015476 T=222.40653099999986s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 9] Train Loss=0.9486824050134058 T=222.14093300000013s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 10] Train Loss=0.9486824076478653 T=221.4141770000001s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 11] Train Loss=0.9511685786326287 T=222.83806300000015s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 12] Train Loss=0.9536547789257535 T=222.21776s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 13] Train Loss=0.9486823948048755 T=226.17170699999997s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 14] Train Loss=0.9511685914756185 T=223.5613070000004s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 15] Train Loss=0.9511685918049259 T=224.78602600000022s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 16] Train Loss=0.9511686039893008 T=225.99811899999986s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 17] Train Loss=0.9536547822188277 T=229.34864700000026s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 18] Train Loss=0.9511685960859225 T=224.25112900000022s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 19] Train Loss=0.9511685983910745 T=226.01416599999993s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 20] Train Loss=0.9511686039893008 T=227.8825510000006s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 21] Train Loss=0.9511686023427637 T=227.24668999999994s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 22] Train Loss=0.9536547891342837 T=231.79577599999993s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 23] Train Loss=0.9511686059651454 T=227.48067600000013s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 24] Train Loss=0.9511686020134562 T=227.48563000000013s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 25] Train Loss=0.9486824152219361 T=225.5863560000007s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 26] Train Loss=0.9486824201615476 T=221.80185700000038s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 27] Train Loss=0.9511686072823751 T=222.58302899999944s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 28] Train Loss=0.9511686168322906 T=220.37081399999988s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 29] Train Loss=0.9511686115633717 T=217.61475700000028s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 30] Train Loss=0.9511686115633717 T=217.61519999999928s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 31] Train Loss=0.9511686155150608 T=221.39081799999985s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 32] Train Loss=0.9486824198322401 T=217.4700130000001s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 33] Train Loss=0.9486824247718516 T=216.53440500000033s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 34] Train Loss=0.9536548006600438 T=215.7164499999999s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 35] Train Loss=0.9511686247356689 T=219.20409699999982s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 36] Train Loss=0.9536547980255844 T=216.21291700000074s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 37] Train Loss=0.9536547996721215 T=217.60048799999822s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 38] Train Loss=0.9486824237839293 T=217.80505099999937s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 39] Train Loss=0.9511686036599933 T=221.05352800000037s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 40] Train Loss=0.9486824254304664 T=221.01860599999964s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 41] Train Loss=0.9536547960497398 T=220.08077799999955s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 42] Train Loss=0.9511686122219866 T=223.32578099999955s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 43] Train Loss=0.9486824267476962 T=222.5393839999997s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 44] Train Loss=0.9486824323459225 T=222.55362799999966s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 45] Train Loss=0.9511686023427637 T=222.70722000000023s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 46] Train Loss=0.9511686036599933 T=223.65236600000026s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 47] Train Loss=0.9511686059651454 T=222.57912299999953s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 48] Train Loss=0.9536547894635912 T=223.7721619999993s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 49] Train Loss=0.9486824139047064 T=223.58448600000156s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---
[Epoch 50] Train Loss=0.9486824125874767 T=220.14713099999972s
Test loss=0.90144944190979
Output Distribution={2: 1120}
Accuracy=0.65
---

    glove_path = './embeddings/glove_{}_{}.pkl'.format(dataset,mode)
    if(os.path.isfile(glove_path)):
            print("Reusing glove dictionary to save time")
            #with open(glove_path,'r') as f:
            with open(glove_path, 'rb') as f: #python 3 for pickle byte size object is needed not str
                    glove = pickle.load(f)
            save = False

    Reusing glove dictionary to save time
    Traceback (most recent call last):
      File "prepare.py", line 315, in <module>
        glove = pickle.load(f)
    TypeError: a bytes-like object is required, not 'str'

with open(glove_path, 'rb') as f: #python 3 for pickle byte size object is needed not str

Reusing glove dictionary to save time
Traceback (most recent call last):
  File "prepare.py", line 315, in <module>
    glove = pickle.load(f)
EOFError: Ran out of input