guillitte / pytorch-sentiment-neuron Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Hi,
I converted the model to run on cpu
using,
checkpoint = torch.load(opt.load_model)
embed = checkpoint['embed']
rnn = checkpoint['rnn']
embed = embed.cpu()
rnn = rnn.cpu()
because I'm interested in running unsupervised trained networks on low resource hardware. When I sample the model with command,
python visualize.py -seq_length 1000 -load_model mlstm_ns.pt -temperature 0.4 -neuron 2388 -init "I couldn't figure out"
The sentiment is not consistent,
Have you got any ideas how I could try to fix it please? I'd like to get the model to run on the cpu
somehow?
Thanks a lot 👍
I am trying to use the repository for training a new model but I got the following error
/Users/virtualenv-dir/py3/bin/python /Users/Documents/Codes1/pytorch-sentiment-neuron-master/convert_to_cpu.py /Users/virtualenv-dir/py3/lib/python3.6/site-packages/torch/serialization.py:325: SourceChangeWarning: source code of class 'torch.nn.modules.linear.Linear' has changed. you can retrieve the original source code by accessing the object's source attribute or set
torch.nn.Module.dump_patches = Trueand use the patch tool to revert the changes. warnings.warn(msg, SourceChangeWarning) Traceback (most recent call last): File "/Users/Documents/Codes1/pytorch-sentiment-neuron-master/convert_to_cpu.py", line 18, in <module> checkpoint = torch.load(opt.load_model) File "/Users/virtualenv-dir/py3/lib/python3.6/site-packages/torch/serialization.py", line 267, in load return _load(f, map_location, pickle_module) File "/Users/virtualenv-dir/py3/lib/python3.6/site-packages/torch/serialization.py", line 426, in _load result = unpickler.load() File "/Users/virtualenv-dir/py3/lib/python3.6/site-packages/torch/serialization.py", line 395, in persistent_load data_type(size), location) File "/Users/virtualenv-dir/py3/lib/python3.6/site-packages/torch/serialization.py", line 87, in default_restore_location result = fn(storage, location) File "/Users/virtualenv-dir/py3/lib/python3.6/site-packages/torch/serialization.py", line 69, in _cuda_deserialize return obj.cuda(device) File "/Users/virtualenv-dir/py3/lib/python3.6/site-packages/torch/_utils.py", line 68, in _cuda with torch.cuda.device(device): File "/Users/virtualenv-dir/py3/lib/python3.6/site-packages/torch/cuda/__init__.py", line 218, in __enter__ self.prev_idx = torch._C._cuda_getDevice() AttributeError: module 'torch._C' has no attribute '_cuda_getDevice'
It seems that the code only run with Cuda, could you please give me a hint to continue?
Thanks,
Mohammad
Thanks for your great work. Can you provide an example for feature extraction that corresponds to:
from encoder import Model
model = Model()
text = ['demo!']
text_features = model.transform(text)
in the original code. Or code that produces the .npy model files from the generated .pt file?
Thank you!
The code in models.py constructs the graph in a very sleek way. Is it possible to see how you transformed the weights into mlstm_ns.pt too?
Hi, could you change your readme or include the params as the appear in the openai paper.
Specifically could you set seq_length to 256 and batch_size to 128, or at least mention that those are the params openai used in the paper.
Would've saved me some head scratching as to why I couldn't reproduce their results.
TY
Thanks for this work. In the original project I have worked on a Dockerfile to support training on CPU/GPU with the generation as well.
Would be interesting to have in your Pytorch version a different output like json
format, instead of visualizing the neuron output only.
Hello, I have created this Docker image for the original sentiment neuron that helps to test prediction of the sentiment feature as well as the generative model. I would like to add training, so I was starting from your Pytorch version.
How is the train and validation dataset format? Did you use the Amazon Reviews Full or Polarity Dataset for training i.e. the kaggle version here?
Thank you very much.
Hi Stéphane @guillitte ,
thanks for sharing this, its really very interesting 👍
I don't know where to get the file, mlstm_ns.pt
?
On the front of the repo it says,
Click on release to get model file mlstm_ns.pt
Where is release?
Thanks for your help,
Ajay :)
Hi,
I am trying to clone the function of generating heatmap , Can you please point out why my lines isn't breaking and why my font is so small.
https://github.com/yashkumaratri/testrepo/blob/master/heatmap.ipynb
Through the study on OpenAI's model, here is some useful information for developers who wrote their own version of mlstm and try to import OpenAI's model paramters. In mlstm function in encoder.py, defines the tensors' name, this is the baseline.
Computation Graphic and tensor
Under the name scope model, there are three sub name scope:
Table for the correlation between tensor and .npy files
For detailed information about each tensor and which .npy it is correlated, please check the table
Line of code follows the openAI's orignal code repo.
Name | Correlated-tensor | Array Shape | npy file index | line of code |
---|---|---|---|---|
params[0] | embedding/w | (256,64) | 0 | embd, line 23 |
params[1] | rnn/wx | (64, 16384) | 1 | mlstm, line 47 |
params[2] | rnn/wh | (4096, 16384) | hstack 2-5 | mlstm, line 48 |
params[3] | rnn/wmx | (64, 4096) | 6 | mlstm, line 49 |
params[4] | rnn/wmh | (4096, 4096) | 7 | mlstm, line 50 |
params[5] | rnn/b | (16384,) | 8 | mlstm, line 51 |
params[6] | rnn/gx | (16384,) | 9 | mlstm, line 53 |
params[7] | rnn/gh | (16384,) | 10 | mlstm, line 54 |
params[8] | rnn/gmx | (4096,) | 11 | mlstm, line 55 |
params[9] | rnn/gmh | (4096,) | 12 | mlstm, line 56 |
params[10] | out/w | (4096, 256) | 13 | fc, line 31 |
params[11] | out/b | (256,) | 14 | fc, line 38 |
Hopyfully this would help.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.