facebookresearch / latent-treelstm Goto Github PK

Cooperative Learning of Disjoint Syntax and Semantics

License: MIT License

Python 100.00%

latent-treelstm's Introduction

Cooperative Learning of Disjoint Syntax and Semantics

This code was used to obtain the results described in the paper:

Cooperative Learning of Disjoint Syntax and Semantics
Serhii Havrylov, Germán Kruszewski, Armand Joulin
Presented at NAACL2019

ListOps

Download ListOps dataset. URLs of the original dataset and an extrapolation test set can be found in data/listops/external/urls.txt file.
Run python listops/data_preprocessing/split.py to split the dataset into the train, the valid and the test sets. Make sure that you have downloaded the dataset and the files are present in the data/listops/external folder.
Build vocabulary using python listops/data_preprocessing/build_vocab.py.
Run python listops/ppo/train_ppo_model.py or python listops/reinforce/train_reinforce_model.py to train the model with PPO or REINFORCE estimators.

SST

Run python sst/ppo/train_ppo_model.py to train the model using SST-2 or SST-5 datasets.

NLI

Download SNLI and MultiNLI datasets and extract corresponding archives to data/nli folder. URLs can be found in data/nli/external/urls.txt file.
Run python nli/data_preprocessing/preprocess.py to preprocess dataset and generate vocabulary files.
Run python nli/ppo/train_ppo_model.py to train the model using SNLI or MultiNLI datasets.

The code is tested with Python 3.6.3 and PyTorch 1.0.1.

License

Latent-TreeLSTM is MIT licensed. See the LICENSE file for details.

latent-treelstm's People

Contributors

Stargazers

Watchers

Forkers

serhii-havrylov github30 alexanderkoller jkinx debasishmaji tsvm rezkaaufar celiayao329 cornell-cs5740-19sp chunlinx starshipai zeta1999 iq-scm

latent-treelstm's Issues

A question about the implementation of TreeLSTM

https://github.com/facebookresearch/latent-treelstm/blob/master/modules/BinaryTreeLstmCell.py#L33

I guess using c.tanh_() instead of c.tanh() will change the value of c in place. So what you actually return is tanh(c) instead of c.
Is this a bug?

training

I run listops/ppo/train_ppo_model.py but I cannot tell if the model is training. A directory is created for the model but it's empty, as is the tensorboard file.

A Question About Padding / ListOpsBucketSampler

Hey,

Just wanted to check in on the following:

The ListOps dataset contains samples of different length (different number of tokens). But I didn't find any code in this repo that does some sort of padding, neither did I see a padding symbol in the vocabulary.

Do I understand correctly that the ListOpsBucketSampler only returns batches of samples that have the same number of tokens? Hence, no padding is necessary. This also implies that I may as well disregard the mask object the data loaders return for each batch as it is always only a matrix of ones?

Thanks,
Max

torch_tree missing

Hi guys,

thanks for making the code available!

I'm trying to run the listops experiment, and am running into trouble in Step 4 of the instructions in the README, because Python can't find the module torch_tree. I can't find it on the Internet either. Is this a file that's missing in the repository?

The exact error message is below. Note that I have to run the script with -m because the syntax shown in the README won't work for me ("no module names 'utils'" because Python is looking for modules under the path of the script, not the current working directory).

Best,
Alexander.

koller@hilbert ~/D/w/latent-treelstm> python -m listops.ppo.train_ppo_model           (latent-treelstm) 
Traceback (most recent call last):
  File "/anaconda3/envs/latent-treelstm/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/anaconda3/envs/latent-treelstm/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/koller/Documents/workspace/latent-treelstm/listops/ppo/train_ppo_model.py", line 17, in <module>
    from listops.models import PpoModel
  File "/Users/koller/Documents/workspace/latent-treelstm/listops/models/__init__.py", line 6, in <module>
    from listops.models.IdealTreeModel import IdealTreeModel
  File "/Users/koller/Documents/workspace/latent-treelstm/listops/models/IdealTreeModel.py", line 7, in <module>
    from modules import BinaryTreeLstmRnn
  File "/Users/koller/Documents/workspace/latent-treelstm/modules/__init__.py", line 9, in <module>
    from modules.BinaryTreeLstmRnn import BinaryTreeLstmRnn
  File "/Users/koller/Documents/workspace/latent-treelstm/modules/BinaryTreeLstmRnn.py", line 6, in <module>
    from torch_tree import BinaryTreeReducer
ModuleNotFoundError: No module named 'torch_tree'

Compatibility issue with current tensorboardX version

With tensorboardX version 1.7 (most current when I installed the dependencies), listops/ppo/train_ppo_model.py crashes with the error message shown below.

Downgrading to tensorboardX 1.6 seems to fix the problem. Maybe replace tensorboardX>=1.6 in requirements.txt by tensorboardX==1.6? Alternatively, you might want to fix the code so it is compatible with newer tensorboardX versions.

koller@hilbert ~/D/w/latent-treelstm> env PYTHONPATH=. python listops/ppo/train_ppo_model.py                           (latent-treelstm) 
data/listops/ppo/logs/exp0/l5379958138750912156.log
Traceback (most recent call last):
  File "listops/ppo/train_ppo_model.py", line 393, in <module>
    main(args)
  File "listops/ppo/train_ppo_model.py", line 289, in main
    logger, summary_writer = make_path_preparations(args)
  File "listops/ppo/train_ppo_model.py", line 52, in make_path_preparations
    summary_writer["train"] = SummaryWriter(log_dir=os.path.join(tensorboard_path, 'log' + args_hash, 'train'))
  File "/anaconda3/envs/latent-treelstm/lib/python3.7/site-packages/tensorboardX/writer.py", line 254, in __init__
    self._get_file_writer()
  File "/anaconda3/envs/latent-treelstm/lib/python3.7/site-packages/tensorboardX/writer.py", line 310, in _get_file_writer
    self.file_writer = FileWriter(logdir=self.logdir, **self.kwargs)
TypeError: __init__() got an unexpected keyword argument 'log_dir'

facebookresearch / latent-treelstm Goto Github PK

latent-treelstm's Introduction

Cooperative Learning of Disjoint Syntax and Semantics

ListOps

SST

NLI

License

latent-treelstm's People

Contributors

Stargazers

Watchers

Forkers

latent-treelstm's Issues

A question about the implementation of TreeLSTM

training

A Question About Padding / ListOpsBucketSampler

torch_tree missing

Compatibility issue with current tensorboardX version

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent