princeton-nlp / calm-textgame Goto Github PK

View Code? Open in Web Editor NEW

63.0 5.0 7.0 18.75 MB

[EMNLP 2020] Keep CALM and Explore: Language Models for Action Generation in Text-based Games

Python 100.00%

text-based-game nlp rl gpt n-gram calm

calm-textgame's People

Contributors

Stargazers

Watchers

Forkers

calm-textgame's Issues

Having issues getting training to converge - hyper parameter issue?

I'm attempting to run the training script for the GPT-2 CALM on the ClubFloyd dataset, following the instructions from your EMNLP 2020 paper. I've set up my environment as recommended but am facing challenges with the training process.

Environment:

Python version: 3.6.15
Operating System: Ubuntu 20.04
GPU: Nvidia Titan RTX
Dependencies: torch==1.4, transformers==2.5.1, jericho, fasttext, wandb, importlib_metadata

Issue:

The training doesn't perform as expected (training overfits to training data while validation performance hardly improves or worsens), even after adjusting hyperparameters like batch size and GPU count.

Attempts:

Params	Iteration	Train Acc	Val Acc	Train Loss	Val Loss
num GPU = 1 batch size = 1	1	0.14	0.15	2.38	2.35
	2	0.18	0.14	2.01	2.35
	3	0.22	0.15	1.80	2.43
	4	0.26	0.14	1.63	2.56
	5	0.30	0.14	1.50	2.71
num GPU = 3 batch size = 1	1	0.13	0.14	0.79	2.30
	2	0.17	0.14	0.67	2.26
	3	0.20	0.15	0.61	2.28
	4	0.22	0.15	0.57	2.33
	5	0.25	0.14	0.53	2.38
num GPU = 1 batch size = 15	1	0.10	0.13	0.18	2.32
	2	0.13	0.13	0.15	2.28
	3	0.15	0.14	0.14	2.27
	4	0.17	0.14	0.13	2.27
	5	0.18	0.14	0.13	2.31
num GPU = 3 batch size = 15	1	0.10	0.12	0.06	2.35
	2	0.12	0.13	0.05	2.30
	3	0.14	0.13	0.05	2.29
	4	0.15	0.14	0.05	2.28
	5	0.16	0.13	0.05	2.27
num GPU = 8 batch size = 12	1	0.09	0.11	0.03	2.41
	2	0.12	0.12	0.03	2.34
	3	0.13	0.13	0.02	2.31
	4	0.14	0.13	0.02	2.29
	5	0.14	0.14	0.02	2.29

Request:

Do you have any ideas on why these training runs might not be converging, whether it be hardware difference, hyperparameter difference, or something else?

Thank you for your time.

Train DRNN without CALM

How can I train the DRNN without using CALM?? And only using the default handicap version of Jericho. Just wanted to regenerate baseline results.
Thanks

Any try on other RL agent ?

Hi, thanks for the great work of text game. I have one question about the RL agent. In this paper, your agent is Deep Reinforcement Relevance Network (DRRN) from ACL2016 paper. I am wondering did you ever conduct some preliminary experiments on more powerful encoding function like BERT for better contextualized word embedding ? Do you have some intuition for making Transformer as Q-network in DRL ? Much Thanks !

Inference example

Thanks for this work! It would be worth to provide an inference example using the provided gpt-2 model weights for a given set of observations.

Thank you in advance.

Handicap questions

I've been truly inspired by your work.

I was curious about one aspect. In your paper, you mentioned using the handicap provided by Jericho. Upon reviewing your code, it appears that commands which don't affect 'look' and 'inventory' are being handicapped.

Could you help me understand? Is it that Jericho considers commands that change the environment as valid, instead of specifically checking for valid commands in the environment?

I'd greatly value your insight on this. Thank you!

Thanks!