rgcottrell / pytorch-human-performance-gec Goto Github PK

@rgcottrell @cqlijingwei @tianfeichen

Thanks for the code and nice repo.

I have some doubts regarding generation
python ../fairseq/generate.py
../test/jfleg
--path ../checkpoints/lang-8-fairseq-cnn/checkpoint_best.pt
--batch-size 128
--raw-text
--source-lang en
--target-lang gec

I ran the above script for getting the GLue scores without the lang-model as I could not find it and got this:

Translated 747 sentences (15033 tokens) in 7.9s (94.45 sentences/s, 1900.78 tokens/s)
| Generate test with beam=5: BLEU4 = 65.05, 84.0/69.9/59.6/51.1 (BP=1.000, ratio=1.004, syslen=14286, reflen=14226)

This result stays constant no matter if I change the beam length or not, BP is always 1.000. if we average them, it comes out to be 65.03.
Could you help in understanding the GLUE score calculation?

hi,
I run the code 'train-lang8-cnn.bat' in linux, and I have changed ig from 'bat' to 'sh'. The previous step is all ok. and I traind the model 35 epoches, but when I run this code is wrong.
Could you help me ? Thank you very much.
Namespace(beam=5, cpu=False, criterion='cross_entropy', data='../data-bin/lang-8-fairseq', dataset_impl='cached', diverse_beam_groups=-1, diverse_beam_strength=0.5, force_anneal=None, fp16=False, fp16_init_scale=128, fp16_scale_tolerance=0.0, fp16_scale_window=None, gen_subset='test', iter=500, lang_model_data=None, lang_model_path=None, lazy_load=False, left_pad_source='True', left_pad_target='False', lenpen=1, log_format=None, log_interval=1000, lr_scheduler='fixed', lr_shrink=0.1, match_source_len=False, max_len_a=0, max_len_b=200, max_sentences=128, max_source_positions=1024, max_target_positions=1024, max_tokens=None, memory_efficient_fp16=False, min_len=1, min_loss_scale=0.0001, model_overrides='{}', momentum=0.99, n=4, nbest=1, no_beamable_mm=False, no_early_stop=False, no_progress_bar=False, no_repeat_ngram_size=0, num_shards=1, num_workers=0, optimizer='nag', path='../checkpoints/lang-8-fairseq-cnn/checkpoint_best.pt', prefix_size=0, print_alignment=False, quiet=False, raw_text=False, remove_bpe=None, replace_unk=None, required_batch_size_multiple=8, results_path=None, sacrebleu=False, sampling=False, sampling_topk=-1, sampling_topp=-1.0, score_reference=False, seed=1, sent=False, shard_id=0, skip_invalid_size_inputs_valid_test=False, source_lang=None, target_lang=None, task='translation', tbmf_wrapper=False, temperature=1.0, tensorboard_logdir='', threshold_loss_scale=None, unkpen=0, unnormalized=False, upsample_primary=1, user_dir=None, warmup_updates=0, weight_decay=0.0)
| [en] dictionary: 137960 types
| [gec] dictionary: 121816 types
Traceback (most recent call last):
File "./generate.py", line 236, in
main(args)
File "./generate.py", line 37, in main
task.load_dataset(args.gen_subset)
File "/root/anaconda3/lib/python3.7/site-packages/fairseq/tasks/translation.py", line 188, in load_dataset
max_target_positions=self.args.max_target_positions,
File "/root/anaconda3/lib/python3.7/site-packages/fairseq/tasks/translation.py", line 51, in load_langpair_dataset
fix_lua_indexing=True, dictionary=src_dict))
File "/root/anaconda3/lib/python3.7/site-packages/fairseq/data/indexed_dataset.py", line 39, in make_dataset
return IndexedCachedDataset(path, fix_lua_indexing=fix_lua_indexing)
File "/root/anaconda3/lib/python3.7/site-packages/fairseq/data/indexed_dataset.py", line 165, in init
super().init(path, fix_lua_indexing=fix_lua_indexing)
File "/root/anaconda3/lib/python3.7/site-packages/fairseq/data/indexed_dataset.py", line 100, in init
self.read_index(path)
File "/root/anaconda3/lib/python3.7/site-packages/fairseq/data/indexed_dataset.py", line 106, in read_index
'Index file doesn't match expected format. '
AssertionError: Index file doesn't match expected format. Make sure that --dataset-impl is configured properly.

where do i get this pre-trained model?

@rgcottrell Hi, I test the model by generate-lang8-cnn.bat ,but I got a trouble:
Namespace(beam=5, cpu=False, data=['../data-bin/lang-8-fairseq'], diverse_beam_groups=1, diverse_beam_strength=0.5, fp16=False, fp16_init_scale=128, gen_subset='test', iter=500, lang_model_data=None, lang_model_path=None, left_pad_source='True', left_pad_target='False', lenpen=1, log_format=None, log_interval=1000, max_len_a=0, max_len_b=200, max_sentences=128, max_source_positions=1024, max_target_positions=1024, max_tokens=None, min_len=1, model_overrides='{}', n=4, nbest=1, no_beamable_mm=False, no_early_stop=False, no_progress_bar=False, num_shards=1, path='../checkpoints/lang-8-fairseq-cnn/checkpoint_best.pt', prefix_size=0, print_alignment=False, quiet=False, raw_text=False, remove_bpe=None, replace_unk=None, sampling=False, sampling_temperature=1, sampling_topk=-1, score_reference=False, seed=1, sent=False, shard_id=0, skip_invalid_size_inputs_valid_test=False, source_lang=None, target_lang=None, task='translation', unkpen=0, unnormalized=False, upsample_primary=1) | [en] dictionary: 137960 types | [gec] dictionary: 121816 types | ../data-bin/lang-8-fairseq test 5257 examples | ['../data-bin/lang-8-fairseq'] test 5257 examples | loading model(s) from ../checkpoints/lang-8-fairseq-cnn/checkpoint_best.pt Traceback (most recent call last): File "./generate.py", line 236, in <module> main(args) File "./generate.py", line 93, in main fluency_scorer = FluencyScorer(args.lang_model_path, args.lang_model_data) File "/home/gpower/zhangtianjiu/NLP/pytorch-human-performance-gec-master/fairseq-scripts/fluency_scorer.py", line 58, in __init__ self.task = tasks.setup_task(self.args) File "/home/gpower/zhangtianjiu/NLP/pytorch-human-performance-gec-master/fairseq/fairseq/tasks/__init__.py", line 19, in setup_task return TASK_REGISTRY[args.task].setup_task(args) File "/home/gpower/zhangtianjiu/NLP/pytorch-human-performance-gec-master/fairseq/fairseq/tasks/language_modeling.py", line 90, in setup_task dictionary = Dictionary.load(os.path.join(args.data, 'dict.txt')) File "/home/gpower/anaconda3/envs/tf/lib/python3.6/posixpath.py", line 78, in join a = os.fspath(a) TypeError: expected str, bytes or os.PathLike object, not NoneType
I looked at a lot of relevant solutions on the StackOverflow, but still no solution.
can you help?
My environment used for the development is Linux + Python 3.6.3 + CUDA 9.0 + pytorch 0.4.1.
And I train model is no problem.

Hi, I have a probelm:RuntimeError: CuDNN error: CUDNN_STATUS_SUCCESS.
And, you write this sentence:If RuntimeError: CuDNN error: CUDNN_STATUS_SUCCESS occurs during training, try install pytorch with CUDA 9.2 using conda instead of using default CUDA 9.0.
Can you explain this sentence? I usually install pytorch using pip

What seems to be the problem.

hi, which version 'fairseq' are you used, when I run genetate.py,
for example:
sh generate_lang8.sh,
Traceback (most recent call last):
File "./generate.py", line 238, in
main(args)
File "./generate.py", line 89, in main
sampling=args.sampling, sampling_topk=args.sampling_topk, sampling_temperature=args.sampling_temperature,
AttributeError: 'Namespace' object has no attribute 'sampling_temperature'.

Could you share the fairseq you used, case I only find the latest version in github, thank you very much.

Hello thank you for the great post, :)

I have applied the multi-around correction the F1 score and recall are increased but the precision is decreased -1.
I'm a little worried about this issue which doesn't mention in the paper. Could you explain this issue or any suggestions to fix it?

Kind regards,
Aiman Solyman

@rgcottrell @cqlijingwei @tianfeichen

Hello to all. I am having trouble finding out a way to include different types of models with Fluency Boost technique.

I trained a transformer model, which when I run normally runs fine, but when combined with Language model does not work as Fluency Boost needs a language model.

How can I make the Fconvlanguage model compatible with Transformer Models?

@rgcottrell @tianfeichen @cqlijingwei Hey again. Thanks for all the earlier replies. I could preprocess, train and test everything in Google Colab. But recently, I switched to training it on my Gaming Laptop and i got the error.

dist_c10d is not defined.

Can you explain me more about has_c10d etc?
Because in Google Colab, these were the parameters.
ddp_backend='c10d'
distributed_backend='nccl',
distributed_init_method=None,
distributed_port=-1,
distributed_rank=0, distributed_world_size=1
But in the distributed_utils.py, i cannot import torch.distributed as dist_c10d. It always do to the torch.distributed as dist.no_c10d. Can you guide me here?

and when i would use init_fn = dist_no_c10d.init_process_group. It would start import the data and all.

Any help would be appreciated. Thanks in advance.

@rgcottrell @tianfeichen @cqlijingwei

I am actually researching about the different types of models for grammatical correction.

So, i used arch='transformer_iwslt_de_en' from this link,
https://github.com/pytorch/fairseq/tree/master/examples/translation#iwslt14-german-to-english-transformer

Here, i changed the model from fconv to transformer. I got really disappointing results for the test set, jfleg.

Do you want to share some thoughts? What newer features from fairseq do you think will be the fittest for the extension of your project?
Any help would be appreciated. Thanks in advance.