weijia-xu / fairseq-editor Goto Github PK

❓ Questions and Help

Before asking:

1. search the issues.
2. search the docs.

What is your question?

can you share your checkpoint (WMT14 ende ...) that trained with 300k update times

Code

What have you tried?

I follow the instruction from readme, but can't re-implement your WMT14 en-de result 26.9 BLEU. I got 16.49 BLEU, and only differences are that the max-tokens I set 8000 and I trained the model on two 3090 GPUs with update-freq 4 to imitate 8 GPUs settings

What's your environment?

• fairseq Version (e.g., 1.0 or master):
• PyTorch Version (e.g., 1.8.0)
• OS (e.g., Linux):
• How you installed fairseq (pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version: 11.2
• GPU models and configuration:
• Any other relevant information:

❓ Questions and Help

Before asking:

1. search the issues.
2. search the docs.

What is your question?

Hi, thanks for sharing the coding! I want to implement the EDITOR using English-Chinese datasets. And I have some question:

1. What form of dataset is used in training?
2. Does it require constraint terms?
3. Is only the test input file with soft lexical constraints enough?
   If it's convenient, can you provide me with an example of input for English-Japanese?

Code

What have you tried?

What's your environment?

• fairseq Version (e.g., 1.0 or master):
• PyTorch Version (e.g., 1.0)
• OS (e.g., Linux):
• How you installed fairseq (pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

❓ Questions and Help

Before asking:

1. search the issues.
2. search the docs.

What is your question?

Is there any way to fine-tune the model with my data by first loading the model with some pretrained weights? Is there any way to load the model with auto-regressive transformer's pretrained weights? I want to try EDITOR on English-Hindi corpus and I don't have this much of GPUs available with me.

Code

What have you tried?

What's your environment?

• fairseq Version (e.g., 1.0 or master):
• PyTorch Version (e.g., 1.0)
• OS (e.g., Linux):
• How you installed fairseq (pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

❓ Questions and Help

What is your question?

I tried to train a model from English to Chinese, but there was a mistake in the training.

Code

Traceback (most recent call last): File "/share/qzz/anaconda3/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 59, in _wrap fn(i, *args) File "/share/qzz/en2zh/fairseq-editor/fairseq_cli/train.py", line 286, in distributed_main main(args, init_distributed=True) File "/share/qzz/en2zh/fairseq-editor/fairseq_cli/train.py", line 96, in main train(args, trainer, task, epoch_itr) File "/share/qzz/anaconda3/lib/python3.7/contextlib.py", line 74, in inner return func(*args, **kwds) File "/share/qzz/en2zh/fairseq-editor/fairseq_cli/train.py", line 176, in train log_output = trainer.train_step(samples) File "/share/qzz/anaconda3/lib/python3.7/contextlib.py", line 74, in inner return func(*args, **kwds) File "/share/qzz/en2zh/fairseq-editor/fairseq/trainer.py", line 319, in train_step ignore_grad=is_dummy_batch, File "/share/qzz/en2zh/fairseq-editor/fairseq/tasks/translation_lev.py", line 175, in train_step loss, sample_size, logging_output = criterion(model, sample) File "/share/qzz/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/share/qzz/en2zh/fairseq-editor/fairseq/criterions/nat_loss.py", line 218, in forward factor=outputs[obj].get("factor", 1.0) File "/share/qzz/en2zh/fairseq-editor/fairseq/criterions/nat_loss.py", line 53, in _compute_loss outputs, targets = outputs[masks], targets[masks] IndexError: The shape of the mask [56, 32] at index 1 does not match the shape of the indexed tensor [56, 33] at index 1

What have you tried?

I use the parameters：
fairseq-train \ ${data_dir}/data-bin \ --save-dir ${model_dir}/checkpoints \ --source-lang ${src} --target-lang ${tgt} \ --ddp-backend=no_c10d \ --task translation_lev \ --criterion nat_loss \ --arch editor_transformer \ --noise random_delete \ --optimizer adam --adam-betas '(0.9,0.98)' \ --lr 0.001 --lr-scheduler inverse_sqrt \ --warmup-updates 2000 \ --warmup-init-lr '1e-07' --label-smoothing 0.1 \ --share-all-embeddings --no-share-discriminator \ --dropout 0.3 --weight-decay 0.01 \ --decoder-learned-pos --encoder-learned-pos \ --apply-bert-init \ --log-format 'simple' \ --log-interval 5 \ --fixed-validation-seed 7 \ --max-tokens 2048 \ --save-interval-updates 10000 \ --max-update 100000
I tried to change arch to levenshtein_transformer, and it can train normally.

What's your environment?

• fairseq Version (0.9.0):
• PyTorch Version (1.10.0)
• OS (Linux):
• How you installed fairseq (pip, source):
• Build command you used (if compiling from source):
  git clone https://github.com/pytorch/fairseq
  cd fairseq
  pip install --editable ./
  python setup.py build_ext --inplace
• Python version: 3.7
• CUDA/cuDNN version: 11.0

I want to reproduce reported results of the EDITOR model in the main article (table2) for ro-en dataset.
How long does it take to train that ? How much memory is required? Can I reproduce results of EDITOR by google colab free gpu and also train new models?

What's your environment?
fairseq Version (e.g., 1.0 or master):
PyTorch Version (e.g., 1.0):
OS (e.g., Linux): 18.04
How you installed fairseq (pip, source):
Python version:
CUDA/cuDNN version: 11.4
GPU models and configuration: two NVIDIA 1080