Git Product home page Git Product logo

improving-text-generation-with-dynamic-masking-and-recovering's Introduction

Improving-Text-Generation-with-Dynamic-Masking-and-Recovering

The repository contains the code for our paper "Improving Text Generation with Dynamic Masking and Recovering" in IJCAI-2021.

Requirements

For NMT and AMR-to-Text generation:

  • Python version = 3.6.10
  • Pytorch version = 1.1.0

For Image Caption:

  • Python version = 2.7.16
  • Pytorch version = 1.1.0

Datasets

We upload the datasets we used for NMT task. However, due to the amount of data pre-processed data is very large, for AMR-to-Text generation and Image Caption tasks, we give repositories to the data processing that we use.

NMT

For NMT, first, go into the dynamic-denoise-nmt directory. Before run with wmt14en-de.pl and iwslt14.pl, you need to modify the working directory inside.

   #!bin/perl
   my $cur_dir = "/home/lzd/mt-exp/wmt14-en2de";
   my $workspace = "$cur_dir/bert-dynamic-ende";
   my $gpu_id = "1";
   my $python3 = "/home/lzd/anaconda3/bin/python3";
   my $t2t_dir = "/home/lzd/transmodel/dynamic-denoise-nmt";
   my $multi_bleu = "/home/lzd/script/multi-bleu.perl";

Pre-processing, Training and Validation

You need to set $step0-$step3 as 1, $step4-$step5 as 0, such as:

   #!bin/perl
   my $step0 = 1; #move data
   my $step1 = 1; #prepare_data
   my $step2 = 1; #training
   my $step3 = 1; #tuning
   my $step4 = 0; #test
   my $step5 = 0; #infer with ensemble model

Decoding

You need to set $step0-$step3 as 0, $step4 as 1 such as:

   #!bin/perl
   my $step0 = 0; #move data
   my $step1 = 0; #prepare_data
   my $step2 = 0; #training
   my $step3 = 0; #tuning
   my $step4 = 1; #test
   my $step5 = 0; #infer with ensemble model

Also, you need to specify the $best_iteration to run testing.

if ($step4) {
  my $best_iteration = "290000";
  foreach my $set (@test_sets) {
    my $iter = $best_iteration;
    makedir("$model_dir/$iter");
    $cmd = "CUDA_VISIBLE_DEVICES=$gpu_id $python3 $t2t_dir/translate.py -model $model_dir/model_step_$iter.pt -src $train_test_data_dir/$set.$src -output $model_dir/$iter/$set.tran -beam_size 5 -gpu 1 1>$model_dir/$iter/$set.log 2>&1";
    run("$cmd");

    $cmd = "sed -i \"s/@@ //g\" $model_dir/$iter/$set.tran";
    run("$cmd");
    eval_multibleu("$model_dir/$iter/$set.tran", "$train_test_data_dir/$set.tok.$tgt");
    #eval_fastscore("$model_dir/$iter/$set.tran", "$tmp_dir/$set.$tgt");
  }
} 

Then you can get the final results in the directory you specified.

Also note that, for results in our paper, we do not perform $step5.

AMR

For AMR-to-Text, go into the dynamic-denoise-amr directory, then follow the same steps as NMT.

Cite

If you like our paper, please cite

@inproceedings{ijcai2021-534,
  title     = {Improving Text Generation with Dynamic Masking and Recovering},
  author    = {Liu, Zhidong and Li, Junhui and Zhu, Muhua},
  booktitle = {Proceedings of the Thirtieth International Joint Conference on
               Artificial Intelligence, {IJCAI-21}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  pages     = {3878--3884},
  year      = {2021},
  month     = {8},
  note      = {Main Track}
  doi       = {10.24963/ijcai.2021/534},
  url       = {https://doi.org/10.24963/ijcai.2021/534},
}

improving-text-generation-with-dynamic-masking-and-recovering's People

Contributors

answer3664 avatar

Stargazers

 avatar  avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.