- adding a dictionary to the corpus?
-
Neural Machine Translation for Low Resource Languages using Bilingual Lexicon Induced from Comparable Corpora - this requires a bootstrap dataset of sentence alignments in the magnitude of 100K pairs
-
Thesis - overview of different alignment methods
-
- bicorpus: format = each line is a tokenized sentence
- dictionary: an item consists of a target language phrase and a source language phrase, separated by the " @ " sequence
hunalign dictionary_file source_text target_text
e.g.
./hunalign/src/hunalign/hunalign szl-pl.dict source_token.pl target_token.szl -text > /tmp/align.txt ./hunalign/src/hunalign/hunalign -realign szl-pl.dict source_token.pl target_token.szl -text > michael-jackson.align.tsv
Batch job
./hunalign/src/hunalign/hunalign -realign -ppthresh=30 -headerthresh=100 -topothresh=30 szl-pl.dict -batch batch_job
- no unified orthography in use
- lack of bilingual corpus
- translations are not exactly 1-to-1