format for vocabulary file about tongrams HOT 1 CLOSED

Hi there,
try to add a line at the beginning of the vocab file with the number of lines
and compress it with gzip, like:
gzip vocab.txt
which should produce a gzipped file vocab.txt.gz.

To see another example, using the test files contained in the test_data directory of the project, run the following:
./sort_arpa 5 ../test_data/arpa ../test_data/1-grams.sorted.gz arpa.sorted --tmp ./tmp.

Let me know if I can assist you further.

from tongrams.