Retraining on floydhub.com is not available about attacut HOT 5 CLOSED

@cnlinxi sorry for your inconvenience. I thought using floydhub would be sustainable but it seems very costly in a long run. So, I've decided to cancel my subscription, hence losing the datasets there.

I'll get back to you regarding the corpus. Would you mind sharing a bit on what you plan to do with the code?

from attacut.

@heytitle Sorry for reply too late. I hope to use this model to segment Thai words, and hope to improve it. I hope to provide a good Thai text regularization method.

from attacut.

@cnlinxi sorry again for my response. You can find the data at https://codeforthailand.s3-ap-southeast-1.amazonaws.com/attacut-related/data.zip

Please unzip and make sure the root directory is at ./data. The content of the archive contains

Only the first two are relevant for training; sampling-0 means all the dateset, while sampling-10 means only 10 files are used. You can use sampling-10 for quick training.

Before running the training command below, make sure that you have the ./artifacts directory.

python ./scripts/train.py --model-name seq_sy_ch_conv_concat \
 --model-params "embc:8|embs:8|conv:8|l1:6|do:0.1" \
 --data-dir ./data/best-syllable-crf-and-character-seq-feature-sampling-0  \
 --output-dir ./artifacts/model-xx  \
 --epoch 2 \
 --batch-size 1024 \
 --lr 0.001 \
 --lr-schedule "step:5|gamma:0.5"

from attacut.

@heytitle thank you very much. I have trained this model on BEST 2010. Great work:)

from attacut.

https://codeforthailand.s3-ap-southeast-1.amazonaws.com/attacut-related/data.zip

Is word split by "~" in "best-syllable-tokenized" dataset?

from attacut.