Chinese traineddata does not use ISO 639-3 code about tessdata_fast HOT 7 OPEN

The Tesseract documentation is not clear about the question whether ISO 639-2 or ISO 639-3 is used:

doc/combine_lang_model.1.asc:   Tesseract uses 3-character ISO 639-2 language codes. (See LANGUAGES)
doc/tesseract.1.asc:  Tesseract uses 3-character ISO 639-2 language codes
include/tesseract/baseapi.h:   * The language is (usually) an ISO 639-3 string or nullptr will default to
src/api/baseapi.cpp: * The language is (usually) an ISO 639-3 string or nullptr will default to eng.

I don't mind fixing this and using ISO 639-3 everywhere, but would like to get more feedback here from people who are affected by a renaming of the models for Chinese.

from tessdata_fast.

So the suggestion is that the OCR models starting with chi_ be renamed to zho_ instead.

Comments from Chinese users of Tesseract are welcome.

from tessdata_fast.

AFAIK chi is (also) valid code for Chinese:
https://www.w3.org/WAI/ER/IG/ert/iso639.htm
https://www.loc.gov/standards/iso639-2/php/code_list.php
https://iso639-3.sil.org/code/chi

Never rely on one source (especially if anybody can change it).

from tessdata_fast.

Here is the official site, I should have probably linked to that instead of wikipedia in the first place.

While yes, chi is also a valid code for chinese, it is the ISO 639-2/B code (as can also be seen on the official site which you also linked to).
All other languages use the ISO 639-3 codes however. For example Czech is ces.traineddata (ISO 639-3) and not cze.traineddata (ISO 639-2/B).

from tessdata_fast.

Thats really unfortunate, that tesseract it self isnt even consistent with the used standard.
Note the (usually) 😆

from tessdata_fast.

There exist more ISO 639-3 codes for Chinese variants (at least cmn, yue, nan).

from tessdata_fast.

Yes those are for variants of the language although Id argue that the model is applicable for the macro language chinese hence zho fits best. Ofcause it would also be possible to have multiple models for individual chinese languages.

from tessdata_fast.