when I input some sql that contains Chinese vocabulary ,it Separate with space about sql-formatter HOT 3 CLOSED

I still don't know how to resolve this question,even if I've tried it that update tokenizer.js instead of the existing regex function

Tokenizer.prototype.createWordRegex = function createWordRegex() {
    var specialChars = arguments.length > 0 && arguments[0] !== undefined ? arguments[0] : [];

    return new RegExp(`^([A-Za-z\u00C0-\u00D6\u00D8-\u00f6\u00f8-\u00ff` + specialChars.join("") + "]+)");
};

but it did not work
please tell me how to resolve this bug

from sql-formatter.

Duplicate of #33

from sql-formatter.

@BaooLi 兄弟他给的解决方案 匹配一下中文字符就可以了 \u4e00-\u9fa5

from sql-formatter.