Some issue with Unicode characters, maybe about trienet HOT 2 OPEN

Hi,

we have a similar issue.
The Add was wokring all the time until someone added a word with an "ss" in the database.
After some data research we found out, that normaly we add those words with the character "ß" (we are from germany, "ss" and "ß" as interchangable ;)
So we pulled the code and debugged it.

The issue appeared here:

            var newEdge = new Edge<T>(remainder, newNode);
            e.Label = e.Label.Substring(remainder.Length);
            newNode.AddEdge(e.Label[0], e); // !!! HERE !!!
            s.AddEdge(t, newEdge);

(UkkonenTrie.cs -> Line: 207)

word: "walross"
remainder at that point: "oss"
e.label at that point = "oße"
and "e.Label = e.Label.Substring(remainder.Length);" will result in an empty string instead of the "e", which lets the next line fail with an out of index exception:
"newNode.AddEdge(e.Label[0], e);"

I guess, that you internally transform the "ß" to ss? Or that the code is interpretating the "ss" as "ß"?
Anyhow the code wants to use the "oss" node for the "oß" value :(

Our current workaround is to tralce all "ß" with "ss" and thats it, but it has annoying implications.

Thank you

Kind Regards

from trienet.

This issue seems like being caused by globalization, I solved this issue by add the following runtime option:
{ "runtimeOptions": { "configProperties": { "System.Globalization.Invariant": true } } }

UPDATED======

For my case, the root cause of this issue is that at least in "en-US" and "中文（**）" Culture, "ANYSTR".StartsWith("ANYSTR\u200B") returns True
This issue happens on Linux (for my case Ubuntu 18.04) but does not exist on Windows 10。

from trienet.