rick-de-water / lingo Goto Github PK

Implement encoding and character set aware basic_string

See what works and what breaks, and make sure that utf8 strings can be implicitly constructed from all string literals with an utf8 encoding

Reading encoded data backwards is probably a bad idea.

Constructors

• iterator (Not easy to restrict iterators to either points or units, skip for now)
• std::initializer_list (Same for initializer list)

Member functions

• assign (Skip iterators and initializer list here as well)
• shrink_to_fit
• insert
• erase
• pop_back (would require reading encoding backwards, probably not be worth the effort) Skip push_back and pop_back for now, use append or += to add code points.
• append
• starts_with
• ends_width
• replace
• substr
• copy
• swap
• find
• rfind
• find_first_of
• find_first_not_of
• find_last_of
• find_last_not_of

Free functions

• operator+
• operator== with cstring literals

Streams could be a pain

• operator<<
• operator>>
• getline

Hashing

• std::hash Moved to a separate issue #48

string_storage should be responsible not just for allocating memory, but also moving code units around efficiently.

Design a better interface that allows string to simply tell the string_storage where the data needs to go, while string_storage puts everything in the right place.

It should also be efficient, switching to std::memcpy or std::move where possible, and also have a strong exception guarantee.

Implement utf_iterator for UTF8, UTF16 and UTF32

• Add normalize to the Unicode page.
• Add normalize as a free function that can take any string object with a page that has a normalize function.
• Add normalize as a member function to lingo::strings with a page that has a normalize function.

Member functions

• copy
• substr
• starts_with
• ends_with
• find
• rfind
• find_first_of
• find_first_not_of
• find_last_of
• find_last_not_of

Free functions

• operator+ (actually not standard, but I want it anyway!)
• operator<<

Hashing

• std::hash Moved to a separate issue #48

Use Python to generate a ton of validated data and throw it at the library.

Make fields such as size have a predictable value on error.

Specialize std::hash for lingo::string and lingo::string_view.

Implement encoding and character set aware basic_string_view