Support for UTF-16/32 encoded buffer in ASN1_get_object about openssl HOT 2 CLOSED

It sounds like there are a few points of confusion here:

First, a UniversalString carries UTF-32, not UTF-16.

Second, using a char16_t or char32_t output would actually be forbidden (undefined behavior) in C and C++. There is no guarantee that the UniversalString payload is correctly aligned for a char32_t, to say nothing of strict aliasing requirements. Even setting those requirements aside, UniversalString stores in big-endian, and most machines are little-endian these days.

Rather, the correct way to decode a UniversalString payload is to pass the byte output into a UTF-32BE decoder. Such a decoder is a mostly a trivially loading 4-byte big-endian words, though there are some checks it should do for whether each codepoint is valid. None of that is done by ASN1_get_object because ASN1_get_object's job is to just parse an individual TLV, leaving the caller to interpret the contents.

from openssl.

A good resource about Unicode, UTF-8 and co :

https://tonsky.me/blog/unicode/

Derived from an old article, I read 20 years ago

from openssl.