Error when using ReplaceString with unicode character about kobopatch HOT 5 CLOSED

This is partly by design (albeit with a cryptic error message). To replace bytes values, you need to use ReplaceBytes, and FindBaseAddressBytes. Try changing that (see the other patches for examples), and if it doesn't work, show me the whole patch.

from kobopatch.

Are you sure that there is a "FindBaseAddressBytes"?

I don't get it:
len("\x20") = 1 = len("\xBB")
len([]byte("\x20")) = 1 = len([]byte("\xBB"))

The default entry in libnickel.so.1.0.0.yaml (v10) looks like that:

Allow searches on Extra dictionaries:
  - Enabled: no
    ## To allow searches on Extra dictionaries change space character at end of
    ## "Extra: " to another char (ex: "Extra:_")
    ## The space char causes a non-desired "English - English" when searching on
    ## Extra dictionary from main menu.
  - FindBaseAddressString: "\0Extra:\x20"
  - ReplaceString: {Offset: 7, Find: "\x20", Replace: "_"}

If I replace the last two lines with - ReplaceBytes: {Offset: 0x00CC184F, FindH: 00 45 78 74 72 61 3A 20, ReplaceH: 00 45 78 74 72 61 3A BB} I get a libnickel.so.1.0.0 where the space \x20 is correctly replaced by "»" (\xBB). Not very convenient because I would have to use a hex editor to search for the correct offset with every firmware update.

If I use instead:

  - FindBaseAddressString: "\0Extra:\x20\0"
  - ReplaceString: {Offset: 7, Find: "\x20\0", Replace: "\xBB"}

For whatever reason, an additional character is inserted - the \x20\0 is replaced by "»" (\xC2\xBB). Why?

That means, in a hex editor, the interesting part looks like that:

00 45 78 74 72 61 3A 20 00 (unpatched libnickel)
00 45 78 74 72 61 3A BB 00 (correctly patched libnickel by replacing bytes)
00 45 78 74 72 61 3A C2 BB (incorrectly patched libnickel by replacing string)

from kobopatch.

Oops, I meant FindBaseAddressHex, not FindBaseAddressBytes.

What I would suggest is:

Allow searches on Extra dictionaries:
  - Enabled: no
    ## To allow searches on Extra dictionaries change space character at end of
    ## "Extra: " to another char (ex: "Extra:_")
    ## The space char causes a non-desired "English - English" when searching on
    ## Extra dictionary from main menu.
  - FindBaseAddressHex: 00 45 78 74 72 61 3A 20 # Hex of \0Extra:\x20\0
  - ReplaceBytes: {Offset: 7, FindH: 20, ReplaceH: BB}

This way, you don't have to update offsets, but it still works.

The issue is due to the way strings are encoded as binary in Go. Golang uses UTF-8, which is a two-byte encoding. \xBB is the ISO-8859-1 encoding of », and when trying to decode it as UTF-8, you get an invalid char. \xC2\xBB is the UTF-8 encoding, which is what Go uses.

from kobopatch.

Also, are you sure this displays properly on the Kobo? AFAIK, Qt uses the UTF-8 encoding by default.

EDIT: Yep, I checked. Replacing it with \xBB will not work (it will show a square box), as it needs \xC2\xBB (the UTF-8 encoding). So, the fact that it counts it as two bytes was correct from the start, as it is (due to the UTF-8 encoding). I've added this to the FAQ on the kobopatch thread.

Here is what you should use:

Allow searches on Extra dictionaries:
  - Enabled: no
    ## To allow searches on Extra dictionaries change space character at end of
    ## "Extra: " to another char (ex: "Extra:_")
    ## The space char causes a non-desired "English - English" when searching on
    ## Extra dictionary from main menu.
  - FindBaseAddressString: "\0Extra:\x20"
  - ReplaceString: {Offset: 7, Find: "\x20\0", Replace: "»"}

from kobopatch.

Thanks for the suggestion.

BTW, where do you see the squares in the GUI? I've attached two pictures that show how it looks like for me (20 replaced by BB: 00 45 78 74 72 61 3A BB 00 (no \xC2\xBB)). One picture is from the dictionary lookup, the other from the language settings.

I don't have to work around utf-8 in libnickel.so but in KoboReader.sqlite and Kobo eReader.conf.

from kobopatch.