gnustep-base 1.28.0 triggers a regression in sogo about libs-base HOT 16 CLOSED

In Alinto/sogo#324, there is a patch that looks good and solves the problem on the SOGo side.

from libs-base.

looking, but i'm suspicious about bd5f290 now.

from libs-base.

testing with sogo 5.2.0 built against gnustep-1.28.0 with bd5f290 reverted, the setup is working fine. So that's definitely a regression from bd5f290, to which i dunno if sogo should adapt ...

from libs-base.

It really would help to understand which encoding your code is requesting here. Most likely it is NSUnicodeStringEncoding with the dataUsingEncoding: method on NSString. You should be fine when using NSUTF16BigEndianStringEncoding.

from libs-base.

Looking into the original bug report you linked to I can see that this is about UTF-8 and there this makes no sense. As far as I am aware we do not add a BOM for UTF-8. What could happen is that there is already a BOM in the string. Now it really would help to understand how this data gets produced.

from libs-base.

From what i can understand (and i understand absolutely nothing to obj-c) getetag (mapped to davEntityTag in sope, https://github.com/inverse-inc/sope/search?q=getetag + https://github.com/inverse-inc/sogo/search?q=davEntityTag) and calendar-data (mapped to davCalendarData in sope, https://github.com/inverse-inc/sope/search?q=calendar-data + https://github.com/inverse-inc/sogo/search?q=davCalendarData) seem to be NSString. maybe @cgx would know ?

from libs-base.

I've been bitten by this, too; for details please see

https://www.sogo.nu/bugs/view.php?id=5416#c15591

@landryb's findings and patch saved my day, thanks!

from libs-base.

From the code I could find in SoGo this is not triggered by a encoding conversion. The most likely davEntityTag method looks like this:
- (id) davEntityTag { return [NSString stringWithFormat: @"\"gcs%.8d\"", version]; }
(where version is an unsigned int value)

But I don't see where this gets used together with the calendar-data. Could somebody with a bit of SoGo knowledge please explain how to reproduce this issue with pure GNUstep code?

from libs-base.

@rfm could you please have a look here? I am a bit confused on what is actually going on here.
While looking at the change that seems to cause all this, I noticed that the optimisation for the dataUsingEncoding: method is only present for the constant string class, not for the other ones. Was that on purpose? This should be unrelated to this issue, as the code in NSString looks correct to me.

from libs-base.

(Drive-by comment: It seems like this would be grounds for adding a minimum unit test demonstrating the expected behavior, so we don't break this in the future..)

from libs-base.

@ivucica you are correct. The problem is just that we don't know what features of NSString SOGo is using here.

from libs-base.

@rfm this is a tests whether you now get mails for bug reports. Your help here is appreciated.

from libs-base.

Well, it appears likely that something is putting a BOM in a string and that's then being output as utf-8, but we really need to find out where. I agree that a unit test to reproduce it would be good.
NB. The only BOM related changes I recall in the last year or two were making gnustep-base behavior consistent with OSX behavior when converting to/from encodings, so ideally we need a testcase that can be run on OSX as well as on gnustep , to see where any problem is.

from libs-base.

The place to look is the code generating that vcard. Perhaps it does something like converting bits of the vcard to NSData objects using NSUnicodeStringEncoding (which will produce a BOM at the start of each NSData object), then concatenating the NSData objects, then creating a string with embedded BOMs? Alternatively, perhaps in gnustep-base there is something that works like that somewhere? Either way, looking at the SoGo code generating the vcard string is a good starting point to see what is happening.

from libs-base.

@cgx please can you have a look at this ? Your insight as a sogo developer would be helpful, since it affects sogo pretty badly.

from libs-base.

fwiw, with sogo 5.8.0 i've tested that Alinto/sogo#324 properly fixed the issue i was seeing, so i'd close this issue as "not a gnustep-base regression" :) thanks !

from libs-base.