String decoding errors on python3 about pyulog HOT 7 CLOSED

I prefer the exception because it makes it clear the log is corrupt.
If you want to make it an option, that works for me as well.

from pyulog.

Thanks, that sounds good. I'll make a PR.

from pyulog.

Just to be clear, the defaut behaviour when parsing strings in pyulog currently is different for python 2 and python 3. Python2 silently ignores and python3 fails over. I think the default behaviour should report the error and print warning but not failover, keeping pyulog robust to external errors.

from pyulog.

I think the default behaviour should report the error and print warning but not failover, keeping pyulog robust to external errors.

There's a trade-off: if you have parsing errors, you cannot trust the data. Can you make it a parameter disable_parser_exceptions to ULog(), and if True, set self._file_corrupt instead of raising the exception?

from pyulog.

There's a trade-off: if you have parsing errors, you cannot trust the data.

While that's true, often log analysis comes down to recovering as much logged info from a logfile as possible - especially for crash logs. Only then you'd consider the matter of 'trustability' of the data. I can't imagine a scenario where a user expects a log parsing pipeline to stop midway and discard rest of the logged data because it encountered unexpected bytes (especially true if the corrupt bytes are not message identifiers, but string payload of message packets!).

Still unclear points to me in discussion are difference in python2 and python3 behavior, and whether ascii should be the default or utf-8.

Can you make it a parameter disable_parser_exceptions to ULog(), and if True, set self._file_corrupt instead of raising the exception?

Sounds like a good idea.

from pyulog.

Also note #40

from pyulog.

As I wrote I'm fine with making this configurable (the default can be either way). What's important to me is that we have the reporting of exceptions, both in Flight Review and the scripts.

Still unclear points to me in discussion are difference in python2 and python3 behavior, and whether ascii should be the default or utf-8.

Preferably utf-8.

from pyulog.