Allow more wire-image indicators about qlog HOT 8 CLOSED

I can see three options:

1. provide a more fine-grained, optional raw_frame_type field that encodes the actual stream type number instead of the conceptual name we have in frame_type (like how UnknownFrame works today)
2. require people to log the (partial) (STREAM) frame header in the raw field to allow for later re-parsing
3. do nothing

Proposal: use 1.

from qlog.

Can you elaborate on 1 more? By the time my parser informs me I have a STREAM frame, I might have discarded the on-wire bytes

from qlog.

This would be an -extra- option to log besides the frame_type in case you really need to know if you got 0x08, 0x09, 0x0A ... or 0x0F (at least for STREAM, for ACK it'd be 0x02 or 0x03).

This could be parsed if you logged the first few bytes in raw for example, but having a separate field would be more direct. We have precedent for this in e.g., ConnectionCloseFrame.raw_error_code.

If you don't have that information available at the point you're logging but still need it well... you'd need to log somewhere closer to the wire? In the ACK case, it is implicit through the logging of ECN info, for stream it is somewhat for the presence of the FIN bool, but for length and offset, it's more difficult to know sometimes.

This is not crucial information, but could be good to know in some highly specific edge case debugging.

Got a bug in pcap2qlog about this just the other day, where I forgot to extrapolate the length field from the QUIC packet length if it wasn't present in the STREAM frame itself (quiclog/pcap2qlog#7)

from qlog.

This could be parsed if you logged the first few bytes in raw for example, but having a separate field would be more direct. We have precedent for this in e.g., ConnectionCloseFrame.raw_error_code.

Some the the question here is what do you mean by wire image. Should something like raw_error_code be raw bytes underlying the varint (maintaining the information on how the number was encoded on the wire), or the parsed varint.

Since raw_error_code a single int, it must be the parsed varint, which is slightly easier. But even today I can't populate that because my implementation has discarded the data by the time I come to generate the qlog. Changing the code would boil down to me changing the logging pipeline, parsing the same value twice, and/or storing the data twice just for qlog.

I'll note that for H3 this is even harder, because we have to handle non-atomic varint reads as STREAM data comes.

from qlog.

So the idea was:
Option 1: parsed varint (so literally the 0x08 - 0x0F for STREAM)
Option 2: raw wire image (so "unparsed" varint)

I do see the confusion... probably we should use another word than raw here, since raw_error_code is intended to also be the parsed varint, but the raw:RawInfo fields are intended to be pure wire bits.

With regards to your implementation not having that info available... I'm not sure what you want me to say... these are optional fields that you can output if needed/available, but not something I would expect many implementations to do (by default). I don't see a way of solving the original issue here (representing all the wire image nuances in some form) without having access to those nuances at the logging location :)

Could you elaborate on because we have to handle non-atomic varint reads as STREAM data comes with an example maybe? Not sure what you mean there?

from qlog.

The main point is that it's fairly straightforward to log the unprocessed bytes of a QUIC packet or QUIC frame, that's because an implementation will have received those in whole and know their length. E.g. Read UDP, everything in the datagram.

For things on top of QUIC streams it becomes harder because they can span lots of packets. A streaming parser will possibly not have an entire frame available when it want to log an event.

If the proposal is to log the parsed varint, it makes things slightly easier

from qlog.

how about an optional field frame_type_value: uin64, which can be used in any frame type that spans a range of possible values. Then we'll update UnknownFrame s/raw_frame_type/frame_type_value.

from qlog.

Fixed by #274

from qlog.