Comments (8)
I can see three options:
- provide a more fine-grained, optional
raw_frame_type
field that encodes the actual stream type number instead of the conceptual name we have in frame_type (like how UnknownFrame works today) - require people to log the (partial) (STREAM) frame header in the raw field to allow for later re-parsing
- do nothing
Proposal: use 1.
from qlog.
Can you elaborate on 1 more? By the time my parser informs me I have a STREAM frame, I might have discarded the on-wire bytes
from qlog.
This would be an -extra- option to log besides the frame_type
in case you really need to know if you got 0x08, 0x09, 0x0A ... or 0x0F (at least for STREAM, for ACK it'd be 0x02 or 0x03).
This could be parsed if you logged the first few bytes in raw
for example, but having a separate field would be more direct. We have precedent for this in e.g., ConnectionCloseFrame.raw_error_code
.
If you don't have that information available at the point you're logging but still need it well... you'd need to log somewhere closer to the wire? In the ACK case, it is implicit through the logging of ECN info, for stream it is somewhat for the presence of the FIN bool, but for length and offset, it's more difficult to know sometimes.
This is not crucial information, but could be good to know in some highly specific edge case debugging.
Got a bug in pcap2qlog about this just the other day, where I forgot to extrapolate the length field from the QUIC packet length if it wasn't present in the STREAM frame itself (quiclog/pcap2qlog#7)
from qlog.
This could be parsed if you logged the first few bytes in raw for example, but having a separate field would be more direct. We have precedent for this in e.g., ConnectionCloseFrame.raw_error_code.
Some the the question here is what do you mean by wire image. Should something like raw_error_code
be raw bytes underlying the varint (maintaining the information on how the number was encoded on the wire), or the parsed varint.
Since raw_error_code
a single int, it must be the parsed varint, which is slightly easier. But even today I can't populate that because my implementation has discarded the data by the time I come to generate the qlog. Changing the code would boil down to me changing the logging pipeline, parsing the same value twice, and/or storing the data twice just for qlog.
I'll note that for H3 this is even harder, because we have to handle non-atomic varint reads as STREAM data comes.
from qlog.
So the idea was:
Option 1: parsed varint (so literally the 0x08 - 0x0F for STREAM)
Option 2: raw wire image (so "unparsed" varint)
I do see the confusion... probably we should use another word than raw
here, since raw_error_code
is intended to also be the parsed varint, but the raw:RawInfo
fields are intended to be pure wire bits.
With regards to your implementation not having that info available... I'm not sure what you want me to say... these are optional fields that you can output if needed/available, but not something I would expect many implementations to do (by default). I don't see a way of solving the original issue here (representing all the wire image nuances in some form) without having access to those nuances at the logging location :)
Could you elaborate on because we have to handle non-atomic varint reads as STREAM data comes
with an example maybe? Not sure what you mean there?
from qlog.
The main point is that it's fairly straightforward to log the unprocessed bytes of a QUIC packet or QUIC frame, that's because an implementation will have received those in whole and know their length. E.g. Read UDP, everything in the datagram.
For things on top of QUIC streams it becomes harder because they can span lots of packets. A streaming parser will possibly not have an entire frame available when it want to log an event.
If the proposal is to log the parsed varint, it makes things slightly easier
from qlog.
how about an optional field frame_type_value: uin64
, which can be used in any frame type that spans a range of possible values. Then we'll update UnknownFrame s/raw_frame_type/frame_type_value.
from qlog.
Fixed by #274
from qlog.
Related Issues (20)
- H3Settings is a bit underspecified HOT 4
- packet_sent doesn't allow logging of IP address (for packets sent outside of a connection) HOT 4
- `ConnectionClose` storing a uint64 error_code and a uint64 error_code_value seems redundant HOT 4
- Split out ECN "capable" state HOT 1
- Difference between negotiated vs actually accepted/enabled TPs HOT 2
- Logging spin bit policy HOT 2
- Give concrete guidance on how to extend qlog
- Connectivity::Connection_closed application trigger type is confusing HOT 1
- Logging non-UTF-8 reasons HOT 2
- Logging of HTTP field names or values that contain illegal bytes
- Discuss extensibility of extension points through type or group sockets
- Make logging raw byte values consistent
- Move Trigger section underneath Data section
- Register application/qlog+json and application/qlog+json-seq media types
- protocol_type or protocol_types
- How is "trigger" a common subfield of "data"? HOT 1
- is parameters_restored missing fields defined in parameters_set?
- Extensions to events can not be indicated by schema
- JSON-SEQ is painful HOT 14
- parsing the `serialization_format` field requires prior knowledge of the serialization format HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from qlog.