Proposal: add a new codepoint for QUIC v1 (RFC 9000), and for future incompatible versions about multiaddr HOT 17 CLOSED

Thank you for your comprehensive response, @MarcoPolo!

I think we should be consistent, i.e. either interpret /quic as /quic-draft29 or /quic-v1 across all our multiaddr implementations.

Agreed. Let’s keep the current codepoint and string /quic to mean quic-draft29. We may even want to alias this to have new nodes represent this as the string /quic-draft29 to be clearer, but this is a bit of a tangent.

That makes sense. The hope is that in the long term (as QUIC draft-29 is phased out), the /quic string representation will also slowly die. For the transition period, we should parse /quic as /quic-draft29. Once we disable draft-29 support (libp2p/go-libp2p#1841 suggests a transition period of 6 months), we might also stop parsing /quic.

from multiaddr.

What's the difference between dropping draft-29 support in 6 months compared to dropping it right now? If I understand it correctly dropping support for draft-29 would avoid the need for a new codepoint?

No we still would want a new codepoint so that new nodes can know before dialing if the node speaks v1 or draft 29. If we don’t adopt a new codepoint and assume everyone uses quicv1 then you incur a penalty when communicating with (the many) old nodes.

from multiaddr.

cc @Stebalien @elenaf9 @MarcoPolo @kpp @thomaseizinger @mxinden

from multiaddr.

3. prefer QUIC v1. This will remove the 1 RTT penalty, but will incur a 1 RTT penalty when connecting to legacy nodes. Given the large number of nodes, this might affect our TTFB metrics.

I guess ideally, we go with this option but somehow minimize the 1 RTT penalty. What do you think of this idea:

• For the time being, prefer draft-29.
• Define a <sunset date> at which point go-libp2p will switch to prefer QUIC v1 instead.
• Emit a warning every time you hit the 1 RTT penalty by having to upgrade to QUIC v1.
• Emit a warning every time you connect you connect to a draft-29 node that this connection will incur a 1 RTT penalty from <sunset date>.

This should give node operators some time to notice the issue and upgrade their software accordingly. Making an informed decision on <sunset date> should allow you to mitigate most of the risk on affecting the TTFB metric.

from multiaddr.

• Emit a warning every time you hit the 1 RTT penalty by having to upgrade to QUIC v1.
• Emit a warning every time you connect you connect to a draft-29 node that this connection will incur a 1 RTT penalty from .

You can't do that: You can't update old nodes, so the node operators who need to see the warning will never see it. Instead, you'd just be spamming the logs of node operators of updated nodes, potentially with multiple messages per second. Even worse, these messages wouldn't be actionable.

from multiaddr.

• Emit a warning every time you hit the 1 RTT penalty by having to upgrade to QUIC v1.
• Emit a warning every time you connect you connect to a draft-29 node that this connection will incur a 1 RTT penalty from .

You can't do that: You can't update old nodes, so the node operators who need to see the warning will never see it.

Hmm okay, I assumed that people will update at some point but yeah, if there are many operators who don't update at all, then this is definitely an issue.

Instead, you'd just be spamming the logs of node operators of updated nodes, potentially with multiple messages per second. Even worse, these messages wouldn't be actionable.

I mean, that is fixable. You can just emit the log only once per peer. Also, you'd remove the log again in the version that prefers QUICv1 so updated nodes would not emit that.

I am a bit on the fence here. On the one hand, adding a new protocol is a legitimate upgrade path. On the other hand, it feels like a bit of a waste to use when the situation is:

a) temporary
b) penalizes mostly nodes which run outdated software

Can we make an estimate of how long the grace-period of a time-based trigger that switches to v1 would have to be to have most of the network prefer v1? Are we talking weeks, months or years?

Another, more sophisticated mechanism for such kind of upgrades is to have nodes keep track¹ of how many of their peers support v1 and switch to preferring that once it hits a configured threshold.

Assuming that nodes connect to a representative subset of the network, this would make the switch gracefully as soon as a configured threshold is capable of v1. This is similar to Bitcoin's softfork activation techniques.

from multiaddr.

b) penalizes mostly nodes which run outdated software

That’s totally fine. We don’t want to penalize nodes that run up-to-date software though, which is what we’d do if we switched to dialing v1 by default.

it feels like a bit of a waste to use when the situation is:

a) temporary

It’s not any more temporary than any other update of QUIC from one incompatible version to the other.

I find it quite instructive to think about what we’d what to do once an incompatible QUIC version is specified, e.g one that’s using a different handshake protocol: As I’ve described in my original post, we’d want to specify a new code point in that case. Thus, minting a new code point for QUIC v1 seems consistent.

from multiaddr.

it feels like a bit of a waste to use when the situation is:
a) temporary

It’s not any more temporary than any other update of QUIC from one incompatible version to the other.

I find it quite instructive to think about what we’d what to do once an incompatible QUIC version is specified, e.g one that’s using a different handshake protocol: As I’ve described in my original post, we’d want to specify a new code point in that case. Thus, minting a new code point for QUIC v1 seems consistent.

Should we then be more elaborate in the naming of the new code point? Like quic-rfc9000? I guess ideally, the current quic one would be called quic-draft-29. Might be a good mental note to take for future protocols.

I am okay with a new code point, it is a valid upgrade / negotiation technique and really the only annoying thing is the aesthetics and documentation effort.

b) penalizes mostly nodes which run outdated software

That’s totally fine. We don’t want to penalize nodes that run up-to-date software though, which is what we’d do if we switched to dialing v1 by default.

I would still find it interesting to back this with some data. Does go-libp2p support RFC9000 today? Does it use RFC9000 if both nodes support it? Are there any numbers on what % of connections that is?

from multiaddr.

@elenaf9 What would your implementation strategy be on the Rust side? Can you give an estimate how much work this would be?

I wrote up what's needed in go-libp2p in libp2p/go-libp2p#1841.
Not sure if you want to support draft-29 in rust-libp2p as well, it might be fine to just support QUIC v1. Since rust-libp2p never had QUIC support to begin with, you'd not be causing a regression.

from multiaddr.

@mxinden @MarcoPolo I'm in favor of moving forward with this proposal, but it would be helpful to get your input here.

from multiaddr.

First off, thanks for the detailed write-up @marten-seemann!

rust-libp2p is adding QUIC support, and quinn only supports RFC 9000. If we don't do anything, this means:

Not sure if you want to support draft-29 in rust-libp2p as well, it might be fine to just support QUIC v1. Since rust-libp2p never had QUIC support to begin with, you'd not be causing a regression.

For the record, does quinn support draft-29 or does it not @elenaf9?

2. prefer QUIC v1. This will remove the 1 RTT penalty, but will incur a 1 RTT penalty when connecting to legacy nodes. Given the large number of nodes, this might affect our TTFB metrics.

To be able to make an informed decision, can someone add numbers to "large number of nodes [running draft-29 in the IPFS network". Unfortunately I don't have this data with kademlia-exporter.max-inden.de/. @dennis-tra maybe?

We could add a new code point for QUIC v1 (string representation: quicv1). The existing code point would be reinterpreted to mean QUIC draft-29.

I think renaming the text representation of the /quic code point to /quic-draft29 adds complexity which is unfortunate. That said, with the suggestion below, in case we are consistent across the many multiaddr implementations, it would be worth it.

Implementations MAY choose to continue accepting /quic multiaddresses (and interpret them as quic-draft29 or quic-v1 at their discretion).

multiformats/multicodec#298

I think we should be consistent, i.e. either interpret /quic as /quic-draft29 or /quic-v1 across all our multiaddr implementations.

Overall I don't have a strong opinion here. My intuition tells me to:

• Not add a new code-point.
• Make the change to prefer QUIC RFC-9000, thus pay the extra cost of 1RTT to older nodes.
• See this as another argument for investing into upgrade-your-IPFS-node advocacy.

from multiaddr.

To be able to make an informed decision, can someone add numbers to "large number of nodes [running draft-29 in the IPFS network". Unfortunately I don't have this data with kademlia-exporter.max-inden.de/. @dennis-tra maybe?

These are the go-ipfs v0.7.0 nodes, so it's a very large fraction of the IPFS network (30-40%).

See this as another argument for investing into upgrade-your-IPFS-node advocacy.

The cost is only paid by nodes that upgraded, so the incentives are severely misaligned here.

from multiaddr.

Thanks Marten! I think this is worth doing sooner rather than later. I think I’m in agreement with the original post. Here’s a bit of clarification on the exact semantics since I think we’ve been discussing a lot of different things in this thread (that’s a good thing!).

This issue is only about adding a new code point for quic-v1, not about whether to rename the existing code-point to quic-draft-29. Let’s talk about the rename in a another issue (I’m for it, but it needs some thought around implementation). Letting nodes be explicit in their support of quic-v1 seems like a great thing, and (besides an extra multiaddr or two) I don’t see any downsides here. rust-libp2p may only want to advertise quic-v1, although according to the release notes on quinn they support draft-29.

I think we should add the new codepoint as soon as possible so that rust-libp2p can start using it and we can start advertising it in go-libp2p. The new codepoint allows dialers to know ahead of time what version to use.

Then there’s the very closely related issue of:

go-libp2p nodes now have two options:
i. continue preferring draft-29. This will incur a 1 RTT penalty when connecting to rust-libp2p nodes, as we'll need to perform version negotiation.
ii. prefer QUIC v1. This will remove the 1 RTT penalty, but will incur a 1 RTT penalty when connecting to legacy nodes. Given the large number of nodes, this might affect our TTFB metrics.

(I’m defining new nodes as nodes post this upgrade; old nodes as nodes prior to this upgrade)
If new nodes advertise their explicit support of quic-v1 then we don’t pay the 1 RTT penalty when new nodes dial other new nodes (they know to use quic-v1). If a new node only sees a node support the quic codepoint (draft 29) then it should dial with draft-29. We don’t incur a 1RTT penalty if rust-libp2p nodes advertise quic-v1 since the new node would have seen that multiaddr.

Option i. is strictly better than ii. because we can avoid the 1RTT penalty by only dialing quic-v1 if the other node explicitly advertises it, which will be the case if we agree on a new this new codepoint sooner and have rust-libp2p use the quic-v1 codepoint. Option ii. forces a 1 RTT penalty when connecting to older nodes (a significant portion of the network currently). Am I missing something here?

I think codepoints are cheap. The way we represent a list of multiaddrs is a bit inefficient, but we can optimize this in the future if it becomes a problem.

I think we should be consistent, i.e. either interpret /quic as /quic-draft29 or /quic-v1 across all our multiaddr implementations.

Agreed. Let’s keep the current codepoint and string /quic to mean quic-draft29. We may even want to alias this to have new nodes represent this as the string /quic-draft29 to be clearer, but this is a bit of a tangent.

tl;dr I agree the new codepoint is a useful addition. I don’t see any major downsides to introducing it. Let’s do it.

from multiaddr.

Apologies for the late reply here.

libp2p/rust-libp2p#2289, and quinn only supports RFC 9000. If we don't do anything, this means:

Not sure if you want to support draft-29 in rust-libp2p as well, it might be fine to just support QUIC v1. Since rust-libp2p never had QUIC support to begin with, you'd not be causing a regression.

For the record, does quinn support draft-29 or does it not @elenaf9?

Quinn supports draft-29, however it does not support version negotiation on the client side. Instead, when initiating a new outbound connection we have to set the QUIC version in the client config. If the server sends back a Version negotiation packet because it does not support that version the connection attempt will error with VersionMismatch. See quinn-rs/quinn#1249.

@elenaf9 What would your implementation strategy be on the Rust side? Can you give an estimate how much work this would be?

Given the above, it wouldn't be much work to support both the new and the old-codepoint. When initiating a new dial we'd check the codepoint, in case of /quic or /quic-draft-29 set out client config version to draft-29, else use default QUIC v1. As a server we support all (>= draf-29) versions, but our listening addresses would have the /quic-v1 codepoint.

Once we disable draft-29 support (libp2p/go-libp2p#1841 suggests a transition period of 6 months)

What's the difference between dropping draft-29 support in 6 months compared to dropping it right now? I understand that go-ipfs v0.7 makes up a significant portion of the IPFS network, however go-ipfs v0.7 is two years old. If those nodes did not upgrade for all that time do we expect anything to change in the next 6 months?
If I understand it correctly dropping support for draft-29 would avoid the need for a new codepoint (at least until a new incompatible version is published)?

from multiaddr.

What's the difference between dropping draft-29 support in 6 months compared to dropping it right now? If I understand it correctly dropping support for draft-29 would avoid the need for a new codepoint?

No we still would want a new codepoint so that new nodes can know before dialing if the node speaks v1 or draft 29. If we don’t adopt a new codepoint and assume everyone uses quicv1 then you incur a penalty when communicating with (the many) old nodes.

Okay makes sense.

Don't have a strong opinion on this; adding a new codepoint sounds reasonable 👍. Will do a PR for it on rust-multiaddr.

from multiaddr.

iiuc we executed on /quic-v1 and /quic-v1/webtransport, implemented in Rust/Go,
and recently shipped in Kubo 0.18 and the rollout has begun:

• https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.18.md#differentiating-quic-versions

is there anything else to be done here, or can we close this?

from multiaddr.

This has been fully resolved and released.

from multiaddr.