Support for permessage-deflate [✅/awaiting_dependency_merge] about tungstenite-rs HOT 26 OPEN

Opened #235. Any feedback is appreciated.

from tungstenite-rs.

Would it maybe be a good idea to implement the parsing of the Sec-Websocket-Extensions header into this crate? The PR agains headers has shown no progress for over ten months. If the PR ever gets merged, it should be pretty simple to switch over to using that version.

from tungstenite-rs.

I'm currently working on this. So far, I've got it to pass some autobahn (no parameter support, so client passes 12, server passes both 12 and 13) by layering compression/decompression on top of the existing flow, so it doesn't need as much code, and it's clearly additive. I don't think there's any major breaking changes.

$ git diff --cached --stat src/
 src/error.rs                |  14 ++++-
 src/extensions/deflate.rs   | 217 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/extensions/mod.rs       |  89 ++++++++++++++++++++++++++++
 src/handshake/client.rs     |  87 +++++++++++++++++++++++----
 src/handshake/server.rs     |  28 ++++++++-
 src/lib.rs                  |   1 +
 src/protocol/frame/frame.rs |  11 ++++
 src/protocol/mod.rs         | 137 +++++++++++++++++++++++++++++++++++++++----
 8 files changed, 558 insertions(+), 26 deletions(-)

For now, I'm not planning to implement server_max_window_bits/client_max_window_bits support because that requires flate2/zlib feature, and I don't need them, but it should be easy to add them later with a separate feature flag.

Later, we should be able to support other compressions like snappy too.

from tungstenite-rs.

Could we make it without headers PR? Such a great feature is blocked because of external dependency for such a long time.

from tungstenite-rs.

Any updates on this? This is the current state of permessage-deflate in rust's websocket libraries:

• websocket does not support permessage-deflate (websockets-rs/rust-websocket#123)
• The repo for twist seems to be gone (https://github.com/rustyhorde/twist)
• tungstenite does not support permessage-deflate (You are here!)
• ws is the only one that supports permessage-deflate, but it is abandoned. There is a security issue that is fixed in its fork parity-ws, but it also does not seem to be actively maintained. Unfortunately, compression in ws-rs no longer works with the newest versions of rustc: housleyjk/ws-rs#330
• Can't find info about permessage-deflate in websocket-lite but it probably isn't supported
• yarws supports permessage-deflate but only for incoming messages (I need it for outgoing messages)

from tungstenite-rs.

As a suggestion I'd say to implement this using something like flate2 and not libz-sys in order to avoid external dependencies on system libraries

from tungstenite-rs.

Yes, we still plan to work on it, but unfortunately we did not have time to work on it yet as it was lower in priorities than other things which we fixed / improved since then.

from tungstenite-rs.

#144

from tungstenite-rs.

The corresponding PR was closed as the author did not have to time to update it / address the remarks, so he decided to close the PR. The new PRs are welcome ;)

from tungstenite-rs.

Thanks, it will be done. The only reason not having it already is, we don't use it ourselves, and the library was written for our production use.

Note for myself: it is RFC 7692.

from tungstenite-rs.

I think I brainfarted when I thought it is mandatory.

from tungstenite-rs.

At least it is required by Autobahn, that's enough for me :)

Currently tungstenite passes all the Autobahn tests with only two of them as "non-strict", and my plan is to achieve 100% strict (I know what exactly fails). For comparison, ws-rs has 11 non-stricts and others are even worse. So the support for RFC 7692 is needed at least to have 100% green lines here. :)

from tungstenite-rs.

It is actually mandatory for a user agent, see step 9 of https://fetch.spec.whatwg.org/#concept-websocket-establish.

from tungstenite-rs.

Is it still planned? ws-rs support permessage-deflate but does not support async :(

from tungstenite-rs.

Someone needs to update the #144 to the newest state (resolve the conflicts and fix some remarks in code), once it's done and clean, we could merge it. Unfortunately we don't plan to make changes at this particular place in the nearest future, but if someone makes a PR and/or updates the existing #144 which seems to be [almost] ready (?), we would be glad to review and merge it.

from tungstenite-rs.

We should probably mention @SirCipher to update pull request :)

from tungstenite-rs.

@application-developer-DA, @Fedcomp, sorry for the delay, we've been updating to Tokio 0.3. The work is nearly done and I'll update the PR soon.

from tungstenite-rs.

Do you have an update?
Thanks!

from tungstenite-rs.

@rymnc, the PR is still awaiting a review at the moment. I've got a local branch that's updated to the most recent version for both this repository and tokio-tungstenite that I'll commit soon too

from tungstenite-rs.

Okay, np
thanks for the reply

from tungstenite-rs.

Hi, do you have any news about the permessage-deflate support?

from tungstenite-rs.

@kazk Is there any assistance needed to help get this PR across the finish line?

from tungstenite-rs.

@TheDan64
It's blocked by hyperium/headers#88. More specifically hyperium/headers#88 (comment). Maybe providing some feedback on that will be helpful to get that merged.

Once that's merged, I'll open a new PR from a branch using it, which is a rebased version of #235 with header parsing extracted and with better error handling.

from tungstenite-rs.

Tried out your rebased branch for our use case and had no issues - appears to be working well. Hope to see this upstreamed sooner than later so that we can use it in production & with the other tungstenite sister libraries

from tungstenite-rs.

The implementation by @kazk resides in https://github.com/snapview/tungstenite-rs/tree/permessage-deflate and has been temporarily reverted from a master branch, see the discussion in the PR for more details.

from tungstenite-rs.

FWIW, I have rebased branches of all the relevant functions all the way to axum-tungstenite (alternative to axum::extract::ws) to use the permessage-deflate. Tested to work fine.

My specific use case needs high performance, and configurable window bits, so there is some hardcoding of window bits as I didn't want to implement full window bits handling yet. I can implement full window bits handling... but I'd want the permessage-deflate implementation by @kazk merged first.

hyperium/headers@master...nakedible-p:hyperium-headers:deflate-release
master...nakedible-p:snapview-tungstenite-rs:deflate-release
snapview/tokio-tungstenite@master...nakedible-p:snapview-tokio-tungstenite:deflate-release
davidpdrsn/axum-tungstenite@main...nakedible-p:davidpdrsn-axum-tungstenite:deflate-release

from tungstenite-rs.