Comments (9)
Preparation
First, I would do your work against the feature/http2-server
branch. Once #687 merges, you might have to rearrange some code, and there's a new compression ns in that PR that you might need to use or add to.
HTTP/1
For the H1 client code, what we'll probably want to do is, look for the wrap-decompression
symbol in the middleware list, and if present, add an instance of the HttpContentDecompressor
handler after the HttpClientCodec
in setup-http1-pipeline
.
HTTP/2
For the H2 client, I believe AlephHttp2FrameCodecBuilder
will already decorate the decompression codec properly. You might not have to do anything extra as long as :compression?
is true.
Though on a side note, we probably shouldn't add h2-compression-handler
to client pipelines, just servers.
Zstd issues
One catch is, Netty doesn't support Zstd decoding yet. We have several options:
- Don't support Zstd at all. It's not too popular yet, and most browsers don't support it (https://caniuse.com/zstd). Should probably parse and strip zstd from the
accept-encoding
header. - Wait for netty/netty#13531 to add it. The PR has been in-progress since August, and it's unclear when it will land.
- Update the
byte-transforms
library to support Zstd and Brotli, and use that to transform the InputStream in Aleph code before handing off the body InputStream to users. This will require extra work for raw mode, since we'd then have to convert back to ByteBufs. - Add Apache commons-codec support. I think this is what clj-http uses. Will require another dep, and probably hsa the same issue as (3) with raw mode.
Might be simplest to do (1), and add full support when (2) is ready.
from aleph.
HTTP2 may complicate this, as it appears the primary way to decompress is to wrap the codec in the Netty pipeline with a decompression version. Nah, that's solved.
from aleph.
I'm interested in working on this ticket.
from aleph.
from aleph.
@KingMob Thanks for your suggestions.
Now #687 is merged, I looked into the code and couldn't find wrap-decompression
middleware. Am I missing anything here?
Also, I didn't get this what do you mean regarding the note -
we probably shouldn't add h2-compression-handler to client pipelines, just servers.
from aleph.
@rajcspsg wrap-decompression
doesn't exist yet. It's in clj-http, and needs to be added to Aleph. Or to be more explicit, support for it needs to be added, but not all of it will be proper middleware.
Part of this will have to be done with Netty. Compression in Netty isn't done with a pipeline handler, but by injecting codec classes into the Http2FrameCodec
handler. So you'll need to look at AlephHttp2FrameCodecBuilder
. Some of the decompression code you need is already in place, but I haven't looked at which pieces are missing for H2, if any.
If you call (aleph.http.AlephHttp2FrameCodecBuilder/setCompression builder true some-options-here)
it will add the necessary DelegatingDecompressorFrameListener
which should already look for the "content-encoding" header and decompress correctly.
So for H2, I think this PR will need to
- Look for the
wrap-decompression
symbol in the user's client middleware list - Call
.setCompression(true, ...)
on AlephHttp2FrameCodecBuilder - Add a real middleware fn that mimics the
update-in
of clj-http'swrap-decompression
to update the "accept-encoding" header's list of supported codecs. - Potentially remove the "content-encoding" header after decompression, if it's not already done by Netty. (This is to discourage anyone downstream from trying to decompress a second time.) Or better still, remove it and add an "original-content-encoding" header for debugging purposes.
Unlike clj-http, we probably want to add Brotli to the default list. clj-http defaults to adding "gzip, deflate" to whatever's already in the header (if anything), which is not quite correct, but good enough 99.99% of the time. We should leave out zstd and snappy from the defaults, I think. Zstd is promising but still only has partial support. Snappy is niche, and not supported by browsers.
we probably shouldn't add h2-compression-handler to client pipelines, just servers.
This is a little out-of-date. Right now, we don't add it to clients, but the docs and key names used aren't very clear that it's server-only.
The h2-compression-handler
is about parsing the client's "accept-encoding" header to determine what encoding to return. In theory, this could be symmetrical, and clients could encode their bodies, but for pragmatic and security reasons, client request body encoding isn't a thing, and we should change the code to make it clearer that it's server-only, and prevent misuse. (E.g., rename the keys, update AlephHttp2FrameCodecBuilder to compress or decompress, but not both simultaneously, etc.) You don't have to do that in this PR, though I think maybe I should update the key names before we do a full release of 0.7.0.
from aleph.
BTW, did you vote on https://clojurians.slack.com/archives/C0G922PCH/p1702723929287379 ?
from aleph.
BTW, did you vote on https://clojurians.slack.com/archives/C0G922PCH/p1702723929287379 ?
yes I did :)
from aleph.
Hmm, I don't see your name on the vote emojis...
from aleph.
Related Issues (20)
- Please bump netty versions because of CVE-2023-34462 HOT 1
- Enable endpoint identification (aka hostname verification) for TLS clients by default? HOT 9
- Look into new Ring websocket support for Aleph HOT 2
- Add write backpressure by checking Channel writability
- Add support for aggregating all inbound data before calling server Ring handler HOT 2
- Default rejected-handler will leak buffers in raw stream HTTP servers HOT 1
- Overhaul docstring formatting for consistency HOT 2
- Hide application protocol config from users with `http-versions`
- Who uses Aleph? Add your name to this, we'd like to know HOT 4
- Reconsider `:aleph/`-namespaced keywords HOT 3
- io.netty.channel.AbstractChannelHandlerContext invokeExceptionCaught HOT 2
- 0.7.0-rc1 is crashing without brotli4j dependency #3530 HOT 2
- Treat de/compression asymmetrically in H2
- Track down potential idle-handler refcount testing leak HOT 11
- Question about single server process using both non-ssl and ssl HOT 3
- Support cancellation of HTTP requests HOT 2
- HTTP client connection pool timeout doesn't free up queue HOT 4
- Drop HTTP client connection pool's acquire queue HOT 1
- CVE-2024-29025 HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aleph.