Echo server grinds to a crawl when handling many sequential requests about tokio HOT 8 CLOSED

Thanks for the report! Can you detail how you generated the numbers in the table? The command you provided just shows throughput for one connection (I think) for me, so I wasn't sure how you were sending successive connections.

from tokio.

In my entire report, I meant request = line (one request for Echo as Service). So yeah, a single connection.

The numbers were generated with the command shown. You have to divide the bytes/s by two to get requests/s since each request (line) is 2 bytes. You can also wait for the command to finish and divide the time reported by the second dd instance by the count= number.

from tokio.

Isn't this just showing that if each requests sends more data it takes longer to process that request?

from tokio.

No, all requests are the same size (note the LineCodec).

from tokio.

I believe this is due to this loop https://github.com/tokio-rs/tokio-proto/blob/8fb8e482dcd55cf02ceee165f8e08eee799c96d3/src/streaming/pipeline/server.rs#L136-L138

This is hitting a pessimistic case where there's millions of concurrent requests on one connection and because they're pipeliend they must be completed in order, and right now they're all polled every time.

If this is a problem for you I'd recommend avoiding tokio-proto and using something like stream.buffered(N) as it has a more scalable implementation.

from tokio.

The docs suggest there might be a way to limit the number of in-flight requests, but I can't figure out how.

from tokio.

I think that might never have been implemented :)

from tokio.

Hopefully this is resolved now. Closing due to inactivity. Feel free to open again on the repo (tokio-proto) that fits best. This repo is being reclaimed to implement tokio-rs/tokio-rfcs#3.

from tokio.