Make server workers channel buffered? about grpc-go HOT 11 OPEN

someone's benchmark shows buffered channel can speed up channel select，it means buffered channel is more effect. if requests queued means cpu overload or slow io. in cpu overload case spawn a goroutine immediately can not effect，because requests queued just from channel queue move to golang runtime queue, it is just in another way to queued, spawn a new goroutine also cost time. in slow io case, just can improve input qps. so I think buffered channel can improve proformance

from grpc-go.

Do you have a repro case where NumStreamWorkers with buffering on the serverWorkerChannel is visibly faster?

Current approach is based on the following logic: if all NumStreamWorkers goroutines are busy then spawning new goroutine is O(us), while buffering the request and waiting for a free goroutine is O(ms to seconds). Hence, the current approach tries to minimize tail latencies of the server.

from grpc-go.

### It supposed that NumStreamWorkers is enough. In golang GMP model, goroutine is task of thread queued in Processor. Spawning new goroutine is O(us), it is just goroutine create time, you ingnored goroutine schedule time. Spawned goroutine may not scheduled immediately，it may also (like queue in channel) queued in Processor.

from grpc-go.

Do you have a repro case where NumStreamWorkers with buffering on the serverWorkerChannel is visibly faster?

Current approach is based on the following logic: if all NumStreamWorkers goroutines are busy then spawning new goroutine is O(us), while buffering the request and waiting for a free goroutine is O(ms to seconds). Hence, the current approach tries to minimize tail latencies of the server.

In another word，if channel buffer appear accumulate. It means goroutine will also accumulate in Processor.In that case must increase thread num and cpu cores.

from grpc-go.

You are totally right for the cases when all gorotines are CPU bound and do not yield. In cases where there is IO involved (e.g. a standard backend that spends most of it's time waiting on an upstream database) you usually can find a P faster than you can process a request end-to end.

That said, it is all theoretical, if you have a repro where unbuffered channel here is a bottleneck, it would simplify decision making. Also, if we were to make this channel buffered, what would be the right buffer size? 1, GOMAXPROC, other?

from grpc-go.

If you think this approach is faster, can you provide a benchmark with the difference?

from grpc-go.

If you think this approach is faster, can you provide a benchmark with the difference?

https://zhuanlan.zhihu.com/p/101063277 In this benchmark shows suitable buffered channel is more faster.

from grpc-go.

On the micro-benchmark side there is no doubt, based on Amdahl's law, that a buffered channel would be faster. But if this was the only concern developers care about channels would not even have a non-buffered mode.

It would be better if you showed the benefit of buffered channel using an end-to-end test with grpc-go with NumStreamWorkers set on a simple web-app using something like k6. (ideally, using different setting for the buffer size to see the scaling properties.)

from grpc-go.

This issue is labeled as requiring an update from the reporter, and no update has been received after 6 days. If no update is provided in the next 7 days, this issue will be automatically closed.

from grpc-go.

Hi @YangXin-Sheep -- Like @SaveTheRbtz suggested - it would be great if you could show us the benefit of buffered channel using an gRPC-go example since the benchmark results that you provided just shows us the benefit of buffered go channels.

that being said, even if we decide to enable NumStreamWorkers with buffering on the serverWorkerChannel - Not sure what the right size of the buffer should be.

from grpc-go.