请问docker一个进程能支持多少并发呀? about openai-forward HOT 7 CLOSED

@seetimee
这里做了一个简单的性能测试：benchmark

from openai-forward.

Bot detected the issue body's language is not English, translate it automatically.

Title: How many concurrencies can one docker process support?

The server is 2c2g. Do I need to open multiple processes?

from openai-forward.

Hi @seetimee ,
支持多大并发量确实是个重要的问题，但我暂时还没有做这个实验 :(

• fastapi + uvicorn的性能还是过得去的，网络请求方面使用httpx库（它的异步性能低于aiohttp, 但一般情况下应是够用的）。
• 通常可以将workers设置为n ~ 2n+1之间(n是物理核心)，但由于整个服务都是异步的，可能workers设置超过n并不会带来性能提升。 单个进程空载时会占用约50MB的内存（但若设置了REQ_RATE_LIMIT会导致内存占用随用户数的升高而相应提升）

from openai-forward.

Bot detected the issue body's language is not English, translate it automatically.

Hi @seetimee ,
How much concurrency is supported is indeed an important issue, but I haven’t done this experiment yet :(

• The performance of fastapi + uvicorn is still passable, and the httpx library is used for network requests (its asynchronous performance is lower than aiohttp, but it should be sufficient under normal circumstances).
• Usually workers can be set to between n ~ 2n+1 (n is the physical core), but since the entire service is asynchronous, setting workers exceeding n may not bring performance improvement. A single process will occupy about 50MB of memory when it is unloaded (but if REQ_RATE_LIMIT is set, the memory usage will increase accordingly as the number of users increases)

from openai-forward.

Bot detected the issue body's language is not English, translate it automatically.

@seetimee
Here is a simple performance test: benchmark

from openai-forward.

@seetimee
这里做了一个简单的性能测试：benchmark

感谢感谢

from openai-forward.

Bot detected the issue body's language is not English, translate it automatically.

@seetimee
Here is a simple performance test: benchmark

thanks thanks

from openai-forward.