[Serve] Performance degradation in Deployments about ray HOT 7 OPEN

@edoakes @shrekris-anyscale

from ray.

@xwu99 I tried it on latest master and this is what I got

Can you try with latest Ray 2.10.0 or nightly and see if you are still still this behavior?

from ray.

from ray.

I tried on another server and also run it as baremetal and inside a container, it still has the issue. already using latest Ray 2.10.

The result from docker

Dockerfile:

# Use an official Python runtime as a parent image
FROM python:3.9-slim

# Set the working directory in the container
WORKDIR /app

# Install Ray and any other dependencies
RUN pip install ray[serve]

# Copy your application code into the container
COPY . /app

However, when I used my laptop's WSL environment, it worked without performance degradation:

Did you try it on a server or a desktop? Could you share your configs?

from ray.

Hi @xwu99 The original screenshot provided was from my local. I just built the image with the exact dockerfile you shared and this is the results

Seems to be consistent within some margin of error. I feel maybe there are something special in your hardware that's slowing down the computation? It also seemed to take a lot more time to even start up (~18s) in your vs even in the container I'm seeing ~5s start up time?

from ray.

Hi @xwu99 The original screenshot provided was from my local. I just built the image with the exact dockerfile you shared and this is the results Seems to be consistent within some margin of error. I feel maybe there are something special in your hardware that's slowing down the computation? It also seemed to take a lot more time to even start up (~18s) in your vs even in the container I'm seeing ~5s start up time?

Thanks for the update. the 18s comes from I changed the calculation range from this result.
for i in range(1, 200000000):
result = i**0.5
After changed back to 100000000, it's 5.856838226318359. I used htop to confirm there is no active processing in the background. But it's still weird to see the perf differences inside Ray deployment.

Could you share your CPU type and freq for your 5 sec result?

I would like to use an AWS instance to try this again.

from ray.

@GeneDer another round of testing. I found when Ray is running in the background. The standalone python script also slowed down.

Also if I disabled dashboard using ray start --head --include-dashboard False, the performance is back to normal. But I can't imagine why background dashboard can impact another script's performance.

from ray.