Different execution times for python spin app on local machine and Fermyon Cloud about spin HOT 6 OPEN

Hello, thanks for the report. I am not aware of any reason for a local execution to be that much slower than Fermyon Cloud. Would it be possible for you to publish the code required to reproduce?

from spin.

You can find the code in this repository to reproduce.
https://github.com/abdulmonum/spin-python-app.git

from spin.

For comparison, on my Linux AMD 5900X desktop, time curl http://127.0.0.1:3000/float takes ~0.22s. Could you give more information about your local environment?

from spin.

Hello, I changed my local environment and for simple runs time curl http://127.0.0.1:3000/float takes ~0.35s which makes sense. However, if I run a Poisson workload of ~ 2 reqs/sec, then many requests take around ~ 0.7s. If I run bombardier (https://github.com/codesenberg/bombardier) ./bombardier http://127.0.0.1:3000/float, I get the following output:

Bombarding http://127.0.0.1:3000/float for 10s using 125 connection(s)
[======================================================================================================================================================================================================] 10s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec         2.19      19.23     250.16
  Latency         8.78s      2.99s     10.01s
  HTTP codes:
    1xx - 0, 2xx - 26, 3xx - 0, 4xx - 0, 5xx - 0
    others - 120
  Errors:
       timeout - 120
  Throughput:     3.61KB/s

Why is the spin app on the local environment not able to handle many requests at the same time? If I understand correctly, for every http request sent, a new webassembly instance is spawned, serves the request, and tears down. Theoretically, that should mean we should have consistent response times (atleast when the arrival rate is as low as 2 reqs/sec). I observe that on Fermyon cloud where I get an average response time of 0.41 sec (including network latency), but shouldn't the response times be consistent in the local environment? Is there some sort of queuing of requests because this does not seem to me a Wasm issue.

My current environment:
Intel E3-1230 v3 @ 3.30GHz
16GB RAM
Ubuntu 22.04

from spin.

@lann Any explanation for this?

from spin.

Sorry, missed your previous update.

Is there some sort of queuing of requests because this does not seem to me a Wasm issue.

Yes, there is implicit queuing of async tasks in the Tokio multi thread runtime.

Any explanation for this?

I ran a few tests at different concurrency levels (bombaridier -c N ...):

• -c 1: ~200ms avg, ~2ms SD
• -c 10: ~260ms avg, ~38ms SD
• -c 100: ~3000ms avg, ~2000ms SD

My host has 24 cores, though bombardier itself will cause some extra contention when testing entirely locally. This roughly makes sense to me for CPU-bound workloads: as request concurrency reaches multiples of the number of cores you would expect avg latency to scale similarly.

The CPU you mention appears to have 8 "cores" (threads), so at a concurrency of 125 / 8 = ~15.6 * 350ms = ~5.4s, which seems reasonably close to your 8.8s avg when accounting for various sources of overhead.

from spin.