Benchmarking against other proxies about kong HOT 15 CLOSED

In regard to benchmarking, I think it would make sense to have a distributed multi-client benchmarking tool that we can start also for benchmarking the performance between critical code changes, something like http://www.process-one.net/en/tsung/

Distributed: the load can be distributed on a cluster of client machines (http://www.process-one.net/docs/tsung/user_manual.pdf)

from kong.

There's a SAAS for that.

from kong.

https://www.blitz.io/
http://blazemeter.com/
https://loader.io/
http://loadimpact.com/

from kong.

SAAS services could be an option, but they are very expensive with many concurrent requests so we need to evaluate that.

from kong.

I'm not sure why we need a "distributed multi-client benchmarking tool". Why can't we just run something like siege on a large ec2 instance and use a lot of threads? We only have to benchmark one instance of kong. As long as the instance used to run siege has more threads than the one running kong I don't see why that wouldn't be fine.

from kong.

Benchmarking the interaction between Kong and Cassandra requires lots of incoming requests.

siege will work for benchmarking a single nginx server on one machine, but it wouldn't work when we want to simulate a bigger load on Cassandra that requires multiple nginx servers.

We can start with siege (and an easy way to launch it and retrieve benchmarking information), but keep in mind that for deeper benchmarking we would need more than one client (and more than one nginx server to simulate lots of incoming connections on Cassandra).

from kong.

Besides Tsung there is also http://locust.io

from kong.

Sounds good, can we set something up @mschurenko @ataner ?

from kong.

@mschurenko @ataner Can we get Kong running on AWS? This would serve for both benchmarking and learning more about how different settings (OS, nginx, kong, cassandra) and architectures (cassandra on the same box vs external vs clusters) effect performance.

When we have that we need to benchmark the following scenarios:

1. Kong vs Nginx vs Tyk proxies
2. Kong Proxy vs Direct API

So that means we also need to set up an AWS instance for raw nginx and tyk using the same instance type and applicable configurations.

from kong.

+1 we need to know where we're at now. Also Umbrella

We should talk about setups (docker and cassandra)

from kong.

So are those finished? Published? Please update the issue @SGrondin

from kong.

I'm still waiting on your Tyk instructions to remove all rate limiting. I couldn't figure it out whatsoever. By default it has some rate limiting and the docs are unclear about wheter it's even possible.

from kong.

My Tyk instructions? For 2 weeks? I don't have such thing lol. i never removed the rate limiting from Tyk.

from kong.

@thibaultcha any updates? what's the results of kong vs tyk?

from kong.

updates on this? very crucial for decision making

from kong.