zaphoyd / wsperf Goto Github PK

I am still unsure how to best approach.

E.g. on one hand, there is I think a fundamental tradeoff between throughput and latency. Maximizing throughput would be simply done by overwhelming the server with so many outstanding WS connections that there is always stuff to process - up to a hard limit of course (like accept queue depth).

This will result in best throughput, but probably bad latencies for individual WS connections.

Then, for a server that can process incoming connections so fast that the high watermark is never reached (when operating in modes other than --close keepopen), it might be hard to find out the best value for high watermark (that is just below hard limit) even if I am only interested in throughput.

At least for me, I could find out more / understand better, if we had the following 2 counters in wsperf:

• total number of transitions to high-watermark reached (pausing)
• total number of transitions to low-watermark reached (resuming)

When running multi-threaded (or also with a single thread and with testee on same machine), it seems important for performance (absolulte and consistency) to set the CPU core affinity of each thread.

E.g. I have now implemented that in the Autobahn testee and was able to bring down (essential more than half!) Median and Q90 latencies:

     Min:       1.5 ms
      SD:      21.0 ms
     Avg:      24.8 ms
  Median:      16.7 ms
  q90   :      42.9 ms
  q95   :      75.1 ms
  q99   :     105.3 ms
  q99.9 :     122.6 ms
  q99.99:     129.0 ms
     Max:     132.6 ms

When running on e.g. a 8 core machine, having the option to start 'wsperf` like this would be ideal:

 wsperf --workers 0,1,2,3 ...

This would then allow to start the testee

testee --workers 4,5,6,7 ...

With Python, there is the psutil package, which relies on the sched_setaffinity system call on Linux.

The feature (setting thread/process CPU affinity) should be available on other platforms as well.

It would be helpful to have 4 options for the behavior after opening handshake has been completed:

1. leave open
2. perform WebSocket closing handshake (as now)
3. perform orderely close of TCP connection
4. just hard drop the TCP

1 This is useful to test scalability of server wrt maximum open WebSocket connections.

3 This is useful to test scalability for this scenario: a plain old Web site (with multiple HTML pages, not single-page Web app), that has some embedded JS which opens a WS. Think somelike like Google Analytics. When a user click a link or closes the tab, the browser will not perform a WebSocket closing handshake. In fact, it will never do, unless explicitly asked for.

4 This is useful to test how the server behaves when TCP are just hard dropped. DoS resistence, or when there are networking issues.

That could be a command line option like --close keepopen, --close wsclose, --close tcpclose or --close tcpdrop.

Environment variable has:

WSPP_ROOT=/Users/arungupta/workspaces/websocketpp
BOOST_ROOT=/Users/arungupta/tools/boost_1_55_0

Invoking "scons" in wsperf directory gives the following error:

Traceback (most recent call last):
File "/usr/local/bin/scons", line 5, in
pkg_resources.run_script('SCons==2.3.0', 'scons')
File "build/bdist.macosx-10.9-intel/egg/pkg_resources.py", line 488, in run_script
ns['name'] = name
File "build/bdist.macosx-10.9-intel/egg/pkg_resources.py", line 1354, in run_script
def _extract_resource(self, manager, zip_path):
File "/Library/Python/2.7/site-packages/SCons-2.3.0-py2.7.egg/EGG-INFO/scripts/scons", line 188, in
import SCons.Script
ImportError: No module named SCons.Script