etsy / mctop Goto Github PK

View Code? Open in Web Editor NEW

505.0 68.0 63.0 295 KB

a top like tool for inspecting memcache key values in realtime

License: MIT License

Ruby 100.00%

non-sox

mctop's Introduction

mctop is an Archived Prjoect

mctop is no longer actively maintained. Your mileage with patches may vary.

mctop

Inspired by "top", mctop passively sniffs the network traffic passing in and out of a server's network interface and tracks the keys responding to memcache get commands. The output is presented on the terminal and allows sorting by total calls, requests/sec and bandwidth.

You can read more detail about why this tool evovled over on our code as craft blog.

mctop depends on the ruby-pcap gem, if you don't have this installed you'll need to ensure you have the development pcap libraries (libpcap-devel package on most linux distros) to build the native gem.

How it works

mctop sniffs network traffic collecting memcache VALUE responses and calculates from traffic statistics for each key seen. It currently reports on the following metrics per key:

calls - the number of times the key has been called since mctop started
objsize - the size of the object stored for that key
req/sec - the number of requests per second for the key
bw (kbps) - the estimated network bandwidth consumed by this key in kilobits-per-second

Getting it running

the quickest way to get it running is to:

ensure you have libpcap-devel installed
git clone this repo
in the top level directory of this repo bundle install (this will install the deps)
then either:
- install it locally rake install; or
- run it from the repo (good for hacking) sudo ./bin/mctop --help

Command line options

Usage: mctop [options]
    -i, --interface=NIC              Network interface to sniff (required)
    -p, --port=PORT                  Network port to sniff on (default 11211)
        --host=HOST                  Network host to sniff on (default all)
    -d, --discard=THRESH             Discard keys with request/sec rate below THRESH
    -r, --refresh=MS                 Refresh the stats display every MS milliseconds
    -h, --help                       Show usage info

User interface commands

The following key commands are available in the console UI:

C - sort by number of calls
S - sort by object size
R - sort by requests/sec
B - sort by bandwidth
T - toggle sorting by ascending / descending order
Q - quits

Status bar

The following details are displayed in the status bar

sort mode - the current sort mode and ordering
keys - total number of keys in the metrics table
packets - packets received and dropped by libpcap (% is percentage of packets dropped)
rt - the time taken to sort and render the stats

Changelog

2012-12-14 - Now compatible with Ruby 1.8.x (tested on 1.8.7-p371)

Known issues / Gotchas

ruby-pcap drops packets at high volume

from my testing the ruby-pcap native interface to libpcap struggles to keep up with high packet rates (in what we see on a production memcache instance) you can keep an eye on the packets recv/drop and loss percentage on the status bar at the bottom of the UI to get an idea of the packet

No binary protocol support

There is currently no support for the binary protocol. However, if someone is using it and would like to submit a patch, it would be welcome.

mctop's People

Contributors

Stargazers

Watchers

mctop's Issues

Does it work with elastic cache?

Hi. Check with you does it work with AWS elastic cache?

need some exception handling in the sniffer thread

ruby-pcap will throw exceptions at initialization (eg. invalid interface passed in). Given it's running in a seperate thread these fail whilst the UI is blissfully unaware and so is the user until exit time.

Need to work our a nice way to pass them through to the UI or bail if pcap can't initialize correctly

curses dependency?

I'm unable to follow the steps in the README and get it working - I'm on Ubuntu 12.10.

I installed libpcap-dev, bundle installed the deps, then did rake install. Running mctop, either system-wide or from within the repo, gives me:

``require': cannot load such file -- curses`

I installed libncurses-ruby and libncurses5-dev, but doing a gem install ncurses failed during native compilation with the error:

form_wrap.c: In function ‘make_arg’:
form_wrap.c:1126:38: error: ‘struct RArray’ has no member named ‘len’
form_wrap.c:1130:5: warning: format not a string literal and no format arguments [-Wformat-security]

So I'm stuck. Any help?

`require': cannot load such file -- curses (LoadError)

I have problems running mctop with Ruby 2.1.2p95 (on OS X):

$ ./bin/mctop --help
/Users/dentarg/.rubies/ruby-2.1.2/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- curses (LoadError)
    from /Users/dentarg/.rubies/ruby-2.1.2/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /Users/dentarg/src/mctop/lib/ui.rb:1:in `<top (required)>'
    from /Users/dentarg/.rubies/ruby-2.1.2/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /Users/dentarg/.rubies/ruby-2.1.2/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from ./bin/mctop:12:in `<main>'

Probably due to the fact that curses was removed from standard library in Ruby 2.1.0: https://bugs.ruby-lang.org/issues/8584

NoMethodError on run

Tried both with the gem install and cloning the repo, when running mctop i get the follwing message:

$ bin/mctop 
/usr/local/share/gems/gems/ruby-pcap-0.7.9/lib/pcap/tcpdump_time_format.rb:10:in `<top (required)>': private method `include' called for Time:Class (NoMethodError)
    from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /usr/local/share/gems/gems/ruby-pcap-0.7.9/lib/pcap_misc.rb:2:in `<top (required)>'
    from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:135:in `require'
    from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:135:in `rescue in require'
    from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:144:in `require'
    from /home/khernandez/mctop/lib/cmdline.rb:2:in `<top (required)>'
    from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
    from bin/mctop:10:in `<main>'

Is this a missing dependency in the documentation (such as Ruby version) or something of the sort? Currently running on CentOS with ruby 2.0.0p598

P.O.C. statsd sniffer for mctop

Did a super quick POC of adding a statsd key sniffer:
https://github.com/draco2003/mctop/tree/statsd_sniffer

Could see this being useful for other simple key type protocols like graphite, redis and so on...

Wondering if this is something that would get pulled in if it was cleaned up and implemented in more of a plugin type model?

wrong number of arguments (1 for 0) (ArgumentError)

I just installed mctop on our dev box, (yum install libpcap-devel, gem install pcap, gem install mctop) and I get this error when trying to run it:

[root@dev temp]# mctop --interface=eth0
/usr/lib/ruby/gems/1.8/gems/mctop-0.0.3/bin/../lib/ui.rb:77:in round': wrong number of arguments (1 for 0) (ArgumentError) from /usr/lib/ruby/gems/1.8/gems/mctop-0.0.3/bin/../lib/ui.rb:77:inrender_stats'
from /usr/lib/ruby/gems/1.8/gems/mctop-0.0.3/bin/mctop:39
from /usr/bin/mctop:19:in `load'
from /usr/bin/mctop:19

This is on Centos 6.3 x86_64

MemcacheSniffer error on load

I'm sure there's something obvious I'm missing, but I'm not overly familiar with Ruby so excuse if it's something obvious.

When trying to run sudo mctop -i eth1 , I got the following error:

/var/lib/gems/1.8/gems/mctop-0.0.3/bin/../lib/sniffer.rb:16:in initialize': uninitialized constant MemcacheSniffer::Mutex (NameError) from /var/lib/gems/1.8/gems/mctop-0.0.3/bin/mctop:16:innew'
from /var/lib/gems/1.8/gems/mctop-0.0.3/bin/mctop:16
from /usr/local/bin/mctop:19:in `load'
from /usr/local/bin/mctop:19

Binary protocol

Are there plans to support the binary protocol? I had a quick look at the RFC to see if I can implement it quickly but it doesn't return the key in the response which would make it rather difficult...

Feature Request: make cap.setfilter port configurable

We use twemproxy, and it would be very handy to be able to sniff an alternate port. Request additional cmdline flag for that (default to 11211 of course).