trbs / bucky Goto Github PK

Is there anyone still maintaining this project? Given the list of pending pull requests, it looks like no one at Cloudant is giving the project any attention.

In production I have several counters that wont change for various nodes, this shouldn't log.error.

If a type is specified without using the (optional) comma as follows:

rabbitmq_details avg:GAUGE:0:U avg_rate:GAUGE:0:U rate:GAUGE:0:U samples:GAUGE:0:U

Bucky fails to start complaining of to many values to unpack.

This line works fine

rabbitmq_details avg:GAUGE:0:U, avg_rate:GAUGE:0:U, rate:GAUGE:0:U, samples:GAUGE:0:U

As the commas are optional, perhaps this could be addressed.

https://collectd.org/documentation/manpages/types.db.5.shtml

delimited by spaces and, optionally, a comma (",")

Original statsd support percentile configuration:

https://github.com/etsy/statsd/blob/master/exampleConfig.js

  percentThreshold: for time information, calculate the Nth percentile(s)
                    (can be a single value or list of floating-point values)
                    negative values mean to use "top" Nth percentile(s) values
                    [%, default: 90]

However bucky have hardcoded value -

https://github.com/trbs/bucky/blob/master/bucky/statsd.py#L182

Hi,

Using Bucky to recieve statsd data from a slow (i.e. 5 mins update cycle) probe. enqueue_gauges seems to reset metric to 0 during its flush operation. This results in a example data stream "value,0 . . . 0,value, 0 , 0 " being passed to carbon.

Ideally gauge values need to be only pushed to carbon when updated i.e. remove the key post flush. I've worked around by commenting out "self.gauges[k]=0" to maintain the previous value which isn't ideal but gives more accurate data than current default behaviour.

Update setup.py with new maintainership and new github url.

Hi,

When i stop carbon-cache, bucky crashes.
Not sure if this is expected but shouldn't happen.

It would be great feature to disable some timer's metrics like sum, upper, lower, etc.

StatsD talks to graphite directly, and collectd has the write_graphite plugin so it's not clear what advantages Bucky has to offer.

It would be great if the README.rst could clarify this.

Using bucky as a sink for collectd and statsd it would be nice to be able to define rules (regex or functions) that allow arbitrary namespace rewriting. With this it would be possible to ensure any arbitrary resulting graphite namespace tree regardless of what collectd or statsd try to name things. Would others find this useful? When implement this are there any obvious considerations to be take into account?

Merge bucky2 with bucky

This would make it possible to update gauges only by delta values, even on long term.

Hello,

I'm quite new to statsd world and behaviour, but it seems to me that etsy's statsd send metrics about is own behaviour to Graphite, in the 'stats_counts' namespace.

It seems that I'm not receiving this metrics from my bucky instance, is there any posibility to activate this behaviour?

I opened the dummy PR #44 to trigger the travis-ci tests.

Python 3.4 tests always freezes on scenario 001-test-statsd.test_larger_count and never completes.
All the rest python versions fail in scenarios 002-test-collectd.test_crypto_auth_reload and 004-test-helpers.test_file_monitor, both depending on watchdog module. See https://travis-ci.org/trbs/bucky/builds/33185352

After pinning watchdog to version 0.7.1 (the latest release is 0.8.1), python 3.4 tests still freeze on 001-test-statsd.test_larger_count, but all other versions succeed. See https://travis-ci.org/trbs/bucky/builds/33186621

So we have 2 issues here, one with python 3.4 and the statsd tests, and one with the recent watchdog version.

While InfluxDB supports the carbon protocol and can act as a carbon server, native InfluxDB support would be much appreciated. The client code would probably have to be factored out into a interface/module architecture first though.

If I try to use collectd-types, I get the following:

(wheezy)root@extinctvm:~# bucky --collectd-types=/opt/telemetry/statsbox/types.db
Traceback (most recent call last):
  File "/usr/local/bin/bucky", line 9, in <module>
    load_entry_point('bucky==2.2.2', 'console_scripts', 'bucky')()
  File "/usr/local/lib/python2.7/dist-packages/bucky-2.2.2-py2.7.egg/bucky/main.py", line 216, in main
    parser.parse_args(values=cfg)
  File "/usr/lib/python2.7/optparse.py", line 1399, in parse_args
    stop = self._process_args(largs, rargs, values)
  File "/usr/lib/python2.7/optparse.py", line 1439, in _process_args
    self._process_long_opt(rargs, values)
  File "/usr/lib/python2.7/optparse.py", line 1514, in _process_long_opt
    option.process(opt, value, values, self)
  File "/usr/lib/python2.7/optparse.py", line 788, in process
    self.action, self.dest, opt, value, values, parser)
  File "/usr/lib/python2.7/optparse.py", line 800, in take_action
    values.ensure_value(dest, []).append(value)
AttributeError: 'module' object has no attribute 'ensure_value'

The workaround I am using is to symlink /usr/local/share/collectd/types.db to /opt/telemetry/statsbox/types.db

No idea how to fix this.

This occurs for both the 'pip install' version and the version from git.

Hey guys,

I'm having a lot of issues with Bucky, and I'm wondering whether this might be related to me using the python version in the Squeeze repositories (2.6.6).

So before I try to debug any issues, should Bucky even work on that old a version?

I just discovered that when running --debug bucky doesn't actually ship metrics and instead just prints to stdout. I found this behavior surprising. Can someone confirm that is correct?

If so, I'll happily submit a PR which notes that in the --help and readme documentation. And also perhaps mentions using --log-level=DEBUG for live-ish debugging.

StatsDServer process' StatsDHandler thread appears not alive after starting.

Apparently due to where StatsDHandler is created vs. started.
Moving StatsDHandler creation to StatsDServer.run seems to solve this problem.

https://github.com/trbs/bucky/blob/cda507241c8898c3a1926cae18371bce84be6d2c/bucky/carbon.py#L102C25-L102C25

[2023-11-06 18:11:29,614][INFO] carbon - Connected to Carbon at 127.0.0.1:2003
Process PlaintextClient-5:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/data/coral/graphite/lib/python3.6/site-packages/bucky/client.py", line 45, in run
    self.send(*sample)
  File "/data/coral/graphite/lib/python3.6/site-packages/bucky/carbon.py", line 102, in send
    self.sock.sendall(mesg)
TypeError: a bytes-like object is required, not 'str'

adding .encode('utf8') to mesg in failing line seems to help.

Shutting down Bucky is currently done with a very blunt axe.
All childs are terminated, which could have some edge cases:

http://docs.python.org/2/library/multiprocessing.html#multiprocessing.Process.terminate

Using the close() method to stop the run() main loop should go a long way to make this better.

The pypi package claims to be version 0.3.1, however, the files included are largely 0.2.6. In particular, the "statsd_legacy_namespace" feature is not implemented.

For validation, I downloaded the tgz files for bucky 0.2.6 and 0.3.1 from pypi, and a zip file of master from github & diff'd them.

diff between 0.2.6 & 0.3.1

$ diff -qrs bucky-0.2.6/ bucky-0.3.1 | grep -v identical
Files bucky-0.2.6/.gitignore and bucky-0.3.1/.gitignore differ
Files bucky-0.2.6/PKG-INFO and bucky-0.3.1/PKG-INFO differ
Files bucky-0.2.6/README.rst and bucky-0.3.1/README.rst differ
Files bucky-0.2.6/bucky/__init__.py and bucky-0.3.1/bucky/__init__.py differ
Files bucky-0.2.6/bucky/client.py and bucky-0.3.1/bucky/client.py differ
Files bucky-0.2.6/bucky/udpserver.py and bucky-0.3.1/bucky/udpserver.py differ
Files bucky-0.2.6/bucky.egg-info/PKG-INFO and bucky-0.3.1/bucky.egg-info/PKG-INFO differ
Files bucky-0.2.6/setup.py and bucky-0.3.1/setup.py differ
$

diff between 0.3.1 and master

$ diff -qrs bucky-0.3.1/ bucky-master/ | grep -v identical
Only in bucky-master/: .coveragerc
Files bucky-0.3.1/.gitignore and bucky-master/.gitignore differ
Only in bucky-master/: .travis.yml
Only in bucky-master/: CHANGES
Files bucky-0.3.1/MANIFEST.in and bucky-master/MANIFEST.in differ
Only in bucky-0.3.1/: PKG-INFO
Files bucky-0.3.1/README.rst and bucky-master/README.rst differ
Files bucky-0.3.1/bucky/__init__.py and bucky-master/bucky/__init__.py differ
Files bucky-0.3.1/bucky/carbon.py and bucky-master/bucky/carbon.py differ
Files bucky-0.3.1/bucky/cfg.py and bucky-master/bucky/cfg.py differ
Files bucky-0.3.1/bucky/collectd.py and bucky-master/bucky/collectd.py differ
Files bucky-0.3.1/bucky/errors.py and bucky-master/bucky/errors.py differ
Files bucky-0.3.1/bucky/main.py and bucky-master/bucky/main.py differ
Files bucky-0.3.1/bucky/metrics/counter.py and bucky-master/bucky/metrics/counter.py differ
Files bucky-0.3.1/bucky/metrics/gauge.py and bucky-master/bucky/metrics/gauge.py differ
Files bucky-0.3.1/bucky/metrics/histogram.py and bucky-master/bucky/metrics/histogram.py differ
Files bucky-0.3.1/bucky/metrics/meter.py and bucky-master/bucky/metrics/meter.py differ
Files bucky-0.3.1/bucky/metrics/metric.py and bucky-master/bucky/metrics/metric.py differ
Files bucky-0.3.1/bucky/metrics/stats/ewma.py and bucky-master/bucky/metrics/stats/ewma.py differ
Files bucky-0.3.1/bucky/metrics/stats/expdec_sample.py and bucky-master/bucky/metrics/stats/expdec_sample.py differ
Files bucky-0.3.1/bucky/metrics/stats/usample.py and bucky-master/bucky/metrics/stats/usample.py differ
Files bucky-0.3.1/bucky/metricsd.py and bucky-master/bucky/metricsd.py differ
Files bucky-0.3.1/bucky/statsd.py and bucky-master/bucky/statsd.py differ
Files bucky-0.3.1/bucky/udpserver.py and bucky-master/bucky/udpserver.py differ
Only in bucky-0.3.1/: bucky.egg-info
Only in bucky-master/: bucky.py
Only in bucky-master/: requirements.txt
Only in bucky-master/: runit
Only in bucky-0.3.1/: setup.cfg
Files bucky-0.3.1/setup.py and bucky-master/setup.py differ
Only in bucky-master/: supervisord
Only in bucky-master/: systemd
Only in bucky-master/: tests
$

on ubuntu 11.04 (x64), python version 2.7.1+

when specifying a config file:
bucky /path/to/bucky_conf.py

I get the error:
"No handlers could be found for logger "bucky.main""

edit:
sorry, neglected to mention the version of bucky I'm using, which is the latest master

This repo is still sending commit notifications to its previous home's IRC channel; it'd be great to disable those.

Cheers!