feuerlabs / exometer Goto Github PK
View Code? Open in Web Editor NEWBasic measurement objects and probe behavior
License: Mozilla Public License 2.0
exometer's People
Contributors
Stargazers
Watchers
Forkers
blt sylane hwinkel dcorbacho roland-karlsson-erlang-solutions-com james-kelly chinnurtb dszoboszlay igorkarymov linearregression philwitty unisontech eryx67 loucash dalmatinerdb msch dskliarov jbiel mpmiszczyk marknijhof gordonguthrie wdshin reachfh puzza007 paraphraseagh pspdfkit-labs runninghooligan wuchuguang ruanpienaar mykook dasudian parsifal-47 hardikus br xerions laijichang mweibel nyco ates bougueil synlay dzimi esl satishck1992 rmoorman sifoxdevteam kairion-dev ulfa simplydata systra philosodad tahteche gomoripeti imdeps platbox namjae ctrlshiftbryan blitzstudios ruchika1001 lastcanal xiaofengliao stephb9959 enikki tolbrino jacktang m-2k nx2501 exosite-archive zorbash nt6 xaptum hippware thecodeboss petarmarinov arkgil afatsini hillcool mkacper felipeevangelista tklebanoff xircleag alex9spiridonov 5l1v3r1 adampetersen-act3 c1lo miloudeloumri stjordanis floatingfront singularitypostman shpaky mohamedalikhechine isabella232 loissotolopez aef-exometer's Issues
SNMP reporter: Counter32 limitation
Hello,
Looks like the metrics which has type counter or fast_counter always has type Counter32 during MIB creation. It's hardcoded in code.
Is it ok to use Counter32 for values bigger than 2 ^ 32 or it's may cause an overflow?
Thanks.
histogram values crash on retrieval of 999
> erl -pa ebin deps/*/ebin
Erlang R16B02 (erts-5.10.3) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Eshell V5.10.3 (abort with ^G)
1> exometer:start().
16:56:06.986 [info] Application lager started on node nonode@nohost
16:56:06.998 [info] Starting reporter with []
16:56:07.002 [info] Application exometer started on node nonode@nohost
ok
2> exometer:new([a,simple,histogram], histogram).
ok
3> exometer:info([a,simple,histogram]).
[{name,[a,simple,histogram]},
{type,histogram},
{module,exometer},
{status,enabled},
{cache,0},
{value,[]},
{timestamp,undefined},
{options,[{type_arg,exometer_histogram}]},
{ref,{<0.60.0>,exometer_histogram}},
{datapoints,[n,mean,min,max,median,50,75,90,95,99,999]}]
4> exometer:update([a,simple,histogram], 100).
ok
5> exometer:update([a,simple,histogram], 50).
ok
6> exometer:update([a,simple,histogram], 25).
ok
7> exometer:update([a,simple,histogram], 12.5).
ok
8> exometer:update([a,simple,histogram], 6.25).
ok
9> exometer:get_value([a,simple,histogram], median).
{ok,[{median,25}]}
10> exometer:get_value([a,simple,histogram], mean).
{ok,[{mean,38}]}
11> exometer:get_value([a,simple,histogram], 999).
** exception error: no function clause matching
exometer:exo_proc_get_datapoints(999,
{exometer_entry,
[a,simple,histogram],
histogram,exometer,enabled,0,undefined,
undefined,
[{type_arg,exometer_histogram}],
{<0.60.0>,exometer_histogram}}) (src/exometer.erl, line 514)
in function exometer:get_value_/2 (src/exometer.erl, line 221)
in call from exometer:get_value/2 (src/exometer.erl, line 196)
Lack of static metric configuration makes exometer restart tricky.
The documentation section Creating Metrics notes the use of exometer:new/2 to create metrics at runtime. If for whatever reason the exometer application is stopped and then restarted, metrics so created will be lost.
A method of statically creating metrics--similar to the static creation of subscriptions--is desirable.
Starting app using exometer crashes in exometer_folsom_monitor
Context: Application is using rebar and relx. Including exometer and recon.
Retrieving tagged version 1.1 using rebar I get the following error on app startup:
2015-02-11 11:05:49.657 [error] <0.91.0> CRASH REPORT Process <0.91.0> with 0 neighbours exited with reason: call to undefined function setup:find_env_vars(exometer_folsom_monitor) in gen_server:init_it/6 line 330
2015-02-11 11:05:49.657 [error] <0.87.0> Supervisor exometer_sup had child exometer_folsom_monitor started with exometer_folsom_monitor:start_link() at undefined exit with reason call to undefined function setup:find_env_vars(exometer_folsom_monitor) in context start_error
2
xref and test targets fail to build
Both are part of the all target, which is where the following output came from,
...
Compiled src/exometer_slot_slide.erl
Compiled src/exometer_report_snmp.erl
Compiled src/exometer_uniform.erl
ERL_LIBS=./deps rebar xref skip_deps=true
WARN: 'xref' command does not apply to directory /Users/juan/dev/src/exometer/src
==> exometer (xref)
erlc +'{parse_transform, lager_transform}' -pz deps/lager/ebin -I src -o examples/snmp_manager/ examples/snmp_manager/*.erl
ERL_LIBS=./examples rebar ct skip_deps=true
WARN: 'ct' command does not apply to directory /Users/juan/dev/src/exometer/src
==> exometer (ct)
ERROR: grep -e "TEST COMPLETE" -e "{error,make_failed}" /Users/juan/dev/src/exometer/logs/raw.log failed with error: 1 and output:
ERROR: ct failed while processing /Users/juan/dev/src/exometer: rebar_abort
make: *** [test] Error 1
Exit 2
Formating error in exometer_report_graphite
Hi,
I am trying exometer with graphite reporter and the reporter fail when trying to format the report:
CRASH REPORT Process exometer_report_graphite with 0 neighbours crashed with reason: bad argument in call to io:format(<0.1780.0>, "L = ~s~n", [["iris",".",[[database,select,failure],".","value"]," ","1"," ","1394816918",10]]) in exometer_report_graphite:exometer_report/5 line 78
I see it fails trying to convert the list of atom to string.
I use static definition of everything:
{exometer,
[{predefined,
[{[database, select, latency], histogram, []},
{[database, select, success], counter, []},
{[database, select, failure], counter, []}
]},
{report,
[{subscribers,
[{exometer_report_graphite, [database, select, latency], mean, 1000, true},
{exometer_report_graphite, [database, select, latency], min, 1000, true},
{exometer_report_graphite, [database, select, latency], max, 1000, true},
{exometer_report_graphite, [database, select, success], value, 1000, true},
{exometer_report_graphite, [database, select, failure], value, 1000, true}
]},
{reporters,
[{exometer_report_graphite,
[{connect_timeout, 5000},
{prefix, "iris"},
{host, "localhost"},
{port, 2003},
{api_key, ""}
]}
]}
]}
]},
exometer_report_snmp:create_bin/1 not defined for some embedded Built-in entries (spiral, uniform, etc).
So i have no idea how to use it with snmp reporter. Also probably it will be more consistence to use the same way for all entries EntryModule:create_bin/1 callback ?
exometer:get_value/2 wrong specs?
According to specs: datapoints can be only atom() or [atoms()].
However for example 99 is correct datapoint.
Should we replace 99 with '99' or extend specs? Maybe also introduce new type datapoint() ?
SNMP support for exometer_function
the current version of exometer_report_snmp.erl can not deal with exometer_function entries. It will run into
{error, {function_not_exported,
{snmp_bin, 1}}}
Is there a simple workaround besides specifying my own function entry module?
Histogram 'n' datapoint is inaccurate?
In reference to fecbad3, I have a small module which updates three histograms:
-module(histcount).
-export([main/0]).
main() ->
exometer:start(),
exometer:new([hist, ten], histogram, []),
do(10, fun() -> exometer:update([hist, ten], 1) end),
exometer:new([hist, one_hundred], histogram, []),
do(100, fun() -> exometer:update([hist, one_hundred], 1) end),
exometer:new([hist, one_thousand], histogram, []),
do(1000, fun() -> exometer:update([hist, one_thousand], 1) end),
timer:sleep(timer:seconds(10)),
io:format(user, "TEN :: ~p~n", [exometer:get_value([hist, ten])]),
io:format(user, "ONE HUNDRED :: ~p~n", [exometer:get_value([hist, one_hundred])]),
io:format(user, "ONE THOUSAND :: ~p~n", [exometer:get_value([hist, one_thousand])]),
exometer:stop().
%%%===================================================================
%%% Internal Functions
%%%===================================================================
do(0, _Fun) -> ok;
do(N, Fun) ->
Fun(),
do(N-1, Fun).When I run this I find the following:
> erl -pa ebin deps/*/ebin
Erlang R16B02 (erts-5.10.3) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V5.10.3 (abort with ^G)
1> c("/tmp/histcount").
{ok,histcount}
2> histcount:main().
10:09:30.770 [info] Application lager started on node nonode@nohost
10:09:30.821 [info] Starting reporters with []
10:09:30.822 [info] Application exometer started on node nonode@nohost
TEN :: {ok,[{n,5},
{mean,1},
{min,1},
{max,1},
{median,1},
{50,1},
{75,1},
{90,1},
{95,1},
{99,1},
{999,1}]}
ONE HUNDRED :: {ok,[{n,28},
{mean,1},
{min,1},
{max,1},
{median,1},
{50,1},
{75,1},
{90,1},
{95,1},
{99,1},
{999,1}]}
ONE THOUSAND :: {ok,[{n,253},
{mean,1},
{min,1},
{max,1},
{median,1},
{50,1},
{75,1},
{90,1},
{95,1},
{99,1},
{999,1}]}
10:09:40.843 [info] Application exometer exited with reason: stopped
ok
=INFO REPORT==== 23-Jun-2014::10:09:40 ===
application: lager
exited: stopped
type: permanent
I would have expected n to be 10, 100 and 1000 instead of 5, 28 and 253. Am I interpreting what n is incorrectly, perhaps?
options to histogram
Maybe the percentiles in exometer_histogram should be initialize as [] ? to make sure that percentiles are not generated unless wanted. Same with median ? Just and idea.
erl_first_files is not working correct
NOTE - this bug is after applying the pull request "Fix OTP 17". How this is treated before I have no idea.
I am not sure if this is a rebar problem or some problem in the build settings for exometer.
In rebar 2.2.0 the file EXOMETER-MIB is distributed to all ebin directories in dependencies when running "make". This behaviour disappears if the section erl_first_files is removed from the rebar.config.
In rebar 2.3.0 the same thing happens. But, apart from that, if you run make more than once, you get the following crash.
==> afunix (compile)
Compiling /Users/roland/Projects/wombat/git/exometer/deps/afunix/src/exometer_util.erl failed:
/Users/roland/Projects/wombat/git/exometer/deps/afunix/src/exometer_util.erl:none: no such file or directory
ERROR: compile failed while processing /Users/roland/Projects/wombat/git/exometer/deps/afunix: rebar_abort
make: *** [compile] Error 1
This seems to be, more or less, the same thing. It also disappears if you comment away the erl_first_files section.
exometer accumulates process memory until system exhaustion.
I've begun to notice that long-running exometer systems allocate progressively more memory. In particularly, the value reported by erlang:memory(processes) grows at approximately 5MB per hour in one proprietary system. The VM metrics can be seen here:
https://p.datadoghq.com/sb/dead22e6ae
In the above graph, large drops in allocation are a result of stopping and starting the exometer application.
No individual process has a particularly large heap allocation:
> recon:proc_count(total_heap_size, 5).
[{<0.199.0>,589951,
[exometer_report,
{current_function,{gen_server,loop,6}},
{initial_call,{proc_lib,init_p,5}}]},
{<0.7.0>,139267,
[application_controller,
{current_function,{gen_server,loop,6}},
{initial_call,{erlang,apply,2}}]},
{<0.229.0>,121535,
[{current_function,{exometer_histogram,loop,1}},
{initial_call,{proc_lib,init_p,3}}]},
{<0.32.0>,53194,
[code_server,
{current_function,{code_server,loop,1}},
{initial_call,{erlang,apply,2}}]},
{<0.228.0>,46798,
[{current_function,{exometer_histogram,loop,1}},
{initial_call,{proc_lib,init_p,3}}]}]
The total number of system processes is relatively stable, as the above graphs demonstrate.
I have not had a chance to isolate this observation into an open system, nor can I find an obvious smoking-gun. I believe exometer to be at fault as the system did not display this issue before it's inclusion and the allocated memory is freed when the exometer application is stopped via application:stop(exometer). Allocation growth resumes on application:start(exometer).
I'll be happy to share any data from the running system that might be helpful and will also attempt, tomorrow, to create an open system which demonstrates this problem.
build is failing on 17.3
I get the following wrror whil ebuilding exometer on latest Erlang release:
Compiling /Users/benoitc/work/exometer/deps/meck/src/meck_proc.erl failed:
/Users/benoitc/work/exometer/deps/meck/src/meck_proc.erl:51: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
/Users/benoitc/work/exometer/deps/meck/src/meck_proc.erl:408: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
/Users/benoitc/work/exometer/deps/meck/src/meck_proc.erl:445: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
/Users/benoitc/work/exometer/deps/meck/src/meck_proc.erl:446: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
/Users/benoitc/work/exometer/deps/meck/src/meck_proc.erl:476: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
/Users/benoitc/work/exometer/deps/meck/src/meck_proc.erl:477: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
/Users/benoitc/work/exometer/deps/meck/src/meck_proc.erl:482: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
/Users/benoitc/work/exometer/deps/meck/src/meck_proc.erl:483: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
/Users/benoitc/work/exometer/deps/meck/src/meck_proc.erl:488: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
/Users/benoitc/work/exometer/deps/meck/src/meck_proc.erl:489: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
ERROR: compile failed while processing /Users/benoitc/work/exometer/deps/meck: rebar_abort
This is an error in meck itself but wanted to also report it there.
histogram max initial value
The initial value for the histogram max datapoint is 0 which does not work when operating on negative numbers. Can it be changed to -infinity or something similar (with calculating min of negative values)?
https://github.com/Feuerlabs/exometer/blob/master/src/exometer_histogram.erl#L187
Support buffering of raw data-points.
After using the statsd reporter introduced in #2 for some time I've got some pain points around the mismatch between statsd's model and exometer's. To explain, statsd defines two partners in a transaction: client and server with the server being synonymous with the statsd daemon. statsd provides the following types, broken out by where the values are computed (via the StatsD Spec):
| server | client |
|---|---|
| counter | gauge |
| histogram | |
| meter |
That is, for instance, in the case of histograms, statsd assumes that the client will send a stream of numbers and statsd will do all the work to create histograms. Exometer assumes, however, that it will catch the stream of numbers and a report will selectively forward aggregate statistics. You can see in Issue #7 that I work around this by sending exometer histograms' mean, median and max over the wire as statsd gauges.
The statsd protocol is flawed in that it does not allow the client to specify when a metric occurred: every metric is assumed to have happened on receipt at the server. I mention this as it complicates one possible solution to the above mismatch: add a buffering of data-points per metric and allow reporters to read this metric in blocks, instead of per point. One a short enough interval, the inaccuracy might be acceptable.
I'm also finding that I could use a file-based reporter, something to write formatted lines into a log file. This sort of reporter would fit very well with a buffering exometer_entry.
Would there be any interest in having buffering--either by extending all existing entries or via a new entry--and a file-based reporter in exometer? I'd happily contribute them both.
Crash resetting uniform metric
Calling exometer:reset/1 on a uniform probe with some entries generates a case_clause.
Without knowledge of the internals, my first guess is that in https://github.com/Feuerlabs/exometer/blob/master/src/exometer.erl#L395 the module=?MODULE must be removed. The call to exometer_probe:reset/3 should ensure the appropriate callback module is called (exometer_uniform).
If I'm right, I can send a pull request with this change. At least with manual testing seems to work properly.
1> exometer:new([a], uniform).
ok
2> exometer:update([a], 3).
ok
3> exometer:reset([a]).
** exception error: no case clause matching [{exometer_entry,[a],
uniform,probe,exometer_uniform,enabled,0,undefined,
undefined,[],<0.192.0>}]
in function exometer:reset/1 (src/exometer.erl, line 377)
2 new report timers is started for each time a report is triggered
restart_subscr_timer in exometer_report is called twice for each {report, #key{} Interval} message, once in handle_info and once in do_report (called from handle_info). Which will results in a lots of reporting after a while!
Difficulty with "very dynamic" subscriptions.
I want to export data for a set of LevelDB instances. Each has a dynamic name (here a Riak partition number) and a possibly changing set of levels. Here's what I think should work, from the docs:
% LevelDB statistics per partition and level, e.g.
% [leveldb, files, Partition, Level]
% files: number of files
% We set a default for the "leveldb" tree of names, and subscribe using
% wildcards. These variables should then be created per-partition and per-level.
true = exometer_admin:set_default([leveldb, '_'], gauge, [{module, exometer}]),
ok = exometer_report:subscribe(exometer_report_graphite,
{find, [leveldb, files]},
[value], ?INTERVAL, [], true),
% ... further subscriptions for size, compaction time, compaction read/write size, etc.
Unfortunately, this doesn't seem to do the trick: while the entries are created, the subscription never happens:
> exometer_report:list_metrics([leveldb, files]).
{ok,[{[leveldb,files,
479555224749202520035584085735030365824602865664,0],
[value,ms_since_reset],
[],enabled},
{[leveldb,files,
479555224749202520035584085735030365824602865664,1],
[value,ms_since_reset],
[],enabled}]}
From various tinkering in the console with exometer_report:subscribe/6, it seems that the only way to make this work is to fully specify the entry, e.g.
[leveldb, files, 479555224749202520035584085735030365824602865664, 0]
Am I misunderstanding how this should work?
failing `make ci`
I can not run tests on my local machine (same results with my colleague).
mpm@t400s [06:20:15] [~/esl/exometer] [master *]
-> % make ci
[...]
erlc +'{parse_transform, lager_transform}' -pz deps/lager/ebin -I src -o examples/snmp_manager/ examples/snmp_manager/*.erl
ERL_LIBS=./examples rebar ct skip_deps=true
==> exometer (ct)
ERROR: sh(grep -e "TEST COMPLETE" -e "{error,make_failed}" /home/mpm/esl/exometer/logs/raw.log)
failed with return code 1 and the following output:
ERROR: ct failed while processing /home/mpm/esl/exometer: rebar_abort
Makefile:27: recipe for target 'test' failed
make: *** [test] Error 1mpm@t400s [06:24:14] [~/esl/exometer] [master *]
-> % cat logs/raw.log
--- Test run on 2014/11/21 18:24:12 ---
Converting "/home/mpm/esl/exometer/." to "/home/mpm/esl/exometer" and re-inserting with add_patha/1
Common Test v1.8.2 starting (cwd is /home/mpm/esl/exometer)
Common Test: Running make in test directories...
Including the following directories:
"/home/mpm/esl/exometer/include"
CWD set to: "/home/mpm/esl/exometer/logs/[email protected]_18.24.12"
TEST INFO: 1 test(s), 27 case(s) in 2 suite(s)
Updating /home/mpm/esl/exometer/logs/all_runs.html... done
Test run failed! Reason:
{'EXIT',{error,{enoent,"src"}}}
exometer is not xref clean
Running rebar xref skip_deps=true against 394b23c results in:
==> exometer (xref)
src/exometer_duration.erl:78: Warning: exometer_duration:get_value/3 calls undefined function exomete
r_probe:get_value/3 (Xref)
src/exometer_duration.erl:63: Warning: exometer_duration:probe_init/3 calls undefined function exomet
er_slot_slide:new/4 (Xref)
src/exometer_folsom.erl:52: Warning: exometer_folsom:delete/3 calls undefined function folsom_metrics
:delete_metric/1 (Xref)
src/exometer_folsom.erl:141: Warning: exometer_folsom:get_value_/3 calls undefined function folsom_me
trics:get_metric_value/1 (Xref)
src/exometer_folsom.erl:141: Warning: exometer_folsom:get_value_/3 calls undefined function folsom_me
trics_counter:get_value/1 (Xref)
src/exometer_folsom.erl:141: Warning: exometer_folsom:get_value_/3 calls undefined function folsom_me
trics_duration:get_value/1 (Xref)
src/exometer_folsom.erl:141: Warning: exometer_folsom:get_value_/3 calls undefined function folsom_me
trics_histogram:get_values/1 (Xref)
src/exometer_folsom.erl:141: Warning: exometer_folsom:get_value_/3 calls undefined function folsom_me
trics_spiral:get_values/1 (Xref)
src/exometer_folsom.erl:26: Warning: exometer_folsom:new/3 calls undefined function folsom_metrics:ne
w_counter/1 (Xref)
src/exometer_folsom.erl:26: Warning: exometer_folsom:new/3 calls undefined function folsom_metrics:ne
w_duration/1 (Xref)
src/exometer_folsom.erl:26: Warning: exometer_folsom:new/3 calls undefined function folsom_metrics:ne
w_histogram/3 (Xref)
src/exometer_folsom.erl:26: Warning: exometer_folsom:new/3 calls undefined function folsom_metrics:ne
w_meter/1 (Xref)
src/exometer_folsom.erl:26: Warning: exometer_folsom:new/3 calls undefined function folsom_metrics:ne
w_spiral/1 (Xref)
src/exometer_folsom.erl:60: Warning: exometer_folsom:reset/3 calls undefined function folsom_metrics_
counter:clear/1 (Xref)
src/exometer_folsom.erl:55: Warning: exometer_folsom:update/4 calls undefined function folsom_metrics
:notify_existing_metric/3 (Xref)
src/exometer_netlink.erl:59: Warning: exometer_netlink:probe_init/3 calls undefined function exometer
_slot_slide:new/4 (Xref)
src/exometer_netlink.erl:107: Warning: exometer_netlink:probe_sample/1 calls undefined function netli
nk_stat:get_value/1 (Xref)
src/count_example.erl:18: Warning: count_example:count_sample/3 is unused export (Xref)
src/count_example.erl:26: Warning: count_example:count_transform/2 is unused export (Xref)
src/exometer.erl:185: Warning: exometer:get_value/1 is unused export (Xref)
src/exometer.erl:573: Warning: exometer:get_values/1 is unused export (Xref)
src/exometer.erl:540: Warning: exometer:info/1 is unused export (Xref)
src/exometer.erl:86: Warning: exometer:new/2 is unused export (Xref)
src/exometer.erl:108: Warning: exometer:new/3 is unused export (Xref)
src/exometer.erl:122: Warning: exometer:re_register/3 is unused export (Xref)
src/exometer.erl:311: Warning: exometer:reset/1 is unused export (Xref)
src/exometer.erl:282: Warning: exometer:sample/1 is unused export (Xref)
src/exometer.erl:593: Warning: exometer:select/1 is unused export (Xref)
src/exometer.erl:605: Warning: exometer:select/2 is unused export (Xref)
src/exometer.erl:613: Warning: exometer:select_cont/1 is unused export (Xref)
src/exometer.erl:357: Warning: exometer:setopts/2 is unused export (Xref)
src/exometer.erl:79: Warning: exometer:start/0 is unused export (Xref)
src/exometer.erl:139: Warning: exometer:update/2 is unused export (Xref)
src/exometer_admin.erl:159: Warning: exometer_admin:code_change/3 is unused export (Xref)
src/exometer_admin.erl:163: Warning: exometer_admin:create_ets_tabs/0 is unused export (Xref)
src/exometer_admin.erl:221: Warning: exometer_admin:default_definition_/2 is unused export (Xref)
src/exometer_admin.erl:229: Warning: exometer_admin:exometer_default/2 is unused export (Xref)
...
Interpreting more complex function returns with exometer_function.
erlang:statistics(garbage_collection) has as its return {Number_of_GCs, Words_Reclaimed, 0}. It's unclear, referencing the exometer_function:new/3 documentation, how I might get exometer to follow the two important values in this tuple, modulo creating a utility function which returns a proplist like [{gc_total, Number_of_GCs}, {words_reclaimed, Words_Reclaimed}]. For tuple-valued returns, it would be very handy if I could assign datapoints to specific indexes of the tuple, something like:
exometer:new(
[gc],
{function,erlang,statistics,[garbage_collection],
[{gc_total, 1}, {words_reclaimed, 2}], [gc_total, words_reclaimed]})Test run failed! Reason: {'EXIT',{error,{enoent,"src"}}}
Erlang/OTP 17 [erts-6.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V6.1 (abort with ^G)
==> exometer (compile)
ERL_LIBS=./deps rebar xref skip_deps=true
==> exometer (xref)
erlc +'{parse_transform, lager_transform}' -pz deps/lager/ebin -I src -o examples/snmp_manager/ examples/snmp_manager/*.erl
ERL_LIBS=./examples rebar ct skip_deps=true
==> exometer (ct)
ERROR: sh(grep -e "TEST COMPLETE" -e "{error,make_failed}" /***/deps/exometer/logs/raw.log)
failed with return code 1 and the following output:
ERROR: ct failed while processing /***/deps/exometer: rebar_abort
make: *** [test] Error 1
pc10:exometer laurent$ more logs/raw.log
--- Test run on 2015/05/09 13:35:19 ---
Converting "/***/deps/exometer/." to "/***/deps/exometer" and re-inserting with add_patha/1
Common Test v1.8.1 starting (cwd is /***/deps/exometer)
Common Test: Running make in test directories...
Including the following directories: "/***/deps/exometer/include"
CWD set to: "/***/deps/exometer/logs/[email protected]_13.35.20"
TEST INFO: 1 test(s), 27 case(s) in 2 suite(s)
Updating /***/deps/exometer/logs/all_runs.html... done
Test run failed! Reason:
{'EXIT',{error,{enoent,"src"}}}
Document how to use exometer_report:subscribe() with "raw" processes
The README and the exometer_report documentation make various references to what happens when exometer_report is used with processes (as opposed to modules that implement the behaviour).
There is no documentation on how to use just processes (without the bahaviour). If there is, then I was unable to find it.
User error: snmpa_trap: Invalid value: wrongType
How to reproduce:
1> application:ensure_all_started(snmp).
{ok,[snmp]}
2> application:ensure_all_started(exometer).
{ok,[syntax_tools,compiler,goldrush,lager,exometer]}
3> exometer:new([foo, bar], histogram, [{snmp, [{min, 1000}, {mean, 1000}]}]).
ok
Got in shell:
18:17:16.226 [error] ** User error: snmpa_trap: Invalid value: wrongType
Oid: [1,3,6,1,4,1,40076,1,3,0]
Val: {value,0}
Type: {asn1_type,'OCTET STRING',0,64,[],true,'OCTET STRING',false,undefined}
Exometer configuration:
{exometer, [
{report, [
{reporters, [
{exometer_report_snmp, [
{mib_template, "priv/mibs/APP-METRICS-MIB.mib"},
{mib_dir, "tmp/exometer"}
]}
]}
]}
]}
rebar compile error
Try this steps:
$ git clone https://github.com/Feuerlabs/exometer.git
$ cd exometer
$ wget --no-check-certificate https://github.com/rebar/rebar/wiki/rebar
$ chmod u+x rebar
$ ./rebar get-deps
$ ./rebar compile
...
Compiled src/setup.erl
Uncaught error in rebar_core: {'EXIT',
{badarg,
[{re,run,[git,".*",[{capture,none}]],[]},
{rebar_deps,is_app_available,3,[]},
{rebar_deps,find_deps,3,[]},
{rebar_deps,preprocess,2,[]},
{rebar_core,acc_modules,5,[]},
{rebar_core,process_dir,4,[]},
{rebar_core,process_commands,1,[]},
{rebar,main,1,[]}]}}
ERROR: Command [compile] failed!
Actually the problem in git://github.com/uwiger/setup.git repo.
What for it included in deps?
Exometer dependency causes build failure.
Using erlhttp_bench as a motivating example, when I have a rebar executable in my path the exometer dependency builds fine:
> which rebar
/Users/blt/bin/rebar
> make clean && make
==> cowlib (clean)
==> ranch (clean)
==> cowboy (clean)
==> goldrush (clean)
==> lager (clean)
==> afunix (clean)
==> netlink (clean)
==> bear (clean)
==> meck (clean)
==> folsom (clean)
==> src (clean)
==> exo (clean)
==> edown (clean)
==> setup (clean)
==> src (clean)
==> exometer (clean)
==> core (clean)
==> http_api (clean)
==> erlhttp_bench (clean)
==> cowlib (compile)
Compiled src/cow_cookie.erl
Compiled src/cow_http.erl
Compiled src/cow_date.erl
Compiled src/cow_qs.erl
Compiled src/cow_spdy.erl
Compiled src/cow_mimetypes.erl
==> ranch (compile)
Compiled src/ranch_protocol.erl
Compiled src/ranch_transport.erl
Compiled src/ranch_acceptor.erl
Compiled src/ranch_acceptors_sup.erl
Compiled src/ranch_app.erl
Compiled src/ranch_listener_sup.erl
Compiled src/ranch.erl
Compiled src/ranch_conns_sup.erl
Compiled src/ranch_sup.erl
Compiled src/ranch_server.erl
Compiled src/ranch_ssl.erl
Compiled src/ranch_tcp.erl
==> cowboy (compile)
Compiled src/cowboy_http_handler.erl
Compiled src/cowboy_loop_handler.erl
Compiled src/cowboy_middleware.erl
Compiled src/cowboy_sub_protocol.erl
Compiled src/cowboy_websocket_handler.erl
Compiled src/cowboy_app.erl
Compiled src/cowboy.erl
Compiled src/cowboy_bstr.erl
Compiled src/cowboy_clock.erl
Compiled src/cowboy_handler.erl
Compiled src/cowboy_client.erl
Compiled src/cowboy_multipart.erl
Compiled src/cowboy_protocol.erl
Compiled src/cowboy_http.erl
Compiled src/cowboy_router.erl
Compiled src/cowboy_rest.erl
Compiled src/cowboy_spdy.erl
Compiled src/cowboy_sup.erl
Compiled src/cowboy_static.erl
Compiled src/cowboy_req.erl
Compiled src/cowboy_websocket.erl
==> goldrush (compile)
Compiled src/glc.erl
Compiled src/glc_ops.erl
Compiled src/glc_lib.erl
Compiled src/gr_context.erl
Compiled src/gre.erl
Compiled src/glc_code.erl
==> lager (compile)
Compiled src/lager_util.erl
Compiled src/lager_transform.erl
Compiled src/lager_app.erl
Compiled src/lager_backend_throttle.erl
Compiled src/lager.erl
Compiled src/lager_config.erl
Compiled src/lager_console_backend.erl
Compiled src/error_logger_lager_h.erl
Compiled src/lager_default_formatter.erl
Compiled src/lager_crash_log.erl
Compiled src/lager_handler_watcher.erl
Compiled src/lager_handler_watcher_sup.erl
Compiled src/lager_msg.erl
Compiled src/lager_file_backend.erl
Compiled src/lager_sup.erl
Compiled src/lager_format.erl
Compiled src/lager_stdlib.erl
Compiled src/lager_trunc_io.erl
==> afunix (compile)
Compiled src/afunix.erl
Compiling c_src/afunix_drv.c
Compiling c_src/packet_parser.c
c_src/packet_parser.c:348:21: warning: implicitly declaring library function 'memchr' with type 'void *(const void *, int, unsigned long)'
if ((ptr2 = memchr(ptr, '\n', n)) == NULL) {
^
c_src/packet_parser.c:348:21: note: please include the header <string.h> or explicitly provide a declaration for 'memchr'
c_src/packet_parser.c:420:13: warning: implicitly declaring library function 'memcmp' with type 'int (const void *, const void *, unsigned long)'
if (memcmp(hp->magic, CDR_MAGIC, 4) != 0)
^
c_src/packet_parser.c:420:13: note: please include the header <string.h> or explicitly provide a declaration for 'memcmp'
c_src/packet_parser.c:565:14: warning: implicitly declaring library function 'strncmp' with type 'int (const char *, const char *, unsigned long)'
(strncmp(ap->name, name, len) == 0))
^
c_src/packet_parser.c:565:14: note: please include the header <string.h> or explicitly provide a declaration for 'strncmp'
3 warnings generated.
==> netlink (compile)
Compiled src/netlink_app.erl
Compiled src/netlink_codec.erl
Compiled src/netlink_drv.erl
Compiled src/netlink.erl
Compiled src/netlink_stat.erl
Compiled src/netlink_sup.erl
Compiled src/netlink_gen.erl
Compiled src/netl_codec.erl
==> bear (compile)
Compiled src/bear.erl
==> meck (compile)
Compiled src/meck_cover.erl
Compiled src/meck_mod.erl
Compiled src/meck.erl
==> folsom (compile)
Compiled src/folsom.erl
Compiled src/folsom_ewma.erl
Compiled src/folsom_meter_timer_server.erl
Compiled src/folsom_metrics.erl
Compiled src/folsom_metrics_counter.erl
Compiled src/folsom_metrics_duration.erl
Compiled src/folsom_metrics_gauge.erl
Compiled src/folsom_ets.erl
Compiled src/folsom_metrics_histogram_ets.erl
Compiled src/folsom_metrics_histogram.erl
Compiled src/folsom_metrics_history.erl
Compiled src/folsom_metrics_meter.erl
Compiled src/folsom_metrics_meter_reader.erl
Compiled src/folsom_metrics_spiral.erl
Compiled src/folsom_sample.erl
Compiled src/folsom_sample_none.erl
Compiled src/folsom_sample_exdec.erl
Compiled src/folsom_sample_slide_server.erl
Compiled src/folsom_sample_slide.erl
Compiled src/folsom_sample_slide_sup.erl
Compiled src/folsom_sample_slide_sorted.erl
Compiled src/folsom_sample_slide_uniform.erl
Compiled src/folsom_sample_uniform.erl
Compiled src/folsom_utils.erl
Compiled src/folsom_sup.erl
Compiled src/folsom_vm_metrics.erl
==> src (compile)
==> exo (compile)
Compiled src/exo_socket_server.erl
Compiled src/exo_app.erl
Compiled src/exo_http_server.erl
Compiled src/exo_json.erl
Compiled src/exo_socket_cache.erl
Compiled src/exo_http.erl
Compiled src/exo_socket.erl
Compiled src/exo_sup.erl
Compiled src/exo_socket_session.erl
Compiled src/exo_url.erl
/Users/blt/projects/com/adroll/erlhttp_bench/deps/exo/src/exo_ssh.erl:18: Warning: undefined callback function code_change/3 (behaviour 'ssh_channel')
/Users/blt/projects/com/adroll/erlhttp_bench/deps/exo/src/exo_ssh.erl:18: Warning: undefined callback function handle_call/3 (behaviour 'ssh_channel')
/Users/blt/projects/com/adroll/erlhttp_bench/deps/exo/src/exo_ssh.erl:18: Warning: undefined callback function handle_cast/2 (behaviour 'ssh_channel')
Compiled src/exo_ssh.erl
==> edown (compile)
Compiled src/edown_lib.erl
Compiled src/edown_make.erl
Compiled src/edown_doclet.erl
Compiled src/edown_xmerl.erl
Compiled src/edown_layout.erl
==> setup (compile)
Compiled src/setup_lib.erl
Compiled src/setup_gen.erl
Compiled src/setup.erl
==> setup (escriptize)
==> src (compile)
==> exometer (compile)
Compiled mibs/EXOMETER-MIB.mib
Compiled src/exometer_igor.erl
Compiled src/exometer_util.erl
Compiled src/exometer_entry.erl
Compiled src/exometer_probe.erl
Compiled src/exometer_report.erl
Compiled src/exometer_app.erl
Compiled src/exometer_cache.erl
Compiled src/exometer_cpu.erl
Compiled src/exometer_admin.erl
Compiled src/exometer_duration.erl
Compiled src/exometer_folsom.erl
Compiled src/exometer_function.erl
Compiled src/exometer_netlink.erl
Compiled src/exometer_proc.erl
Compiled src/exometer_histogram.erl
Compiled src/exometer_report_graphite.erl
Compiled src/exometer_report_collectd.erl
Compiled src/exometer.erl
Compiled src/exometer_report_riak.erl
Compiled src/exometer_report_statsd.erl
Compiled src/exometer_slide.erl
Compiled src/exometer_report_snmp.erl
Compiled src/exometer_report_tty.erl
Compiled src/exometer_sup.erl
Compiled src/exometer_spiral.erl
Compiled src/exometer_uniform.erl
Compiled src/exometer_slot_slide.erl
==> core (compile)
Compiled src/core.erl
Compiled src/core_app.erl
Compiled src/core_sup.erl
==> http_api (compile)
Compiled src/http_api.erl
Compiled src/http_api_app.erl
Compiled src/http_api_sup.erl
Compiled src/http_api_blob_handler.erl
==> erlhttp_bench (compile)
When I do not, the setup dependency fails.
> which rebar
rebar not found
> make clean && make
==> cowlib (clean)
==> ranch (clean)
==> cowboy (clean)
==> goldrush (clean)
==> lager (clean)
==> afunix (clean)
==> netlink (clean)
==> bear (clean)
==> meck (clean)
==> folsom (clean)
==> src (clean)
==> exo (clean)
==> edown (clean)
==> setup (clean)
==> src (clean)
==> exometer (clean)
==> core (clean)
==> http_api (clean)
==> erlhttp_bench (clean)
==> cowlib (compile)
Compiled src/cow_cookie.erl
Compiled src/cow_http.erl
Compiled src/cow_date.erl
Compiled src/cow_qs.erl
Compiled src/cow_spdy.erl
Compiled src/cow_mimetypes.erl
==> ranch (compile)
Compiled src/ranch_protocol.erl
Compiled src/ranch_transport.erl
Compiled src/ranch_acceptor.erl
Compiled src/ranch_acceptors_sup.erl
Compiled src/ranch_app.erl
Compiled src/ranch_listener_sup.erl
Compiled src/ranch.erl
Compiled src/ranch_conns_sup.erl
Compiled src/ranch_server.erl
Compiled src/ranch_sup.erl
Compiled src/ranch_ssl.erl
Compiled src/ranch_tcp.erl
==> cowboy (compile)
Compiled src/cowboy_http_handler.erl
Compiled src/cowboy_loop_handler.erl
Compiled src/cowboy_middleware.erl
Compiled src/cowboy_sub_protocol.erl
Compiled src/cowboy_websocket_handler.erl
Compiled src/cowboy_app.erl
Compiled src/cowboy.erl
Compiled src/cowboy_bstr.erl
Compiled src/cowboy_clock.erl
Compiled src/cowboy_handler.erl
Compiled src/cowboy_client.erl
Compiled src/cowboy_multipart.erl
Compiled src/cowboy_protocol.erl
Compiled src/cowboy_http.erl
Compiled src/cowboy_router.erl
Compiled src/cowboy_rest.erl
Compiled src/cowboy_spdy.erl
Compiled src/cowboy_static.erl
Compiled src/cowboy_sup.erl
Compiled src/cowboy_req.erl
Compiled src/cowboy_websocket.erl
==> goldrush (compile)
Compiled src/glc.erl
Compiled src/glc_lib.erl
Compiled src/glc_ops.erl
Compiled src/gr_context.erl
Compiled src/gre.erl
Compiled src/glc_code.erl
==> lager (compile)
Compiled src/lager_util.erl
Compiled src/lager_transform.erl
Compiled src/lager_app.erl
Compiled src/lager_backend_throttle.erl
Compiled src/lager_config.erl
Compiled src/lager.erl
Compiled src/lager_console_backend.erl
Compiled src/error_logger_lager_h.erl
Compiled src/lager_crash_log.erl
Compiled src/lager_default_formatter.erl
Compiled src/lager_handler_watcher.erl
Compiled src/lager_handler_watcher_sup.erl
Compiled src/lager_msg.erl
Compiled src/lager_format.erl
Compiled src/lager_file_backend.erl
Compiled src/lager_sup.erl
Compiled src/lager_stdlib.erl
Compiled src/lager_trunc_io.erl
==> afunix (compile)
Compiled src/afunix.erl
Compiling c_src/afunix_drv.c
Compiling c_src/packet_parser.c
c_src/packet_parser.c:348:21: warning: implicitly declaring library function 'memchr' with type 'void *(const void *, int, unsigned long)'
if ((ptr2 = memchr(ptr, '\n', n)) == NULL) {
^
c_src/packet_parser.c:348:21: note: please include the header <string.h> or explicitly provide a declaration for 'memchr'
c_src/packet_parser.c:420:13: warning: implicitly declaring library function 'memcmp' with type 'int (const void *, const void *, unsigned long)'
if (memcmp(hp->magic, CDR_MAGIC, 4) != 0)
^
c_src/packet_parser.c:420:13: note: please include the header <string.h> or explicitly provide a declaration for 'memcmp'
c_src/packet_parser.c:565:14: warning: implicitly declaring library function 'strncmp' with type 'int (const char *, const char *, unsigned long)'
(strncmp(ap->name, name, len) == 0))
^
c_src/packet_parser.c:565:14: note: please include the header <string.h> or explicitly provide a declaration for 'strncmp'
3 warnings generated.
==> netlink (compile)
Compiled src/netlink_app.erl
Compiled src/netlink_codec.erl
Compiled src/netlink_drv.erl
Compiled src/netlink.erl
Compiled src/netlink_gen.erl
Compiled src/netlink_sup.erl
Compiled src/netlink_stat.erl
Compiled src/netl_codec.erl
==> bear (compile)
Compiled src/bear.erl
==> meck (compile)
Compiled src/meck_mod.erl
Compiled src/meck_cover.erl
Compiled src/meck.erl
==> folsom (compile)
Compiled src/folsom.erl
Compiled src/folsom_ewma.erl
Compiled src/folsom_meter_timer_server.erl
Compiled src/folsom_metrics.erl
Compiled src/folsom_metrics_counter.erl
Compiled src/folsom_metrics_duration.erl
Compiled src/folsom_metrics_gauge.erl
Compiled src/folsom_ets.erl
Compiled src/folsom_metrics_histogram_ets.erl
Compiled src/folsom_metrics_histogram.erl
Compiled src/folsom_metrics_history.erl
Compiled src/folsom_metrics_meter.erl
Compiled src/folsom_metrics_meter_reader.erl
Compiled src/folsom_metrics_spiral.erl
Compiled src/folsom_sample.erl
Compiled src/folsom_sample_none.erl
Compiled src/folsom_sample_exdec.erl
Compiled src/folsom_sample_slide_server.erl
Compiled src/folsom_sample_slide.erl
Compiled src/folsom_sample_slide_sup.erl
Compiled src/folsom_sample_slide_sorted.erl
Compiled src/folsom_sample_slide_uniform.erl
Compiled src/folsom_sample_uniform.erl
Compiled src/folsom_utils.erl
Compiled src/folsom_sup.erl
Compiled src/folsom_vm_metrics.erl
==> src (compile)
==> exo (compile)
Compiled src/exo_socket_server.erl
Compiled src/exo_app.erl
Compiled src/exo_http_server.erl
Compiled src/exo_json.erl
Compiled src/exo_socket_cache.erl
Compiled src/exo_http.erl
Compiled src/exo_socket.erl
Compiled src/exo_socket_session.erl
Compiled src/exo_sup.erl
Compiled src/exo_url.erl
/Users/blt/projects/com/adroll/erlhttp_bench/deps/exo/src/exo_ssh.erl:18: Warning: undefined callback function code_change/3 (behaviour 'ssh_channel')
/Users/blt/projects/com/adroll/erlhttp_bench/deps/exo/src/exo_ssh.erl:18: Warning: undefined callback function handle_call/3 (behaviour 'ssh_channel')
/Users/blt/projects/com/adroll/erlhttp_bench/deps/exo/src/exo_ssh.erl:18: Warning: undefined callback function handle_cast/2 (behaviour 'ssh_channel')
Compiled src/exo_ssh.erl
==> edown (compile)
Compiled src/edown_lib.erl
Compiled src/edown_make.erl
Compiled src/edown_doclet.erl
Compiled src/edown_xmerl.erl
Compiled src/edown_layout.erl
==> setup (compile)
Compiled src/setup_lib.erl
Compiled src/setup_gen.erl
Compiled src/setup.erl
sh: line 0: exec: rebar: not found
ERROR: Command [compile] failed!
make: *** [compile] Error 1
Reporter's erlang:send_after/3 fails under heavy load
When performing some heavier computations with many processes, I noticed that the reporter skips some of the logs. For example after running a program for 30s and logging data every 1s, the reporter ends up logging only 15 of them or sometimes even less.
In my opinion using send_after/3 might be the issue. Reporter uses it to indicate the moment when it should activate, fetch appropriate data and "report it". Every time he receives the message {report, Key, Interval} he performs some action and then calls send_after/3 once again. However when there are many other processes waiting to be scheduled, a significant amount of time may pass between the event of receiving the message, and the reporter being activated by the scheduler to process it and call the function once again. Therefore even though the interval is officially 1s, the period between the events of calling send_after/3 might be much more than 1s. This offset accumulates over time resulting in a significant deviation in the logs.
This issue might be solved by substituting send_after/3 with send_interval/3, which doesn't require resetting the timer every 1s (or whatever the interval is).
function_clause error when creating pre-defined function entry?
Using 279b61c I'm able to create a new function entry dynamically easily enough:
(bodhi@walden)1> exometer:new([mem], {function,erlang,memory,['$dp'],proplist,[total]}).
ok
but when I load it through the predefined entries:
[
{exometer,
[
{predefined, [
{[mem], {function, erlang, memory, ['$dp'], proplist, [total]}}
]},
...
]
},
I get a crash on boot:
Erlang R16B02 (erts-5.10.3) [source] [64-bit] [smp:8:8] [async-threads:256] [hipe] [kernel-poll:true]
Eshell V5.10.3 (abort with ^G)
(bodhi@walden)1>
{"init terminating in do_boot",{{badmatch,{error,{bad_return_value,{{exometer_app,start_phase,[preset_defaults,normal,[]]},{'EXIT',{function_clause,[{exometer_admin,'-do_load_predef/2-fun-2-',[{[mem],{function,erlang,memory,['$dp'],proplist,[total]}}],[{file,"src/exometer_admin.erl"},{line,141}]},{lists,foreach,2,[{file,"lists.erl"},{line,1323}]},{exometer_admin,load_predefined,0,[{file,"src/exometer_admin.erl"},{line,97}]},{application_starter,run_the_phase,4,[{file,"application_starter.erl"},{line,98}]},{application_starter,start_apps,3,[{file,"application_starter.erl"},{line,58}]},{application_starter,start,3,[{file,"application_starter.erl"},{line,44}]},{application_master,start_the_app,5,[{file,"application_master.erl"},{line,308}]},{application_master,start_it_new,7,[{file,"application_master.erl"},{line,291}]}]}}}}}},[{bodhi,start,0,[{file,"src/bodhi.erl"},{line,40}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
Crash dump was written to: erl_crash.dump
init terminating in do_boot ()
Surely I'm doing something trivially silly here, but I cannot see it.
exo application in rebar.deps is not used
The exo application in rebar.config is not used. Can it be removed?
The exo applications points to Feuerlabs/folsom, "1.1". But this is a non-existant branch. This leads to all sorts of rebar deps problems.
Unexpected behavior with fast_counter
Hi,
I'm testing out fast_counter metric type with exometer, and experiencing unexpected behavior:
> erl -pa ebin -pa deps/*/ebin -s exopose_app
Erlang/OTP 17 [erts-6.2.1] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Eshell V6.2.1 (abort with ^G)
1> 16:02:07.712 [info] Application lager started on node nonode@nohost
16:02:07.731 [info] Application exometer started on node nonode@nohost
16:02:07.735 [info] Application exopose started on node nonode@nohost
1> exopose:i().
[...]
2> dbg:tracer(), dbg:p(all,c), dbg:tp(exopose, i, x).
{ok,[{matched,nonode@nohost,1},{saved,x}]}
3> exometer:new([test,fastcounter], {function, exopose, i, [], fast_counter, value}, []).
ok
4> (<0.77.0>) call exopose:i()
(<0.77.0>) exception_from {exopose,i,0} {exit,{timeout,{gen_server,call,[exopose,info]}}}I've omitted what exopose:i/0 returns for simplicity, but my point is that, after creating fast_counter providing exopose and i as module and function, erlang:trace_pattern/3 leaves the call non-responsive. Am I misusing the feature or there's a potential issue?
As a side note, exometer:create_entry/1 returns ok if I launch the dbg, but I can't see anything printed out if I trace down exometer:set_call_count/3, i.e: https://github.com/Feuerlabs/exometer/blob/master/src/exometer.erl#L1070
Thanks,
counter not incrementing?
I use 1.2 tag and this code (exometer is started with app):
exometer:new([the, test], counter).
ok
exometer:update([the, test], 4).
ok
exometer:info(the, test], value).
[]
Also, without update result is the same:
exometer:new([bla, bla], counter).
ok
exometer:info([bla, bla]).
[{name,[bla,bla]},
{type,counter},
{behaviour,undefined},
{module,exometer},
{status,enabled},
{cache,0},
{value,[]},
{timestamp,192799789183},
{options,[]},
{ref,undefined},
{datapoints,[value,ms_since_reset]}]
It is strange, as I can see in exometer:create_entry/1 default 0 for counters.
Exometer fails to compile in Elixir 1.1
It's due to exometer_core requiring a file in another project; See the discussion here:
elixir-lang/elixir#4044
BTW, we really like exometer; it powers all of our Elixir metrics. Thanks for your work!
exometer crashes when creating dynamic subscriptions for histogram integer attributes
This is in reference to commit eaf0fb8:
1> application:start(lager).
ok
2> 16:51:02.720 [info] Application lager started on node nonode@nohost
2> application:start(exometer).
16:51:07.759 [info] Starting reporter with []
16:51:07.764 [info] Application exometer started on node nonode@nohost
ok
3> exometer:new([hist], histogram).
ok
4> exometer:get_value([hist]).
{ok,[{n,0},
{mean,0},
{min,0},
{max,0},
{median,0},
{50,0},
{75,0},
{90,0},
{95,0},
{99,0},
{999,0}]}
5> exometer_report:add_reporter(exometer_report_tty, []).
ok
6> 16:55:02.745 [info] exometer_report_tty([]): Starting
6> exometer_report:subscribe(exometer_report_tty, [hist], mean, 10000, []).
ok
7> exometer_report_tty: hist_mean 1396396538:0 (unknown)
7> exometer_report:subscribe(exometer_report_tty, [hist], 999, 10000, []).
16:55:44.753 [error] gen_server exometer_report terminated with reason: no function clause matching exometer_report:first_datapoint(999) line 709
16:55:44.754 [error] CRASH REPORT Process exometer_report with 0 neighbours exited with reason: no function clause matching exometer_report:first_datapoint(999) line 709 in gen_server:terminate/6 line 744
16:55:44.754 [error] Supervisor exometer_sup had child exometer_report started with exometer_report:start_link() at <0.59.0> exit with reason no function clause matching exometer_report:first_datapoint(999) line 709 in context child_terminated
16:55:44.754 [info] Starting reporter with []
** exception exit: {{function_clause,[{exometer_report,first_datapoint,
[999],
[{file,"src/exometer_report.erl"},{line,709}]},
{exometer_report,handle_call,3,
[{file,"src/exometer_report.erl"},{line,448}]},
{gen_server,handle_msg,5,
[{file,"gen_server.erl"},{line,585}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]},
{gen_server,call,
[exometer_report,
{subscribe,{key,exometer_report_tty,[hist],999,false,[]},
10000}]}}
in function gen_server:call/2 (gen_server.erl, line 180)
I believe the problem is ultimately here: https://github.com/Feuerlabs/exometer/blob/master/src/exometer_report.erl#L709 There is no clause for an integer value while histogram defines its percentiles--here: https://github.com/Feuerlabs/exometer/blob/master/src/exometer_histogram.erl#L51--as integers.
Process exometer_report_graphite crashed with reason: call to undefined function lager:info
With 17.3 I have a problem I don't have in 17.1 :
Erlang/OTP 17 [erts-6.2] [source] [async-threads:10] [hipe] [kernel-poll:false]
Setup running ...
Directories verified. Res = {[ok],[]}
Setup finished processing hooks ...
23:15:56.740 [info] Application lager started on node '[email protected]'
23:15:56.751 [info] Starting reporters with []
23:15:56.751 [info] Application exometer_core started on node '[email protected]'
23:15:56.753 [info] Application exometer started on node '[email protected]'
23:15:56.759 [error] CRASH REPORT Process exometer_report_graphite with 0 neighbours crashed with reason: call to undefined function lager:info("Exometer Graphite Reporter; Opts: ~p~n", [[{connect_timeout,5000},{prefix,"guanzhi.debianraw"},{host,"***.com"},{port,...},...]])
Eshell V6.2 (abort with ^G)
([email protected])1> 23:15:56.827 [error] CRASH REPORT Process exometer_report_graphite with 0 neighbours crashed with reason: call to undefined function lager:info("Exometer Graphite Reporter; Opts: ~p~n", [[{connect_timeout,5000},{prefix,"guanzhi.xxx"},{host,"****.com"},{port,...},...]])
23:15:56.827 [error] CRASH REPORT Process exometer_report_graphite with 0 neighbours crashed with reason: call to undefined function lager:info("Exometer Graphite Reporter; Opts: ~p~n", [[{connect_timeout,5000},{prefix,"guanzhi.xxx"},{host,"****.com"},{port,...},...]])
23:15:56.827 [error] CRASH REPORT Process <0.96.0> with 0 neighbours exited with reason: no match of right hand value unknown_reporter in guanzhi_exo:init/1 line 49 in gen_server:init_it/6 line 330
As we can see in this log, lager is well started on node, but lager:info is undefined , problem is here in exometer_report_graphite.erl line 53, ?info is defined in log.hrl.
%% Probe callbacks
exometer_init(Opts) ->
?info("Exometer Graphite Reporter; Opts: ~p~n", [Opts]),
Can we make afunix an optional include?
Can we make afunix an optional include for exometer?
And if so, how can we get that dependency in the rebar.config?
See also: zotonic/zotonic#797
`exometer:sample` always fails.
A typo in exometer_probe.erl causes all sample calls to probes to timeout.
https://github.com/Feuerlabs/exometer/blob/master/src/exometer_probe.erl#L471 should be exometer_proc:cast(Pid, sample).
(I could make a pr if you like, but this was such a small fix)
Support erlang 18
Exometer depends on netlink, which will not compile due to using pre-18 dict semantics. Netlink emits the following errors under erlang 17:
Compiled src/netlink_stat.erl
src/netlink_gen.erl:48: Warning: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
src/netlink_gen.erl:49: Warning: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
src/netlink_gen.erl:50: Warning: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
src/netlink_gen.erl:51: Warning: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
src/netlink.erl:64: Warning: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
src/netlink.erl:72: Warning: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2
There has been a PR sitting in netlink's queue since July of 2015 which fixes this behavior. Can you please merge it?
exometer function type= proplist calls once per element in the list
I have a function that returns a proplist
[{key1, val1}, {key2, val2} ... ]
and a monitoring configuration file that looks similar to this.
[{predefined,
[{[mynode, myapp, state],
{function, myapp, state, [], proplist, [ke1, key2, key3, key4, key5]},
[]
}
...
{report,
[{subscribers,
[{exometer_report_statsd, [mynode, myapp, state], key1, 10000, true},
{exometer_report_statsd, [mynode, myapp, state], key2, 10000, true},
{exometer_report_statsd, [mynode, myapp, state], key3, 10000, true},
{exometer_report_statsd, [mynode, myapp, state], key4, 10000, true},
{exometer_report_statsd, [mynode, myapp, state], key5, 10000, true}
]}
...
{reporters,
[{ exometer_report_statsd,
{[mynode, myapp, state, key1], gauge},
{[mynode, myapp, state, key2], gauge},
{[mynode, myapp, state, key3], gauge},
{[mynode, myapp, state, key4], gauge},
{[mynode, myapp, state, key5], gauge},
...
I noticed that every time interval, the proplist is called 5 times, one time per key.
Is there any way to force that it is only called once?
Dynamic parts in the metric name.
Hi,
what would be the best way to support metrics with dynamic names ?
For example having metrics that contains the hostname and node name; something like:
prefix.HOSTNAME.vm.NODENAME.memory.binary
Crash when retrieving various attributes of histogram metric type.
Using exometer 5f10cc1
(dyno_dev@walden)10> exometer:new([a,histogram], histogram).
ok
(dyno_dev@walden)11> exometer:info([a,histogram]).
[{name,[a,histogram]},
{type,histogram},
{behaviour,probe},
{module,exometer_histogram},
{status,enabled},
{cache,0},
{value,[]},
{timestamp,undefined},
{options,[]},
{ref,<0.176.0>},
{datapoints,[n,mean,min,max,median,50,75,90,95,99,999]}]
(dyno_dev@walden)12> exometer:get_value([a,histogram], median).
{ok,[{median,0}]}
(dyno_dev@walden)13> exometer:get_value([a,histogram], 99).
** exception error: no case clause matching
{99,
{exometer_entry,[a,histogram],
histogram,probe,exometer_histogram,
enabled,0,undefined,undefined,[],
<0.176.0>}}
in function exometer:get_value_/2 (src/exometer.erl, line 537)
in call from exometer:get_value/2 (src/exometer.erl, line 199)
(dyno_dev@walden)14> exometer:get_value([a,histogram], 50).
** exception error: no case clause matching
{50,
{exometer_entry,[a,histogram],
histogram,probe,exometer_histogram,
enabled,0,undefined,undefined,[],
<0.176.0>}}
in function exometer:get_value_/2 (src/exometer.erl, line 537)
in call from exometer:get_value/2 (src/exometer.erl, line 199)
(dyno_dev@walden)15>It seems that mean, min, max and median work as you would expect.
Static exometer_report_tty subscription is seemingly canceled on dynamic metric creation
Running from the root of exometer at commit 394b23c I find that exometer_report_tty stops printing anything after I dynamically create a metric:
> erl -pa deps/*/ebin ebin -config /tmp/sys.config -s lager -s crypto -s exometer
Erlang R16B02 (erts-5.10.3) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Eshell V5.10.3 (abort with ^G)
1> 18:49:52.560 [info] Application lager started on node nonode@nohost
18:49:52.578 [info] Application crypto started on node nonode@nohost
18:49:52.584 [info] Starting reporter with [{subscribers,[{exometer_report_tty,[erlang,memory],total,5000,true},{exometer_report_tty,[erlang,memory],processes,5000,true}]}]
18:49:52.584 [info] Subscribe_(Intv(5000), self(<0.64.0>))
18:49:52.584 [info] Subscribe_(Intv(5000), self(<0.64.0>))
18:49:52.587 [info] Application exometer started on node nonode@nohost
18:49:57.586 [info] Metric([erlang,memory]) Datapoint(total) not found. Will try again in 5000 msec
18:49:57.586 [info] Metric([erlang,memory]) Datapoint(processes) not found. Will try again in 5000 msec
18:50:02.587 [info] Metric([erlang,memory]) Datapoint(processes) not found. Will try again in 5000 msec
18:50:02.587 [info] Metric([erlang,memory]) Datapoint(total) not found. Will try again in 5000 msec
1> exometer:new([erlang,memory], {function,erlang,memory,['$dp'],value,[total,processes]}).
ok
2> os:timestamp().
{1389,667808,534175}
3> exometer:get_value([erlang,memory], processes).
{ok,[{processes,5249120}]}
4> os:timestamp().
{1389,667848,516144}
The referenced sys.config is:
[
{exometer, [
{ report,
[
{ subscribers,
[
{ exometer_report_tty, [erlang, memory], total, 5000, true }
, { exometer_report_tty, [erlang, memory], processes, 5000, true }
]}
]},
{ reporters,
[
{ exometer_report_tty, [] }
]}
]},
{lager, [
{handler, [
{lager_console_backend, debug}
]}
]}
].Debug log is cluttered by collectd entries.
When enabling debug logging, collectd reporter write 3 lines every time it publishes a datapoint. This is cluttering the log and making it unusable for debugging anything else.
Is there a way to selectively remove the logging from a module with lager ?
If not, how would you want it to be changed ? Some conditional compilation flag ?
Thank you.
Problem with matching nested tuples
When defining the following metric no datapoint is matched.
> exometer:new([io], {function, erlang, statistics, [io], match, {{'_', input}, {'_', output}} }, []).
> exometer:get_values([io]).
[{[io],[]}]While the output should obviously match the specification
> erlang:statistics(io)
{{input, 15062587},{output,5819150}}
there is exist way how-to retrieve exometers entries values with snmpwalk instead notifications traps?
Sorry for noob question, but could you please advise me some direction?
Should i modify my mib template or configure snmp agent some special way?
At the current moment snmpwalk results looks pretty basic and not contain any my custom metrics:
snmpwalk -m /tmp/exometer/EXOMETER-METRICS-MIB.mib -v 2c -c public localhost:4000
SNMPv2-SMI::mib-2.1.1.0 = STRING: "Exometer SNMP agent"
SNMPv2-SMI::mib-2.1.2.0 = OID: joint-iso-ccitt.46.1.4.1.193.19
SNMPv2-SMI::mib-2.1.3.0 = Timeticks: (4763) 0:00:47.63
SNMPv2-SMI::mib-2.1.4.0 = STRING: "[email protected]"
SNMPv2-SMI::mib-2.1.5.0 = STRING: "exometer agent"
SNMPv2-SMI::mib-2.1.6.0 = STRING: "erlang"
SNMPv2-SMI::mib-2.1.7.0 = INTEGER: 72
SNMPv2-SMI::mib-2.1.8.0 = Timeticks: (0) 0:00:00.00
SNMPv2-SMI::mib-2.11.1.0 = Counter32: 26
SNMPv2-SMI::mib-2.11.3.0 = Counter32: 0
SNMPv2-SMI::mib-2.11.4.0 = Counter32: 0
SNMPv2-SMI::mib-2.11.5.0 = Counter32: 0
SNMPv2-SMI::mib-2.11.6.0 = Counter32: 0
SNMPv2-SMI::mib-2.11.30.0 = INTEGER: 1
SNMPv2-SMI::mib-2.11.31.0 = Counter32: 0
SNMPv2-SMI::mib-2.11.32.0 = Counter32: 0
Dynamic modification of function match datapoints
I've got a use-case where it would be remarkably handy to update a function match's datapoints at runtime. For example's sake, let's say I have the following function:
user_metrics:times_seen/1
which takes a user ID--which we'll assume is from a large but finite pool--and returns an integer. Even though the possible users are finite I don't know which will be "active" at system startup and those that become "inactive" may have any materials in the system about them destroyed.
It would be remarkably handy to set up the system with the following static configuration:
{[users, times_seen],
{function, user_metrics, times_seen, ['$dp'], value, []},
[]
}and then on notification of a new user:
handle_event({user_join, UserId}, State) ->
Datapoints = exometer:info([users, time_seen], datapoints),
ok = exometer:update_datapoints([users, time_seen], [UserID | Datapoints]),
{ok, State}.and on deletion of a user:
handle_event({user_remove, UserId}, State) ->
NewDatapoints = [U || U <- exometer:info([users, time_seen], datapoints),
U =/= UserId],
ok = exometer:update_datapoints([users, time_seen], DataPoints),
{ok, State}.the primary function of interest being a new exometer:update_datapoints/2 which replaces the stored values. (Presumably real code would be destroying subscriptions at the same time.)
Does something like this makes sense for exometer?
exometer_report_snmp oid_conflict error
Hi! I faced with this error when trying to create very basic example of using of exometer with snmp integration.
(agent@cabal)2> 17:20:30.877 [info] MIB /tmp/exometer/EXOMETER-METRICS-MIB unloaded
17:20:30.877 [error] Error {error,{'load aborted at',"/tmp/exometer/EXOMETER-METRICS-MIB",oid_conflict}} when loading MIB /tmp/exometer/EXOMETER-METRICS-MIB
17:20:30.877 [error] CRASH REPORT Process exometer_report_snmp with 0 neighbours crashed with reason: no match of right hand value {error,{'load aborted at',"/tmp/exometer/EXOMETER-METRICS-MIB",oid_conflict}} in exometer_report_snmp:enable_metric/2 line 219
17:20:30.877 [info] exometer_report_snmp([{mib_template,"priv/EXOMETER-METRICS-MIB.mib"},{mib_dir,"/tmp/exometer"}]): Starting
17:20:30.879 [info] MIB /tmp/exometer/EXOMETER-METRICS-MIB loaded
17:20:30.886 [info] MIB /tmp/exometer/EXOMETER-METRICS-MIB unloaded
17:20:30.887 [error] Error {error,{'load aborted at',"/tmp/exometer/EXOMETER-METRICS-MIB",oid_conflict}} when loading MIB /tmp/exometer/EXOMETER-METRICS-MIB
17:20:30.887 [error] CRASH REPORT Process exometer_report_snmp with 0 neighbours crashed with reason: no match of right hand value {error,{'load aborted at',"/tmp/exometer/EXOMETER-METRICS-MIB",oid_conflict}} in exometer_report_snmp:enable_metric/2 line 219
17:20:30.887 [info] exometer_report_snmp([{mib_template,"priv/EXOMETER-METRICS-MIB.mib"},{mib_dir,"/tmp/exometer"}]): Starting
17:20:30.889 [info] MIB /tmp/exometer/EXOMETER-METRICS-MIB loaded
17:20:30.895 [info] MIB /tmp/exometer/EXOMETER-METRICS-MIB unloaded
17:20:30.895 [error] Error {error,{'load aborted at',"/tmp/exometer/EXOMETER-METRICS-MIB",oid_conflict}} when loading MIB /tmp/exometer/EXOMETER-METRICS-MIB
17:20:30.895 [error] CRASH REPORT Process exometer_report_snmp with 0 neighbours crashed with reason: no match of right hand value {error,{'load aborted at',"/tmp/exometer/EXOMETER-METRICS-MIB",oid_conflict}} in exometer_report_snmp:enable_metric/2 line 219
My example configuration:
https://github.com/IgorKarymov/exometer/tree/oid_conflict_error
Steps to reproduce:
-
make
-
make devnode_snmp_agent (note: start command and configuration file was changed by me)
- exometer:update_or_create([api, opiwa1], 3000, histogram, [{snmp, []}])
Also looks like, that changes of dynamically created MIB was stopped after this error occurred (you can introduce new metrics, but file still be the same, however calls will returns 'ok')
dependency on erlang-snmp
Initially this problem arose while attempting to install zotonic.
The "make" would fail at exometer.
So I focused on exometer specifically.
.../temp/exometer
make
...
==> Entering directory /home/rodger/temp/exometer/deps/jiffy' ==> jiffy (compile) ==> Leaving directory/home/rodger/temp/exometer/deps/jiffy'
==> Entering directory /home/rodger/temp/exometer/src' ==> src (compile) ==> Leaving directory/home/rodger/temp/exometer/src'
==> exometer (compile)
make: *** [compile] Error 1
rebar -vvv compile
==> exometer (compile)
DEBUG: Matched required ERTS version: 5.10.4 -> .*
DEBUG: Matched required OTP release: R16B03 -> .*
DEBUG: is_app_available, looking for App exometer_core with Path "/home/rodger/temp/exometer/deps/exometer_core"
INFO: Looking for exometer_core-.* ; found exometer_core-1.0-13-g1527a82 at /home/rodger/temp/exometer/deps/exometer_core
DEBUG: is_app_available, looking for App exometer_collectd with Path "/home/rodger/temp/exometer/deps/exometer_collectd"
INFO: Looking for exometer_collectd-.* ; found exometer_collectd-f75e680 at /home/rodger/temp/exometer/deps/exometer_collectd
DEBUG: is_app_available, looking for App netlink with Path "/home/rodger/temp/exometer/deps/netlink"
INFO: Looking for netlink-.* ; found netlink-1 at /home/rodger/temp/exometer/deps/netlink
DEBUG: is_app_available, looking for App amqp_client with Path "/home/rodger/temp/exometer/deps/amqp_client"
INFO: Looking for amqp_client-.* ; found amqp_client-3.3.5 at /home/rodger/temp/exometer/deps/amqp_client
DEBUG: is_app_available, looking for App jiffy with Path "/home/rodger/temp/exometer/deps/jiffy"
INFO: Looking for jiffy-.* ; found jiffy-0.13.3 at /home/rodger/temp/exometer/deps/jiffy
DEBUG: Starting 3 compile worker(s)
DEBUG: Worker exited cleanly
DEBUG: Worker exited cleanly
DEBUG: Worker compilation failed: [{error,
{'EXIT',
{undef,
[{snmpc,compile,
["mibs/EXOMETER-MIB.mib",
[{outdir,"priv/mibs"},
{i,["priv/mibs"]}]],
[]},
{rebar_erlc_compiler,compile_mib,3,
[{file,"src/rebar_erlc_compiler.erl"},
{line,277}]},
{rebar_base_compiler,compile,3,
[{file,"src/rebar_base_compiler.erl"},
{line,115}]},
{rebar_base_compiler,compile_worker,3,
[{file,"src/rebar_base_compiler.erl"},
{line,176}]}]}}},
{source,"mibs/EXOMETER-MIB.mib"}]
The solution
sudo apt-get install erlang-snmp
Likely the solution is obvious to most users of exometer but for me exometer is one of many dependencies for zotonic and it took me a while before finding my solution.
Static function matches crash reporter.
Based on #17 I have the following configuration in a simple project:
[
{exometer,
[
{ predefined,
[
{[erlang, gc],
{function, erlang, statistics, [garbage_collection], match, {total_collections, reclaimed_words, '_'}},
[]
}
]
},
{ report,
[
{subscribers,
[
{exometer_report_tty, [erlang, gc], total_collections, 5000, true},
{exometer_report_tty, [erlang, gc], reclaimed_words, 5000, true}
]},
{reporters,
[
{exometer_report_tty, []}
]}
]}
]}
].The result, when the tty reporter begins its work, is a crash:
> ./_rel/bin/erlhttp_bench console
Exec: /Users/blt/projects/com/adroll/erlhttp_bench/_rel/erts-5.10.3/bin/erlexec -boot /Users/blt/projects/com/adroll/erlhttp_bench/_rel/releases/0.0.1/erlhttp_bench -env ERL_LIBS /Users/blt/projects/com/adroll/erlhttp_bench/_rel/releases/0.0.1/lib -config /Users/blt/projects/com/adroll/erlhttp_bench/_rel/releases/0.0.1/sys.config -args_file /Users/blt/projects/com/adroll/erlhttp_bench/_rel/releases/0.0.1/vm.args -- console
Root: /Users/blt/projects/com/adroll/erlhttp_bench/_rel
Erlang R16B02 (erts-5.10.3) [source] [64-bit] [smp:4:4] [async-threads:128] [hipe] [kernel-poll:true]
14:34:29.435 [info] Application lager started on node '[email protected]'
=PROGRESS REPORT==== 18-Feb-2014::14:34:29 ===
supervisor: {local,sasl_safe_sup}
started: [{pid,<0.189.0>},
{name,alarm_handler},
{mfargs,{alarm_handler,start_link,[]}},
{restart_type,permanent},
{shutdown,2000},
{child_type,worker}]
=PROGRESS REPORT==== 18-Feb-2014::14:34:29 ===
supervisor: {local,sasl_sup}
started: [{pid,<0.191.0>},
{name,release_handler},
{mfargs,{release_handler,start_link,[]}},
{restart_type,permanent},
{shutdown,2000},
{child_type,worker}]
=PROGRESS REPORT==== 18-Feb-2014::14:34:29 ===
supervisor: {local,exometer_sup}
started: [{pid,<0.197.0>},
{name,exometer_admin},
{mfargs,{exometer_admin,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]
=PROGRESS REPORT==== 18-Feb-2014::14:34:29 ===
supervisor: {local,exometer_sup}
started: [{pid,<0.198.0>},
{name,exometer_cache},
{mfargs,{exometer_cache,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]
14:34:29.494 [info] Starting reporter with [{subscribers,[{exometer_report_tty,[erlang,gc],total_collections,5000,true},{exometer_report_tty,[erlang,gc],reclaimed_words,5000,true}]},{reporters,[{exometer_report_tty,[]}]}]
=PROGRESS REPORT==== 18-Feb-2014::14:34:29 ===
supervisor: {local,exometer_sup}
started: [{pid,<0.199.0>},
{name,exometer_report},
{mfargs,{exometer_report,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]
14:34:29.498 [info] exometer_report_tty([]): Starting
=PROGRESS REPORT==== 18-Feb-2014::14:34:29 ===
application: exometer
started_at: '[email protected]'
Eshell V5.10.3 (abort with ^G)
([email protected])1> 14:34:34.498 [warning] lager_error_logger_h dropped 56 messages in the last second that exceeded the limit of 50 messages/sec
=CRASH REPORT==== 18-Feb-2014::14:34:34 ===
crasher:
initial call: exometer_admin:init/1
pid: <0.197.0>
registered_name: exometer_admin
exception exit: {{case_clause,exometer_report_tty},
[{exometer_admin,handle_info,2,
[{file,"src/exometer_admin.erl"},
{line,229}]},
{gen_server,handle_msg,5,
[{file,"gen_server.erl"},{line,604}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]}
in function gen_server:terminate/6 (gen_server.erl, line 744)
ancestors: [exometer_sup,<0.195.0>]
messages: []
links: [<0.196.0>]
dictionary: [{#Ref<0.0.0.276>,exometer_report_tty},
{<0.200.0>,#Ref<0.0.0.276>}]
trap_exit: false
status: running
heap_size: 610
stack_size: 27
reductions: 235
neighbours:
=SUPERVISOR REPORT==== 18-Feb-2014::14:34:34 ===
Supervisor: {local,exometer_sup}
Context: child_terminated
Reason: {{case_clause,exometer_report_tty},
[{exometer_admin,handle_info,2,
[{file,"src/exometer_admin.erl"},
{line,229}]},
{gen_server,handle_msg,5,
[{file,"gen_server.erl"},{line,604}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]}
Offender: [{pid,<0.197.0>},
{name,exometer_admin},
{mfargs,{exometer_admin,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]
=PROGRESS REPORT==== 18-Feb-2014::14:34:34 ===
supervisor: {local,exometer_sup}
started: [{pid,<0.206.0>},
{name,exometer_admin},
{mfargs,{exometer_admin,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]
14:34:34.498 [error] gen_server exometer_admin terminated with reason: no case clause matching exometer_report_tty in exometer_admin:handle_info/2 line 229
14:34:34.503 [error] CRASH REPORT Process exometer_admin with 0 neighbours exited with reason: no case clause matching exometer_report_tty in exometer_admin:handle_info/2 line 229 in gen_server:terminate/6 line 744
14:34:34.503 [error] Supervisor exometer_sup had child exometer_admin started with exometer_admin:start_link() at <0.197.0> exit with reason no case clause matching exometer_report_tty in exometer_admin:handle_info/2 line 229 in context child_terminated
=CRASH REPORT==== 18-Feb-2014::14:34:35 ===
crasher:
initial call: exometer_report:init/1
pid: <0.199.0>
registered_name: exometer_report
exception exit: {badarg,
[{lists,member,
[total_collections,
{total_collections,reclaimed_words,'_'}],
[]},
{exometer_function,'-get_value/4-lc$^1/1-1-',2,
[{file,"src/exometer_function.erl"},{line,161}]},
{exometer_function,get_value,4,
[{file,"src/exometer_function.erl"},{line,160}]},
{exometer,get_value,2,
[{file,"src/exometer.erl"},{line,200}]},
{exometer_report,handle_info,2,
[{file,"src/exometer_report.erl"},{line,513}]},
{gen_server,handle_msg,5,
[{file,"gen_server.erl"},{line,604}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]}
in function gen_server:terminate/6 (gen_server.erl, line 744)
ancestors: [exometer_sup,<0.195.0>]
messages: [{report,{key,exometer_report_tty,
[erlang,gc],
reclaimed_words,true,undefined},
5000},
{'DOWN',#Ref<0.0.0.273>,process,<0.200.0>,normal}]
links: [<0.196.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1598
stack_size: 27
reductions: 1250
neighbours:
=SUPERVISOR REPORT==== 18-Feb-2014::14:34:35 ===
Supervisor: {local,exometer_sup}
Context: child_terminated
Reason: {badarg,
[{lists,member,
[total_collections,
{total_collections,reclaimed_words,'_'}],
[]},
{exometer_function,'-get_value/4-lc$^1/1-1-',2,
[{file,"src/exometer_function.erl"},{line,161}]},
{exometer_function,get_value,4,
[{file,"src/exometer_function.erl"},{line,160}]},
{exometer,get_value,2,
[{file,"src/exometer.erl"},{line,200}]},
{exometer_report,handle_info,2,
[{file,"src/exometer_report.erl"},{line,513}]},
{gen_server,handle_msg,5,
[{file,"gen_server.erl"},{line,604}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]}
Offender: [{pid,<0.199.0>},
{name,exometer_report},
{mfargs,{exometer_report,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]
=PROGRESS REPORT==== 18-Feb-2014::14:34:35 ===
supervisor: {local,exometer_sup}
started: [{pid,<0.207.0>},
{name,exometer_report},
{mfargs,{exometer_report,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]
14:34:35.499 [error] gen_server exometer_report terminated with reason: bad argument in call to lists:member(total_collections, {total_collections,reclaimed_words,'_'}) in exometer_function:'-get_value/4-lc$^1/1-1-'/2 line 161
14:34:35.499 [info] Starting reporter with [{subscribers,[{exometer_report_tty,[erlang,gc],total_collections,5000,true},{exometer_report_tty,[erlang,gc],reclaimed_words,5000,true}]},{reporters,[{exometer_report_tty,[]}]}]
14:34:35.500 [info] exometer_report_tty([]): Starting
14:34:35.502 [error] CRASH REPORT Process exometer_report with 0 neighbours exited with reason: bad argument in call to lists:member(total_collections, {total_collections,reclaimed_words,'_'}) in exometer_function:'-get_value/4-lc$^1/1-1-'/2 line 161 in gen_server:terminate/6 line 744
14:34:35.503 [error] Supervisor exometer_sup had child exometer_report started with exometer_report:start_link() at <0.199.0> exit with reason bad argument in call to lists:member(total_collections, {total_collections,reclaimed_words,'_'}) in exometer_function:'-get_value/4-lc$^1/1-1-'/2 line 161 in context child_terminated
=CRASH REPORT==== 18-Feb-2014::14:34:40 ===
crasher:
initial call: exometer_admin:init/1
pid: <0.206.0>
registered_name: exometer_admin
exception exit: {{case_clause,exometer_report_tty},
[{exometer_admin,handle_info,2,
[{file,"src/exometer_admin.erl"},
{line,229}]},
{gen_server,handle_msg,5,
[{file,"gen_server.erl"},{line,604}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]}
in function gen_server:terminate/6 (gen_server.erl, line 744)
ancestors: [exometer_sup,<0.195.0>]
messages: []
links: [<0.196.0>]
dictionary: [{#Ref<0.0.0.307>,exometer_report_tty},
{<0.208.0>,#Ref<0.0.0.307>}]
trap_exit: false
status: running
heap_size: 610
stack_size: 27
reductions: 147
neighbours:
=SUPERVISOR REPORT==== 18-Feb-2014::14:34:40 ===
Supervisor: {local,exometer_sup}
Context: child_terminated
Reason: {{case_clause,exometer_report_tty},
[{exometer_admin,handle_info,2,
[{file,"src/exometer_admin.erl"},
{line,229}]},
{gen_server,handle_msg,5,
[{file,"gen_server.erl"},{line,604}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]}
Offender: [{pid,<0.206.0>},
{name,exometer_admin},
{mfargs,{exometer_admin,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]
=PROGRESS REPORT==== 18-Feb-2014::14:34:40 ===
supervisor: {local,exometer_sup}
started: [{pid,<0.210.0>},
{name,exometer_admin},
{mfargs,{exometer_admin,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]
14:34:40.501 [error] gen_server exometer_admin terminated with reason: no case clause matching exometer_report_tty in exometer_admin:handle_info/2 line 229
14:34:40.503 [error] CRASH REPORT Process exometer_admin with 0 neighbours exited with reason: no case clause matching exometer_report_tty in exometer_admin:handle_info/2 line 229 in gen_server:terminate/6 line 744
14:34:40.503 [error] Supervisor exometer_sup had child exometer_admin started with exometer_admin:start_link() at <0.206.0> exit with reason no case clause matching exometer_report_tty in exometer_admin:handle_info/2 line 229 in context child_terminated
=CRASH REPORT==== 18-Feb-2014::14:34:41 ===
crasher:
initial call: exometer_report:init/1
pid: <0.207.0>
registered_name: exometer_report
exception exit: {badarg,
[{lists,member,
[total_collections,
{total_collections,reclaimed_words,'_'}],
[]},
{exometer_function,'-get_value/4-lc$^1/1-1-',2,
[{file,"src/exometer_function.erl"},{line,161}]},
{exometer_function,get_value,4,
[{file,"src/exometer_function.erl"},{line,160}]},
{exometer,get_value,2,
[{file,"src/exometer.erl"},{line,200}]},
{exometer_report,handle_info,2,
[{file,"src/exometer_report.erl"},{line,513}]},
{gen_server,handle_msg,5,
[{file,"gen_server.erl"},{line,604}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]}
in function gen_server:terminate/6 (gen_server.erl, line 744)
ancestors: [exometer_sup,<0.195.0>]
messages: [{report,{key,exometer_report_tty,
[erlang,gc],
reclaimed_words,true,undefined},
5000},
{'DOWN',#Ref<0.0.0.304>,process,<0.208.0>,normal}]
links: [<0.196.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1598
stack_size: 27
reductions: 1233
neighbours:
=SUPERVISOR REPORT==== 18-Feb-2014::14:34:41 ===
Supervisor: {local,exometer_sup}
Context: child_terminated
Reason: {badarg,
[{lists,member,
[total_collections,
{total_collections,reclaimed_words,'_'}],
[]},
{exometer_function,'-get_value/4-lc$^1/1-1-',2,
[{file,"src/exometer_function.erl"},{line,161}]},
{exometer_function,get_value,4,
[{file,"src/exometer_function.erl"},{line,160}]},
{exometer,get_value,2,
[{file,"src/exometer.erl"},{line,200}]},
{exometer_report,handle_info,2,
[{file,"src/exometer_report.erl"},{line,513}]},
{gen_server,handle_msg,5,
[{file,"gen_server.erl"},{line,604}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,239}]}]}
Offender: [{pid,<0.207.0>},
{name,exometer_report},
{mfargs,{exometer_report,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]
=PROGRESS REPORT==== 18-Feb-2014::14:34:41 ===
supervisor: {local,exometer_sup}
started: [{pid,<0.211.0>},
{name,exometer_report},
{mfargs,{exometer_report,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]
14:34:41.502 [error] gen_server exometer_report terminated with reason: bad argument in call to lists:member(total_collections, {total_collections,reclaimed_words,'_'}) in exometer_function:'-get_value/4-lc$^1/1-1-'/2 line 161
14:34:41.503 [info] Starting reporter with [{subscribers,[{exometer_report_tty,[erlang,gc],total_collections,5000,true},{exometer_report_tty,[erlang,gc],reclaimed_words,5000,true}]},{reporters,[{exometer_report_tty,[]}]}]
14:34:41.503 [info] exometer_report_tty([]): Starting
14:34:41.505 [error] CRASH REPORT Process exometer_report with 0 neighbours exited with reason: bad argument in call to lists:member(total_collections, {total_collections,reclaimed_words,'_'}) in exometer_function:'-get_value/4-lc$^1/1-1-'/2 line 161 in gen_server:terminate/6 line 744
14:34:41.506 [error] Supervisor exometer_sup had child exometer_report started with exometer_report:start_link() at <0.207.0> exit with reason bad argument in call to lists:member(total_collections, {total_collections,reclaimed_words,'_'}) in exometer_function:'-get_value/4-lc$^1/1-1-'/2 line 161 in context child_terminated
BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
(v)ersion (k)ill (D)b-tables (d)istribution
^C%
Have I made an error somewhere?
Subscriptions to function metrics no longer work.
I start exometer at commit b68cb79 and create a function metric which defers to erlang:memory/1.
blt> pwd
~/projects/com/Feuerlabs/exometer
blt> erl -pa deps/*/ebin ebin
Erlang R16B02 (erts-5.10.3) [source] [64-bit] [smp:4:4] [async-threads:10]
[hipe] [kernel-poll:false]
Eshell V5.10.3 (abort with ^G)
1> exometer:start().
22:13:42.578 [info] Application lager started on node nonode@nohost
22:13:42.590 [info] Starting reporters with []
22:13:42.590 [info] Application exometer started on node nonode@nohost
ok
2> ok = exometer_report:add_reporter(exometer_report_tty, []).
22:14:01.797 [info] exometer_report_tty([]): Starting
ok
3> ok = exometer:new([mem], {function,erlang,memory,[],proplist, total,processes]}).
ok
4> exometer:get_value([mem], total).
{ok,[{total,17681712}]}
The metric is setup correctly, now to subscribe it:
> exometer_report:subscribe(exometer_report_tty, [mem], total, 1000, []).
not_found
Hmm, that's unexpected. The following is a curious result:
> exometer_report:subscribe(exometer_report_tty, [mem], value, 1000, []).
This returns ok but there's nothing printed by the reporter (and, indeed, I don't think there is anything to be reported).
dbg:tracer(), dbg:p(all, c).
dbg:tpl(exometer_report, is_valid_metric, x).
dbg:tpl(exometer_report, dp_list, x).
dbg:tp(exometer, info, x).
Attempting subscriptions:
12> exometer_report:subscribe(exometer_report_tty, [mem], value, 1000, []).
(<0.71.0>) call exometer_report:is_valid_metric([mem],value)
(<0.71.0>) call exometer_report:dp_list(value)
(<0.71.0>) returned from exometer_report:dp_list/1 -> [value]
(<0.71.0>) call exometer:info([mem],datapoints)
(<0.71.0>) returned from exometer:info/2 -> [value]
(<0.71.0>) returned from exometer_report:is_valid_metric/2 -> true
ok
13> exometer_report:subscribe(exometer_report_tty, [mem], total, 1000, []).
(<0.71.0>) call exometer_report:is_valid_metric([mem],total)
(<0.71.0>) call exometer_report:dp_list(total)
(<0.71.0>) returned from exometer_report:dp_list/1 -> [total]
(<0.71.0>) call exometer:info([mem],datapoints)
(<0.71.0>) returned from exometer:info/2 -> [value]
(<0.71.0>) returned from exometer_report:is_valid_metric/2 -> false
not_found
It seems to me that the ultimate trouble is in is_valid_metric but I'm not entirely certain what's meant to constitute a valid metric or how taking the difference of DataPoints and [value] determines that. (I'm curious about the intention here, if you have the time.)
This seems to only be a problem for function metrics. Am I setting this dynamic subscription up incorrectly or have a stumbled over a bug?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.