Hi,

When I push the Test Monitor button, I get the following error without any details:

alias(carbon.agents.ip-10-214-134-148-a.avgUpdateTime,"test")

The log of the rails app is:

I, [2014-05-15T14:20:00.907000 #22029] INFO -- : Started GET "/rearview/monitors/index.json" for 10.226.195.119 at 2014-05-15 14:20:00 +0300
I, [2014-05-15T14:20:01.621000 #22029] INFO -- : Started GET "/rearview/help/quick.html" for 10.226.195.119 at 2014-05-15 14:20:01 +0300
I, [2014-05-15T14:20:08.558000 #22029] INFO -- : Started POST "/monitor.json" for 10.226.195.119 at 2014-05-15 14:20:08 +0300
I, [2014-05-15T14:20:08.582000 #22029] INFO -- : Processing by Rearview::MonitorController#create as JSON
I, [2014-05-15T14:20:08.584000 #22029] INFO -- : Parameters: {"id"=>"", "dashboardId"=>"1", "userId"=>"1", "name"=>"TestMonitor", "active"=>"true", "jobType"=>"monitor", "version"=>"2", "cronExpr"=>"0 0 * ? Feb Sun", "errorTimeout"=>"60", "minutes"=>"1", "metrics"=>["alias(carbon.agents.ip-10-214-134-148-a.avgUpdateTime,"test")"], "monitorExpr"=>"", "toDate"=>"now", "createdAt"=>"", "modifiedAt"=>"", "description"=>""}
I, [2014-05-15T14:20:09.116000 #22029] INFO -- : Rendered /usr/local/rvm/gems/jruby-1.7.12/gems/rearview-1.2.1-java/app/views/rearview/monitor/create.json.jbuilder (10.0ms)
I, [2014-05-15T14:20:09.158000 #22029] INFO -- : Completed 200 OK in 531ms (Views: 58.0ms | ActiveRecord: 2.0ms)

Does anybody have any ideas?

Hi,

I followed all the steps as per https://github.com/livingsocial/rearview. after running setup script in the bin directory I encountered following error.

rake aborted!
uninitialized constant Rearview::Engine
/home/ubuntu/rearview/config/routes.rb:2:in block in <top (required)>' /home/ubuntu/rearview/config/routes.rb:1:in<top (required)>'
/home/ubuntu/rearview/config/environment.rb:5:in `<top (required)>'

Kindly help me with this issue.

Just started using rearview-v.1.2.1 When using the "Test Monitor" on a single metric and raise an alert when the value exceeds some number I keep receiving the following error on the Output after waiting for about 5 minutes.

/Users/ronmb/.rbenv/versions/1.9.3-p547/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in require': cannot load such file -- bundler (LoadError) from /Users/ronmb/.rbenv/versions/1.9.3-p547/lib/ruby/1.9.1/rubygems/custom_require.rb:36:inrequire'
from /Users/ronmb/Projects/rearview/sandbox/monitor_script20140808-2941-mdlw7a:7:in `

Verified that I have the bundler gem. I've included my ruby environment info as well.

$ gem list | grep bundl
bundle (0.0.1)
bundler (1.6.5)
$ rbenv local
jruby-1.7.5
$ rbenv global
1.9.3-p547

I ran both bin/setup and verification and all passed without issues.

Hi,

Is there a way to have read only account for rearview. Currently I am using admin@localhost and would like to have an read only account which does not allow any modifications to monitors and dashboards.

Thanks,

Hi Rearview team,

I have the following configuration for graphite in config/initializers/rearview.rb

config.graphite_connection = {url: "https://www.hostedgraphite.com/{Hosted Graphite Access Key}/graphite"}

When I run the instructed command to check verifications (rake RAILS_ENV=production rearview:config:verify) I get the following output at the end.

validating...FAILED Graphite connection graphite cannot be reached

I have verified that the graphite url I provided is working (e.g., appending '/render' at the end will get you an empty graph).

Any ideas why it is failing the validation?

Thanks in advance

How is the URL for obtaining the data from Graphite built? I see how the hostname and auth information is supplied, but the URL that hits my Graphite servers doesn't seem to be correct. It seems to be missing some of the base parameters.

I'm not a Scala guy, so its a bit hard to figure out how that is being generated.

Hi Rearview Team,

Our team was on the Scala version and everything worked well. Now we are setting up a new rearview instance using the latest version.

When I took an currently working Metric from one of our rearview monitors, and entered it on the new rearview instance, and hit test monitor, the following errors were shown.

Monitor Test Error!
Metrics contains an invalid metric: {working metrics I entered}

Server logs
I, [2014-06-19T22:06:14.118000 #433] INFO -- : Started POST "/monitor.json" for 24.137.223.133 at 2014-06-19 22:06:14 +0000
I, [2014-06-19T22:06:14.156000 #433] INFO -- : Processing by Rearview::MonitorController#create as JSON
I, [2014-06-19T22:06:14.157000 #433] INFO -- : Parameters: {"id"=>"", "dashboardId"=>"1", "userId"=>"2", "name"=>"test", "active"=>"true", "jobType"=>"monitor", "version"=>"2", "cronExpr"=>"0 0 * ? Mar,May,Sep Sun,Mon,Wed,Thu,Fri,Sat", "errorTimeout"=>"60", "minutes"=>"15", "metrics"=>["{my metric}"], "monitorExpr"=>"limit = 25000\n\nif not @a.nil?\n \nvalue = @a.values[-2] \n\nif not value.nil?\n\n\traise "Max 95 Percentile for Get Anonymous Access Code is currently at #{value}) higher than the #{limit}" if value > limit\n \nend\n\nend", "toDate"=>"now", "createdAt"=>"", "modifiedAt"=>"", "description"=>""}
I, [2014-06-19T22:06:15.135000 #433] INFO -- : Rendered /home/ubuntu/.rvm/gems/jruby-1.7.5@rearview/gems/rearview-1.2.1-java/app/views/rearview/monitor/create.json.jbuilder (2.0ms)
I, [2014-06-19T22:06:15.137000 #433] INFO -- : Completed 200 OK in 970ms (Views: 6.0ms | ActiveRecord: 2.0ms)

I have a valid hosted graphite connection as can be seen here #44

Any idea why its happening?

Thanks

My server is set to UTC and that's what I'm getting for times in the Rearview UI. I'd prefer to not change the server time if I don't have to. Is there a way to tell Rearview which timezone to display in and run scheduled jobs at?

@steveakers, now that @talbright is no longer with LivingSocial, do you guys have plans to continue furthering Rearview or should we consider the project dead?

I'd hate to invest in a project that is no longer maintained.

Under which license has this been released?

Thanks!

This probably isn't an issue with MySQL since you can grant all to a user for a specific database which gives a user permission to create it, but that's not how postgresql permissions work.
So... I'd like the setup to not complain when I have an empty database waiting for it so I don't have to grant CREATE permissions to the rearview postgresql user.

So I am trying to make a chance to the graphite host in my common.conf directory but it doesn't seem to be taking effect.

In my conf I have

# Graphite
graphite.host="http://graphite-internal.int.xxxxx.com"
graphite.auth=""
graphite.timeout=10000

but you can see in the debug logs its still trying the other hostname.

[debug] s.s.c.FixRowNumberOrdering - No row numbers to fix
[debug] s.s.c.QueryCompiler - After phase fixRowNumberOrdering: (no change)
[debug] s.s.s.BaseSession - Preparing statement: select s4.`id`, s4.`user_id`, s4.`app_id`, s4.`name`, s4.`cron_expr`, s4.`metrics`, s4.`monitor_expr`, s4.`minutes`, s4.`to_date`, s4.`description`, s4.`active`, s4.`status`, s4.`last_run`, s4.`next_run`, s4.`alert_keys`, s4.`error_timeout`, s4.`created`, s4.`modified`, s4.`deleted_at` from `jobs` s4 where (s4.`deleted_at` is null) and (s4.`active` = true)
[info] play - Application started (Prod)
[info] play - Listening for HTTP on /0.0.0.0:9000
[info] a.e.s.Slf4jEventHandler - Slf4jEventHandler started
[error] application - Graphite request failure: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Authorization Required</title>
</head><body>
<h1>Authorization Required</h1>
<p>This server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
<hr>
<address>Apache Server at graphite.int.xxxxx.com Port 443</address>
</body></html>

I have tried killing the process, which didnt work. I also tried re-running ./sbt test to see if that maybe would help, but no dice so far. How do I do things like update the config, and reload config ?

Thanks!

-John

Hi,

I am successful in setting up rearview-1.1.1 on ubuntu. I updated the rearview.rb file and ran this command for verifying configuration.

$ rake RAILS_ENV=production rearview:config:verify

output:

using "production" configuration:
authentication={:strategy=>:database}
default_from=[email protected]
default_url_options={:host=>"localhost", :port=>"3000"}
enable_alerts=false
enable_monitor=false
enable_stats=false
graphite_url=https://www.hostedgraphite.com/access-key
logger=#ActiveSupport::Logger:0x5e7cae4e
pagerduty_url=https://events.pagerduty.com/generic/2010-04-15/create_event.json
preload_jobs=false
sandbox_dir=/opt/rearview-1.1.1/sandbox
sandbox_exec=["rvm-exec", "ruby-1.9.3@rearview-sandbox", "ruby"]
sandbox_timeout=10
statsd_connection=nil
verify=false
validating...FAILED
Graphite url is not a reachable URL

I am using Jruby-1.7.10 and no other ruby version. when I run ruby -v I get this:

jruby 1.7.10 (1.9.3p392) 2014-01-09 c4ecd6b on OpenJDK 64-Bit Server VM 1.7.0_51-b00 [linux-amd64]

Not sure if the value of sandbox_exec is correct.

Before I file the bug, have to admit I am a novice Ruby user and quite possibly this is a user error:-)

Installed rearview-1.1.2 on Ubuntu 12.04 using the setup procedure described in the project

When I add a monitor and test it, I get the following error in output window

/home/boks/.rvm/gems/ruby-1.9.3-p484@global/gems/bundler-1.5.2/lib/bundler/spec_set.rb:92:in block in materialize': Could not find i18n-0.6.5 in any of the sources (Bundler::GemNotFound) from /home/boks/.rvm/gems/ruby-1.9.3-p484@global/gems/bundler-1.5.2/lib/bundler/spec_set.rb:85:inmap!'
from /home/boks/.rvm/gems/ruby-1.9.3-p484@global/gems/bundler-1.5.2/lib/bundler/spec_set.rb:85:in materialize' from /home/boks/.rvm/gems/ruby-1.9.3-p484@global/gems/bundler-1.5.2/lib/bundler/definition.rb:133:inspecs'
from /home/boks/.rvm/gems/ruby-1.9.3-p484@global/gems/bundler-1.5.2/lib/bundler/definition.rb:178:in specs_for' from /home/boks/.rvm/gems/ruby-1.9.3-p484@global/gems/bundler-1.5.2/lib/bundler/definition.rb:167:inrequested_specs'
from /home/boks/.rvm/gems/ruby-1.9.3-p484@global/gems/bundler-1.5.2/lib/bundler/environment.rb:18:in requested_specs' from /home/boks/.rvm/gems/ruby-1.9.3-p484@global/gems/bundler-1.5.2/lib/bundler/runtime.rb:13:insetup'
from /home/boks/.rvm/gems/ruby-1.9.3-p484@global/gems/bundler-1.5.2/lib/bundler.rb:119:in setup' from /home/boks/workspace/rearview/rearview-1.1.2/sandbox/monitor_script20140130-27375-1r4o78l:10:in

My gem list is

*** LOCAL GEMS ***

actionmailer (4.0.2)
actionpack (4.0.2)
activemodel (4.0.2)
activerecord (4.0.2)
activerecord-deprecated_finders (1.0.3)
activerecord-jdbc-adapter (1.3.4)
activerecord-jdbcmysql-adapter (1.3.4)
activesupport (4.0.2)
ancestry (2.0.0)
arel (4.0.1)
atomic (1.1.14 java)
bcrypt-ruby (3.1.2 java)
bouncy-castle-java (1.5.0147)
broach (0.3.0)
builder (3.1.4)
bundler (1.5.2)
bundler-unload (1.0.2)
celluloid (0.14.1)
coderay (1.1.0)
coffee-rails (4.0.1)
coffee-script (2.2.0)
coffee-script-source (1.6.3)
devise (3.2.2)
diff-lcs (1.2.5)
dotenv (0.9.0)
erubis (2.7.0)
execjs (2.0.2)
executable-hooks (1.3.1)
faraday (0.8.8)
ffi (1.9.3 java)
foreman (0.63.0 java)
gem-wrappers (1.2.4)
hashie (2.0.5)
hike (1.2.3)
hitimes (1.2.1 java)
httparty (0.12.0)
httpauth (0.2.0)
i18n (0.6.9)
jbuilder (1.5.3)
jdbc-mysql (5.1.27)
jquery-rails (3.0.4)
jruby-launcher (1.0.19 java)
jruby-openssl (0.9.4)
json (1.8.1 java)
jwt (0.1.8)
mail (2.5.4)
method_source (0.8.2)
mime-types (1.25.1)
minitest (4.7.5)
multi_json (1.8.2)
multi_xml (0.5.5)
multipart-post (1.2.0)
nap (0.6.0)
oauth2 (0.8.1)
omniauth (1.1.4)
omniauth-google-oauth2 (0.2.1)
omniauth-oauth2 (1.1.1)
orm_adapter (0.5.0)
polyglot (0.3.3)
protected_attributes (1.0.5)
pry (0.9.12.4 java)
puma (2.7.1 java)
rack (1.5.2)
rack-test (0.6.2)
rails (4.0.2)
railties (4.0.2)
rake (10.1.1)
rdoc (3.12.2)
rearview (1.1.2 jruby)
rspec-core (2.14.7)
rspec-expectations (2.14.4)
rspec-mocks (2.14.4)
rspec-rails (2.14.0)
rubygems-bundler (1.4.2)
rvm (1.11.3.8)
sass (3.2.13)
sass-rails (4.0.1)
sdoc (0.3.20)
slop (3.4.7)
spoon (0.0.4)
sprockets (2.10.1)
sprockets-rails (2.0.1)
state_machine (1.2.0)
statsd-ruby (1.2.1)
therubyrhino (2.0.2)
therubyrhino_jar (1.7.4)
thor (0.18.1)
thread_safe (0.1.3 java)
tilt (1.4.1)
timers (2.0.0)
treetop (1.4.15)
turbolinks (2.1.0)
tzinfo (0.3.38)
uglifier (2.4.0)
warden (1.2.3)

Will appreciate any help understanding this issue

When Graphite doesn't have data or when there is a connection error, rearview throw below error and it triggers an alert.

-:112:in initialize': undefined method[]' for nil:NilClass (NoMethodError)
from -:214:in new' from -:214:inblock in create_timeseries'
from -:214:in map' from -:214:increate_timeseries'
from -:237:in secure_eval' from -:247:in

When running bin/setup, the run_install() function runs /bin/rails which throws this error:

D, [2014-01-23T16:30:17.141000 #32698] DEBUG -- : Shutdown completed cleanly
LoadError: load error: /home/rearview/rearview-1.1.1/config/environment -- java.lang.NoSuchFieldException: isRestricted
require at org/jruby/RubyKernel.java:1083
require at /home/rearview/.rvm/gems/jruby-1.7.10/gems/activesupport-4.0.2/lib/active_support/dependencie
s.rb:229
load_dependency at /home/rearview/.rvm/gems/jruby-1.7.10/gems/activesupport-4.0.2/lib/active_support/dependencie
s.rb:214
require at /home/rearview/.rvm/gems/jruby-1.7.10/gems/activesupport-4.0.2/lib/active_support/dependencie
s.rb:229
require_environment! at /home/rearview/.rvm/gems/jruby-1.7.10/gems/railties-4.0.2/lib/rails/application.rb:189
(root) at /home/rearview/.rvm/gems/jruby-1.7.10/gems/railties-4.0.2/lib/rails/commands.rb:44
require at org/jruby/RubyKernel.java:1083
(root) at bin/rails:12
D, [2014-01-23T16:30:45.149000 #32717] DEBUG -- : Shutdown completed cleanly

When working on a dashboard with many monitors, the interface slows to a crawl. I suspect it is due to graphing. I would like to be able to show a list of the monitors without them trying to graph their data.

Right now, if I want to edit a monitor, I need to turn all the monitors on the dashboard off before expanding a monitor to edit it, otherwise, garbage collection happens every few seconds, which is maddening.

Hello :)
I've pushed dredd - a gem which I've built for an internal alerting system (during my research process I've bumped into rearview - impressive work!), and I think may
also be worth looking at for rearview.

https://github.com/jondot/dredd

It might be useful for people without a technical background, that need to express a detection rule in something close to English.

I've exhausted the "run a rule over a collection in-memory" space, and then had no choice but to build it (common solutions are - keep sqlite in-memory and run SQL over collection, ESPER, etc - are all too heavy).

The simple operation of
puts @a.values.sum

Creates this error.

/opt/rearview/rearview/sandbox/monitor_script20140419-4332-1fd20qm:206:in write': Insecure operation - write (SecurityError) from /opt/rearview/rearview/sandbox/monitor_script20140419-4332-1fd20qm:206:inputs'
from /opt/rearview/rearview/sandbox/monitor_script20140419-4332-1fd20qm:206:in puts' from /opt/rearview/rearview/sandbox/monitor_script20140419-4332-1fd20qm:206:inblock in scoped_eval'

puts @A or puts @timeseries seems to work just fine.

Hi Guys,

Great project I was hoping this could go mainstream because it fits exactly what we need in the graphite's ecosystem!

I set the project up and everything was going fine until the job is being scheduled. I was repeatedly getting the error logs below.

[info] application - Running job 405
[info] application - Completed job 405 (FailedStatus)
[info] application - ScheduleJob 405
[info] application - Scheduled 405 Monitoring 50 * * * * ?
[info] application - ScheduleJob 405
[info] application - Scheduled 405 Monitoring 50 * * * * ?
[info] application - Running job 405
[info] application - Completed job 405 (FailedStatus)
[info] application - ScheduleJob 405
[info] application - Scheduled 405 Monitoring 50 * * * * ?
[info] application - ScheduleJob 405
[info] application - Scheduled 405 Monitoring 50 * * * * ?
[info] application - Running job 405
[info] application - Completed job 405 (FailedStatus)
[info] application - ScheduleJob 405

Any idea what could it be? Test monitor button on the UI worked for me and I tried to search for useful logs regarding this but couldn't find any. Appreciate any tips on this!

I can find no direct documentation, and I'm not well versed in Scala, but t looks like the Graphite.auth config only takes a password. Is there a way to auth against a Graphite server using both username and password? I've tried passing "user:password" but this does not work.

Thanks!

Right now it doesn't look like I can edit or even view the config of existing monitors.

Firstly, kudos for a very useful contribution to the community. For a prospective user evaluating rearview, including the following in the documentation would be of great help:

1. Documentation around failover / restoration considerations for rearview itself.

2. Duration of use in production and documentation of any observed gotchas, numbers related to rearview's uptime, performance (with respect to number of metrics, poll interval) and scale (w.r.t number of metrics and frequency of sampling) in a production environment.

3. Is rearview still in active use / development internally ? Any other adopters outside of LivingSocial?

Thanks.

In https://github.com/livingsocial/rearview/blob/master/app/controllers/users/omniauth_callbacks_controller.rb you reference the method Rearview::User.valid_email?, but I couldn't find its definition.

Am I missing something or it just something that fell off when you were open sourcing the code?

We'd like to do the authentication ourselves. Is there a simple way to turn off the open id authentication?

Hey guys,

How does statsd used with the product? I see this in the logs with it disabled

[warn] application - Send will be NOOP because statsd.enabled is not enabled

Since I dont use stats I am not sure what this actually means...

Am seeing the data in graphite, but don't see the data in rearview. Below is the error log:
I, [2014-04-18T13:19:34.245000 #11876] INFO -- : #Rearview::ResultsHandler:0x64e17c9c firing event :error for #<Rearview::Job id: 5, created_at: "2014-04-18 11:28:31", updated_at: "2014-04-18 13:18:33", name: "my ABC monitor", active: true, last_run: "2014-04-18 13:19:34", cron_expr: "0 * * * * ?", status: "error", user_id: 1, alert_keys: [], deleted_at: nil, error_timeout: 60, next_run: nil, description: "test 123", app_id: 1, metrics: ["alias(servers.prod.abc, "abc")"], monitor_expr: "puts @timeseries\n\nabc = @a.values.sum\ngraph_value["...", minutes: 10, to_date: "now">

Trent/Others,
I want to debug rearview rails application. When i use ruby-debug gem, the application stops and hangs when it hits 'debugger' line of code. Am thinking of using 'pry'. Would rearview work with Pry? Please let me know.

After having gotten rearview up, I tried adding my first monitor. Here's the expressions section:

puts @timeseries

avg = @a.values.mean
raise "Running #{avg} transactions!" if avg > 1

When I hit the test button I get this error output:

/home/rearview/.rvm/gems/jruby-1.7.10/gems/bundler-1.5.2/lib/bundler/spec_set.rb:92:in `block in materialize': Could not find i18n-0.6.5 in any of the sources (Bundler::GemNotFound)
    from /home/rearview/.rvm/gems/jruby-1.7.10/gems/bundler-1.5.2/lib/bundler/spec_set.rb:85:in `map!'
    from /home/rearview/.rvm/gems/jruby-1.7.10/gems/bundler-1.5.2/lib/bundler/spec_set.rb:85:in `materialize'
    from /home/rearview/.rvm/gems/jruby-1.7.10/gems/bundler-1.5.2/lib/bundler/definition.rb:133:in `specs'
    from /home/rearview/.rvm/gems/jruby-1.7.10/gems/bundler-1.5.2/lib/bundler/definition.rb:178:in `specs_for'
    from /home/rearview/.rvm/gems/jruby-1.7.10/gems/bundler-1.5.2/lib/bundler/definition.rb:167:in `requested_specs'
    from /home/rearview/.rvm/gems/jruby-1.7.10/gems/bundler-1.5.2/lib/bundler/environment.rb:18:in `requested_specs'
    from /home/rearview/.rvm/gems/jruby-1.7.10/gems/bundler-1.5.2/lib/bundler/runtime.rb:13:in `setup'
    from /home/rearview/.rvm/gems/jruby-1.7.10/gems/bundler-1.5.2/lib/bundler.rb:119:in `setup'
    from /home/rearview/.rvm/gems/jruby-1.7.10/gems/bundler-1.5.2/lib/bundler/setup.rb:17:in `<top (required)>'
    from /home/rearview/.rvm/rubies/ruby-1.9.3-p484/lib/ruby/site_ruby/1.9.1/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /home/rearview/.rvm/rubies/ruby-1.9.3-p484/lib/ruby/site_ruby/1.9.1/rubygems/core_ext/kernel_require.rb:55:in `require'

The gem does seem to be there, but I'm an rvm n00b.

I think it timed out getting data from Graphite. It might be something else. I got the following email alert:

ALERT: java.util.concurrent.TimeoutException: No response received after 120000

Monitor: Auth-Error-Rate-Prod
Description: None

Alerted On: 11/22/2013 20:48:30 UTC

Another example below. In this case the monitor works fine most of the time:

ALERT: rearview.model.GraphiteMetricException:

Graphite encountered an unexpected error while handling your request.

Please contact your site administrator if the problem persists.

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/opt/g/7ac8d61a7b/webapp/graphite/render/views.py", line 118, in renderView
    seriesList = evaluateTarget(requestContext, target)
  File "/opt/g/7ac8d61a7b/webapp/graphite/render/evaluator.py", line 9, in evaluateTarget
    tokens = grammar.parseString(target)
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 1031, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2543, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2543, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2543, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2398, in parseImpl
    ret = e._parse( instring, loc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2543, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2288, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2735, in parseImpl
    loc, tokens = self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2288, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2651, in parseImpl
    loc, tokens = self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2288, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2543, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2398, in parseImpl
    ret = e._parse( instring, loc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2543, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2398, in parseImpl
    ret = e._parse( instring, loc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2288, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 905, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2355, in parseImpl
    return maxMatchExp._parse( instring, loc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 962, in _parseCache
    value = self._parseNoCache( instring, loc, doActions, callPreParse )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 909, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 2606, in parseImpl
    self.expr.tryParse( instring, loc )
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 947, in tryParse
    return self._parse( instring, loc, doActions=False )[0]
  File "/usr/lib/python2.7/dist-packages/pyparsing.py", line 956, in _parseCache
    value = ParserElement._exprArgCache[ lookup ]
KeyError: (")", u'highestMax(transformNull(summarize(maxSeries(tnt-auth-prod.summarized.com.ea.tnt.service.auth.rest.v1.AccessTokenResource.*.getAccessToken.95percentile),"30min","avg"), 0), 1)', 174, True, False)

Monitor: Auth-AccessToken-Response-30min-Prod
Description: None

Should it send out alerts for internal errors? Is there a way to prevent this from happening?

Hi,

I want to use some simple sql queries such as create, insert, update, select etc from monitor scripts. I am planning to perform these operations on a mysql database used by rearview i.e (database mentioned in database.yml). Mysql is installed on the same box as reaview.

Is it possible to interact with database from monitor scripts ? If not is there any other way to achieve this ?

Thanks

The documentation mentions this file needs to be modified before running rearview, but the file is missing from the source code.

This probably causes running rake RAILS_ENV=production rearview:config:verify to fail with the following reason:

using "production" configuration:
authentication={:strategy=>:database}
default_from=rearview@localhost
default_url_options=nil
enable_alerts=true
enable_monitor=true
graphite_url=nil
logger=nil
pagerduty_url=https://events.pagerduty.com/generic/2010-04-15/create_event.json
preload_jobs=true
sandbox_dir=nil
sandbox_exec=nil
sandbox_timeout=5
verify=false
validating...rake aborted!
can't convert nil into String
org/jruby/RubyFile.java:1895:in `join'
org/jruby/RubyFile.java:826:in `join'
org/jruby/RubyProc.java:271:in `call'
org/jruby/RubyArray.java:1613:in `each'
org/jruby/RubyArray.java:1613:in `each'
org/jruby/RubyKernel.java:1101:in `load'
org/jruby/RubyKernel.java:1121:in `eval'
/Users/khash/.rvm/gems/jruby-1.7.5@global/bin/jruby_executable_hooks:15:in `(root)'
Tasks: TOP => rearview:config:verify
(See full trace by running task with --trace)
D, [2014-01-12T11:52:23.061000 #44446] DEBUG -- : Shutdown completed cleanly

Hi, For setting up email alerts do I just have to configure SMTP settings or have to create a new Mailer as well ? Sorry if its a dumb question as I am new to ruby and rails so not sure how things work.

For SMTP, in config/environments/production.rb I have added these configs.

config.action_mailer.raise_delivery_errors = true
config.action_mailer.delivery_method = :smtp
config.action_mailer.smtp_settings = {
:address => "email-smtp.us-east-1.amazonaws.com",
:port => 25,
:user_name => "myusername",
:password => "mypassword",
:authentication => :login,
:enable_starttls_auto => true }

Also could you give insights on setting up pagerduty ?

Thanks in advance

Hi,

Rearview is sending email alerts after every 60 minutes even if monitors are triggered every 5 minutes or any other interval for that matter.

My graphite server is in Pacific time zone and because of that I made smal changes in the 'create_from_to_dates' function in monitor_runner.rb file.
This is how it looks.

def create_from_to_dates(minutes = nil, to_date = nil)
logger.debug "#{self}#create_from_to_dates minutes:#{minutes} to_date:#{to_date}"
graphite_date_format = '%H:%M_%Y%m%d'
incoming_date_format = '%m/%d/%Y %H:%M'
mins = minutes.nil? ? @@DEFAULT_MINUTES : minutes.to_i
if to_date == "now" || to_date.nil?
local = DateTime.now
pacific_offset = Rational(-6,24)
now = local.new_offset(pacific_offset)
else
now = DateTime.strptime(to_date, incoming_date_format)
pacific_offset = Rational(-6,24)
now = now.new_offset(pacific_offset)
end
to = (now - 10.minutes).strftime(graphite_date_format)
from = (now - (mins + 15).minutes).strftime(graphite_date_format)
logger.debug "#{self}#create_from_to_dates from:#{from} to:#{to}"
[from,to]
end

This is the only change I made.

I spent a lot of time digging into this but did not had any luck. Do you know what might be the reason for this ?

In order to interact with Pacific time zone is there any other place where I need to make any changes ?

Thanks in advance.

Hi,

I have setup rearview-scala version on EC2 instance and using Amazon SES.

In the conf/common.conf file I have updated values as follows:

statsd.enabled=false
graphite.host="https://hostedgraphite.com/acces-key"

email.from="[email protected]"
email.host="email-smtp.us-east-1.amazonaws.com"
email.port=465
email.user="xyz"
email.password="abc"
alert.class_names = ["rearview.alert.LiveEmailAlert"]

with these configs rearview does not send emails on alerts. I tried on both ports i.e 25 and 465. Even telnet to email-smtp.us-east-1.amazonaws.com 465 was successful.

Am I missing anything ?

I'm using postgres 9.1 as my database:
/bin/setup
....
Setting up databases...
== BaseSchema: migrating =====================================================
-- create_table("applications", {:force=>true})
-> 0.0220s
-- add_index("applications", ["ancestry"], {:name=>"index_applications_on_ancestry"})
-> 0.0060s
-> 0 rows
-- add_index("applications", ["user_id"], {:name=>"user_id"})
-> 0.0190s
-> 0 rows
-- create_table("job_data", {:force=>true})
rake aborted!
An error has occurred, this and all later migrations canceled:

The limit on text can be at most 1GB - 1byte./home/rearview/.rvm/gems/jruby-1.7.10/gems/activerecord-jdbc-adapter-1.3.4
/lib/arjdbc/postgresql/adapter.rb:126:in `type_to_sql'

Probably I misunderstood something, but why I need those default alert URI if it not used? Alerts are never sent using the data from default alert URIs, no?

In writing my first monitors, I am not getting any data. Upon running tshark to watch the outgoing queries to graphite I saw this:

330.465848    10.0.2.15 -> 10.11.12.28  HTTP GET /render?from=22:40_20140128&until=22:41_20140128&format=raw&target=alias(stats.gauges.pg.9_1.ca-blah.database.blah_prod_ca.active_sessions,%22active_sessions%22) HTTP/1.1

The target took my literal input into the target field in the web ui. Also, the timestamps provided seem to be in UTC, but I'm in UTC-6, rendering them futuristic, even though is specified the bounds as 1min and now.

The database setup step of bin/setup from a clean rvm gemset and jruby 1.7.5 throws the following exception:

D, [2014-01-12T11:43:47.120000 #44332] DEBUG -- : Shutdown completed cleanly
LoadError: load error: /Users/khash/work/readview/config/environment -- java.lang.ClassNotFoundException: javax/crypto/JceSecurity
               require at org/jruby/RubyKernel.java:1082
               require at /Users/khash/.rvm/rubies/jruby-1.7.5/lib/ruby/gems/shared/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:229
       load_dependency at /Users/khash/.rvm/rubies/jruby-1.7.5/lib/ruby/gems/shared/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:214
               require at /Users/khash/.rvm/rubies/jruby-1.7.5/lib/ruby/gems/shared/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:229
  require_environment! at /Users/khash/.rvm/rubies/jruby-1.7.5/lib/ruby/gems/shared/gems/railties-4.0.2/lib/rails/application.rb:189
                (root) at /Users/khash/.rvm/rubies/jruby-1.7.5/lib/ruby/gems/shared/gems/railties-4.0.2/lib/rails/commands.rb:44
               require at org/jruby/RubyKernel.java:1082
                (root) at bin/rails:12

bundle install
Gemfile syntax error compile error
on line 44: syntax error, unexpected ':', expecting kEND
gem 'sdoc', require: false
^
how to resolve this problem?

Failed to store user
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'created' cannot be null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.6.0_27]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) ~[na:1.6.0_27]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.6.0_27]
at java.lang.reflect.Constructor.newInstance(Constructor.java:534) ~[na:1.6.0_27]
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) ~[mysql-connector-java-5.1.21.jar:na]
at com.mysql.jdbc.Util.getInstance(Util.java:386) ~[mysql-connector-java-5.1.21.jar:na]

Hello there! I seem to be facing a problem of my own design as Rearview::MonitorService is crashing on startup with Celluloid::DeadTaskError as the only clue:

2014-05-16_03:52:52.47370 [JRubyWorker-1] INFO / - #<Rearview::MonitorService:0x2d858dff> starting up service...
2014-05-16_03:52:52.47371
2014-05-16_03:52:52.48289 [RubyThread-13: /data/app/rearview/installs/rearview_bba883c9d4335407b321a9420f047a008aa1d173/vendor/bundle/jruby/1.9/gems/celluloid-0.14.1/lib/celluloid/internal_pool.rb:56] INFO / - Rearview::MonitorService crashed!
2014-05-16_03:52:52.48291 Celluloid::DeadTaskError: cannot resume a dead task (dead fiber called)
2014-05-16_03:52:52.48292       /data/app/rearview/installs/rearview_bba883c9d4335407b321a9420f047a008aa1d173/vendor/bundle/jruby/1.9/gems/celluloid-0.14.1/lib/celluloid/tasks/task_fiber.rb:25:in `deliver'
2014-05-16_03:52:52.48293       /data/app/rearview/installs/rearview_bba883c9d4335407b321a9420f047a008aa1d173/vendor/bundle/jruby/1.9/gems/celluloid-0.14.1/lib/celluloid/tasks.rb:69:in `resume'
2014-05-16_03:52:52.48294       /data/app/rearview/installs/rearview_bba883c9d4335407b321a9420f047a008aa1d173/vendor/bundle/jruby/1.9/gems/celluloid-0.14.1/lib/celluloid/responses.rb:11:in `dispatch'
2014-05-16_03:52:52.48294       /data/app/rearview/installs/rearview_bba883c9d4335407b321a9420f047a008aa1d173/vendor/bundle/jruby/1.9/gems/celluloid-0.14.1/lib/celluloid/actor.rb:331:in `handle_message'
2014-05-16_03:52:52.48295       /data/app/rearview/installs/rearview_bba883c9d4335407b321a9420f047a008aa1d173/vendor/bundle/jruby/1.9/gems/celluloid-0.14.1/lib/celluloid/actor.rb:174:in `run'
2014-05-16_03:52:52.48297       /data/app/rearview/installs/rearview_bba883c9d4335407b321a9420f047a008aa1d173/vendor/bundle/jruby/1.9/gems/celluloid-0.14.1/lib/celluloid/actor.rb:157:in `initialize'
2014-05-16_03:52:52.48299       /data/app/rearview/installs/rearview_bba883c9d4335407b321a9420f047a008aa1d173/vendor/bundle/jruby/1.9/gems/celluloid-0.14.1/lib/celluloid/thread_handle.rb:13:in `initialize'
2014-05-16_03:52:52.48300       org/jruby/RubyProc.java:249:in `call'
2014-05-16_03:52:52.48300       /data/app/rearview/installs/rearview_bba883c9d4335407b321a9420f047a008aa1d173/vendor/bundle/jruby/1.9/gems/celluloid-0.14.1/lib/celluloid/internal_pool.rb:59:in `create'
2014-05-16_03:52:52.48301
2014-05-16_03:52:52.56617 [main] INFO / - An exception happened during JRuby-Rack startup

I'm pretty surprised Celluloid isn't logging some other error here. Also if I manually try to run the same code from the Rails console, it works, which is rather perplexing.

Anyway, as Celluloid's author I really hate seeing hard-to-debug scenarios like this and sure would love to make it easier to figure out what's wrong, particularly since this problem is a roadblock for me personally! 😄

For metrics that have a "raise", highlight the data points in some manner in the graph to show that an alert/exception was thrown at that point. Not sure of the manner of the highlight, blinking would probably be bad annoying :)

Hello,

This product looks great, thanks so much for open sourcing it . I want to deploy this for a few members of my team to play with, but I am not sure how best to deploy it. is the ./sbc start recommended just for development or is that fine for a staging deployment for my team ? Sorry if this is a dumb question, I am just not very familiar w/ Scala so I wanted to make sure my team gets the best experience

Thanks

-John

Graphite allows selecting multiple series using {...}. A useful example of this syntax is given in the documentation for the averageSeriesWithWildcards function

When trying to use this syntax in Rearview, it shows a flash that says "Metrics contains an invalid metric".

Hi,

Can someone help me understand the significance/purpose of 'job_errors' table ?

Is there a way to compute no. of alerts triggered by a monitor over a time interval ?
This interval can be a day, a week or a month.

I feel this can be achieved from 'job_errors' table because I see entries in this table with status 'failed'. But this does not convince me much because in my case the no. of alerts generated by a monitor till now are more than the no. of entries in 'job_errors' with status='failed' for that monitor.

I have been digging into code/logs to understand the purpose of 'Job_errors' table but have not reached to any conclusion yet.

Thanks,