I seem stuck on this.

/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/pencil-0.2.10/lib/models/graph.rb:302:in block in expand': undefined methodfirst' for "jvm.gc.PS-MarkSweep.runs":String (NoMethodError)

Graph is the most basic:

:graphs:
JVM1:
title: "JVM"
targets:
jvm.gc.PS-MarkSweep.runs:
🔑 key
:color: color

:dashboards:
JVM Notes:
title: JVM
graphs:
- JVM1:

I`m trying to build a CPU usage graph, and I can't figure out how to run derivative on the series.
can you please improve the doc to include such an example, its by far the most commonly used function in graphite.

Hi,

I have taken the examples and broken it down to just trying to get cpu info out of graphite.

I don't seem to be able to return any hosts, even when using the wildcard ['*']. The dropdown is empty.

Do I need to do anything special to enable discovery?

graphs.yml

:graphs:
load_average:
title: "1 minute load average"
targets:
system.load.load.shortterm:
🔑 load
hosts: ["*"]

dashboard.yml

:dashboards:
test:
title: "load_dash"
graphs:
- load_average:
hosts: [ "*" ]

Hello,

I am having problems making a basic uptime graph using Pencil rewrite;

The config is as follows;
scale(X_mail_zen_net_uk.uptime.uptime,0.000011574)

-- mode: ruby --

title 'Uptime'
description 'Uptime'

field :uptime,
:alias => "Uptime",

:data => "scale(uptime.uptime, 0.000011574)"

This results in the following being called
scale(averageSeries(X_mail_zen_net_uk.uptime.uptime,0.000011574))

This appears to be from the none optional aggregator key.

Can you think of anyway to either make aggregator optional or alternatives if I am doing it wrong?

x.x.x.x- - [07/May/2013:11:07:55 +0000] "GET /render/?title=Uptime+%2F+x_mail_zen_net_uk&from=-1h&width=1517&height=400&until=now&areaMode=none&yMin=0&fontSize=15&tz=Europe%2FLondon&target=alias%28scale%28averageSeries%28x_mail_zen_net_uk.uptime.uptime%29%2C+averageSeries%28x_mail_zen_net_uk.0.000011574%29%29%2C%22x_mail_zen_net_uk+Uptime%22%29 HTTP/1.1" 500 967 "http://localhost:8080/host/global/x_mail_zen_net_uk" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0) Gecko/20100101 Firefox/20.0"

once i'm looking at:

http://pencil/host/site/hostname#graph

i'd like to be able to make all the other graphs on the page go away by clicking on the graph, so that i can play with the pencil leftnav "time window" options without having pages of graphs reload every time and obstruct my view of the data.

or, in short form, i'd to click on the graph and be taken to:

http://pencil/host/site/hostname/graph

(note the # -> / change)

I'd like to be able to group different categories of graphs into multiple files rather than having a single monolithic file. Thanks!

In lib/models/graph.rb it looks like maybe this is a work in progress:
# fixme proper labeling... maybe
# technically we do know, so this can be fixed

My use case is that I want to graph the top ten tables by insert. My metrics looks like cluster1.host1.table1.insert_count and I want to graph highestAverage(cluster1.host1.*.insert_count, 10) but the legend doesn't tell me what those top ten tables are.

Thanks!

Hi,

I am eagerly trying to get Pencil up and running. Thanks for the good work.

I tried both the original Pencil (0.3.1) as well as the "rewrite" (0.4.0a). The problem I am facing is to get Pencil display existing metrics which are not in the format that Pencil expects i.e they don't have .. format.

Lets say I wanted to display the memUsage stock metric that is exposed by Graphite's carbon instance out-of-the-box. Its format is carbon.agents..memUsage.

How do I configure metric_format to correctly fetch and render the above metric ?

Inside pencil.yml , I changed

:metric_format: "%m.%c.%h" #%m metric, %c cluster, %h host

to

:metric_format: "%m" #%m metric

But that throws up this error when I start Pencil:

pencil

I, [2012-11-23T18:37:14.786240 #20554] INFO -- : no :manifest key for webapp, using default
missing host (%h) in :metric_format

Please see #9 (comment)

As noted there, %c (and %h too) needs to be made optional.
It'd be great if Pencil does not enforce any metric format so that existing metrics can readily imported.
Not sure how various features of Pencil are coupled with the %c,%h assumption though.

But right now, the %h is holding me from using Pencil !

From the examples:
syncsreg_exceptions:
title: "syncsreg exceptions"
targets:
syncsreg.log.exception:
🔑 exceptions/s
:color: red
hosts: ["wp-sreg*"]

In our environment, we don't have hosts named "wp-sreg_", we have hosts named aaa1,bbb2,aaa2, which might all live in a cluster called "wp-sreg". Please add support for a clusters: key so in the above example I could use 'clusters: ["wp-sreg_"]' to determine what graphs and dashboards are applicable to what groups of hosts.

Thanks!

Hello,

No matter what I do I can not get Pencil to find hosts in the dashboard. It does however know about graph data.

The dashboard shows 2 hosts of RX and TX which are not the hosts, but the metric data.

I am using the latest rewrite version and the config is as follows;

pencil.yaml
:metric_format: "%h.%m" #%m metric, %c cluster, %h host

svrs1.yml

title: Bastion01X overview
graphs:

• network_traffic
  hosts:
• bastion01*

network.graph

-- mode: ruby --

title 'network traffic'

field :network_in,
:alias => "bits in",
:color => 'green',
:scale => 8,
:data => "interface-eth0.if_octets.rx"

field :network_out,
:alias => "bits out",
:color => 'blue',
:scale => 8,
:data => "interface-eth0.if_octets.tx"

http://localhost/metrics/expand/?leavesOnly=1&query=bastion0*.interface-eth0.if_octets.rx

{"results": ["bastion01a_mail_X_net_uk.interface-eth0.if_octets.rx", "bastion01b_mail_X_net_uk.interface-eth0.if_octets.rx", "bastion01c_mail_X_net_uk.interface-eth0.if_octets.rx", "bastion01d_mail_X_net_uk.interface-eth0.if_octets.rx"]}

Hello,

One problem I have with Pencil is it seems to freeze the Firefox browser on a regular basis.

I had this problem several months ago when using it and the issue appears to be still around.
Using rewrite branch. Unsure what debugging I can do with Firefox

The metric_format in pencil.yaml doesn't seem to affect the generated image URLs. In stock trim (metric_format: "%m.%c.%h) I get targets like:

apb00.email.emailSent.15MinuteRate.{}.{apb*}

With metric_format swapped around to "%c.%h.%m" or "%c.%m.%h", the target is the same.

First of all, I must say I like pencil.
I looked at all (I think) of the other dashboards, and pencil is the only one that comes close to what I`m after.
I was almost tempted to start yet another dashboard :)

now, to the point:
pencil is not dynamic enough:

1. changes to yaml graph/dashboard files only takes effect after restarting pencil.
2. new servers only appear in the dropdown after a pencil restart.

as for the configuration files:
its worth checking the last modified and automatically reload them when they change.

regarding the servers list, I think you should timeout the cache after a while.

Hi,
I have the a problem with a combined graph (cluster view) of a series I need to run through derivative.
The correct form should be:

sumSeries(derivative({web*}.network.netstat.active)

but instead pencil creates a graph using

derivative(sumSeries({web*}.network.netstat.active))

Since the counters on the web servers are not aligned, you get really messed up graphs from the second call.
This is my netstat graph definition, is there anything I can do to get a correct graph?

  netstat:
    title: Netstat
    targets:
      network.netstat.active:
        !omap
        - :color: orange
        - :derivative:
        - :key: Active
      network.netstat.established:
        !omap
        - :color: green
        - :derivative:
        - :key: Established
      network.netstat.failed:
        !omap
        - :color: red
        - :derivative:
        - :key: Failed
      network.netstat.passive:
        !omap
        - :color: blue
        - :derivative:
        - :key: Passive
      network.netstat.reset:
        !omap
        - :color: purple
        - :derivative:
        - :key: Reset
    hosts: [*]

#! /bin/sh
### BEGIN INIT INFO
# Provides:          pencil
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: pencil graphite frontend
# Description:       Starts pencil graphite frontend
### END INIT INFO

# Author: Omry Yadan

# Source init-functions:
#source /lib/lsb/init-functions
. /lib/lsb/init-functions

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

# Path to Executable
DAEMON=/var/lib/gems/1.8/gems/pencil-0.2.10/bin/pencil

PIDFILE=/var/run/pencil.pid

SCRIPTNAME=/etc/init.d/pencil

ARGS="-d /etc/pencil"

# Exit if the package is not installed
if [ ! -x "$DAEMON" ]; then {
    echo "Couldn't find $DAEMON or not executable"
    exit 99
}
fi

# Load the VERBOSE setting and other rcS variables
[ -f /etc/default/rcS ] && . /etc/default/rcS

#
# Function that starts the daemon/service
#
do_start()
{
    # Return
    #   0 if daemon has been started
    #   1 if daemon was already running
    #   2 if daemon could not be started

        # Test to see if the daemon is already running - return 1 if it is. 
    start-stop-daemon --start --pidfile $PIDFILE \
        --exec $DAEMON --test -- start $ARGS > /dev/null || return 1

        # Start the daemon for real, return 2 if failed
    start-stop-daemon --background --start --make-pidfile --pidfile $PIDFILE --exec $DAEMON $ARGS || return 2
}

#
# Function that stops the daemon/service
#
do_stop() {
    # Return
    #   0 if daemon has been stopped
    #   1 if daemon was already stopped
    #   2 if daemon could not be stopped
    #   other if a failure occurred
    log_daemon_msg "Stopping $DESC" "$NAME"
    start-stop-daemon --stop --signal 2 --retry 5 --quiet --pidfile $PIDFILE
    RETVAL="$?"
    [ "$RETVAL" = 2 ] && return 2

        # Delete the exisitng PID file
    if [ -e "$PIDFILE" ]; then {
        rm $PIDFILE
    }  
        fi

        return "$RETVAL"
}


# Display / Parse Init Options
case "$1" in
  start)
     [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
      do_start
      case "$?" in
        0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
        2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
      esac
  ;;
  stop)
     [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
      do_stop
      case "$?" in
        0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
        2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
      esac
  ;;
  restart)
    log_daemon_msg "Restarting $DESC" "$NAME"
        do_stop
          case "$?" in
            0|1)
            do_start
            case "$?" in
              0) log_end_msg 0 ;;
              1) log_end_msg 1 ;; # Old process is still running
              *) log_end_msg 1 ;; # Failed to start
            esac
       ;;
    *)
      # Failed to stop
    log_end_msg 1
    ;;
  esac
  ;;
  status)
      if [ -s $PIDFILE ]; then
          pid=`cat $PIDFILE`
          kill -0 $pid >/dev/null 2>&1
          if [ "$?" = "0" ]; then
              echo "$NAME is running: pid $pid."
              RETVAL=0
          else
              echo "Couldn't find pid $pid for $NAME."
              RETVAL=1
          fi
      else
          echo "$NAME is stopped (no pid file)."
          RETVAL=1
      fi
      return $RETVAL
  ;;
  *)
  echo "Usage: $SCRIPTNAME {start|stop|restart|status}" >&2
  exit 3
  ;;
esac
:

Hi,

I am using Pencil (rewrite branch 0.4.0a ) and have built and installed the Pencil gem properly.

I have a test metric up that shows up in Graphite's webUI and I tried to pull the same into Pencil.
The test metric is in the format mysrv.server.testmetric where mysrv is the name of the host.

When I browse the Pencil home page at :9292 , I get the below error :

pencil

I, [2012-11-29T07:49:00.916454 #17168] INFO -- : no :manifest key for webapp, using default
== Sinatra/1.3.3 has taken the stage on 9292 for development with backup from Mongrel
192.168.114.58 - - [29/Nov/2012 07:49:07] "GET / HTTP/1.1" 302 - 0.0092
NoMethodError - undefined method each' for nil:NilClass: /usr/lib/ruby/gems/1.8/gems/pencil-0.4.0a/lib/../views/global.erb:43:inevaluate_source'
/usr/lib/ruby/gems/1.8/gems/pencil-0.4.0a/lib/../views/global.erb:37:in each' /usr/lib/ruby/gems/1.8/gems/pencil-0.4.0a/lib/../views/global.erb:37:inevaluate_source'
/usr/lib/ruby/gems/1.8/gems/tilt-1.3.3/lib/tilt/template.rb:144:in cached_evaluate' /usr/lib/ruby/gems/1.8/gems/tilt-1.3.3/lib/tilt/template.rb:127:inevaluate'
/usr/lib/ruby/gems/1.8/gems/tilt-1.3.3/lib/tilt/template.rb:76:in render' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:686:inrender'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:572:in erb' /usr/lib/ruby/gems/1.8/gems/pencil-0.4.0a/lib/pencil.rb:120:inGET /dash/:cluster/?'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:1264:in call' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:1264 /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:835:in[]'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:835:in route!' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:851:inroute_eval'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:835:in route!' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:872:inprocess_route'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:870:in catch' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:870:inprocess_route'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:834:in route!' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:833:ineach'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:833:in route!' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:936:indispatch!'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:769:in call!' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:921:ininvoke'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:921:in catch' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:921:ininvoke'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:769:in call!' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:755:incall'
/usr/lib/ruby/gems/1.8/gems/rack-1.4.1/lib/rack/logger.rb:15:in call' /usr/lib/ruby/gems/1.8/gems/rack-protection-1.2.0/lib/rack/protection/xss_header.rb:22:incall'
/usr/lib/ruby/gems/1.8/gems/rack-protection-1.2.0/lib/rack/protection/path_traversal.rb:16:in call' /usr/lib/ruby/gems/1.8/gems/rack-protection-1.2.0/lib/rack/protection/json_csrf.rb:17:incall'
/usr/lib/ruby/gems/1.8/gems/rack-protection-1.2.0/lib/rack/protection/base.rb:47:in call' /usr/lib/ruby/gems/1.8/gems/rack-protection-1.2.0/lib/rack/protection/xss_header.rb:22:incall'
/usr/lib/ruby/gems/1.8/gems/rack-1.4.1/lib/rack/logger.rb:15:in call' /usr/lib/ruby/gems/1.8/gems/rack-1.4.1/lib/rack/commonlogger.rb:20:incall_without_check'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:136:in call' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:129:incall'
/usr/lib/ruby/gems/1.8/gems/rack-1.4.1/lib/rack/head.rb:9:in call' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/showexceptions.rb:21:incall'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:99:in call' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:1389:incall'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:1471:in synchronize' /usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:1389:incall'
/usr/lib/ruby/gems/1.8/gems/rack-1.4.1/lib/rack/handler/mongrel.rb:77:in process' /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:159:inprocess_client'
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:158:in each' /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:158:inprocess_client'
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:285:in run' /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:285:ininitialize'
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:285:in new' /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:285:inrun'
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:268:in initialize' /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:268:innew'
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.5/lib/mongrel.rb:268:in run' /usr/lib/ruby/gems/1.8/gems/rack-1.4.1/lib/rack/handler/mongrel.rb:38:inrun'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.3.3/lib/sinatra/base.rb:1350:in run!' /usr/lib/ruby/gems/1.8/gems/pencil-0.4.0a/bin/pencil:4 /usr/bin/pencil:23:inload'
/usr/bin/pencil:23

The Sinatra debug error page shows up with something like :

NoMethodError at /dash/global
undefined method `each' for nil:NilClass
file: global.erb location: evaluate_source line: 43

My pencil.yml is

cat pencil.yml

:graphite_url: http:///graphite
:port: 9292
:host_sort: "sensible"
:metric_format: "%h.%m" #%m metric, %h host
:templates_dir: "./conf"
:refresh_rate: 60 # refresh rate for images in seconds

careful! these are properties as specified by the graphite-dsl now

:default_url_opts:
:width: 1400
:height: 400
:fontsize: 15
:template: "noc"
:ymin: 0
:margin: 5
:thickness: 2

:webapp: true
:default_views:

• default:
  -1h: "past hour"
• -8h
• -24h: "past 24 hours"

My conf/global/testmet.graph :

-- mode: ruby --

title 'mysrv test metric'
area :stacked

field :testmetric,
:alias => "testmetric",
:color => 'yellow',
:data => "server.testmetric"

As per the wiki guidelines on configuration , I left the data unqualified i.e I omitted that host details. IOW, instead of
:data => "mysrv.server.testmetric"
I gave
:data => "server.testmetric"

Any pointers on what could be wrong in this setup ?

Hey,

I heard that Pencil is great at DevOpsDays. It would be helpful if there was a sample config file to help get things going. If you can add that, I'll try to pull together a basic README. Thanks,

-nick

You know. For increased awesomeness.