Git Product home page Git Product logo

pagerduty-icinga-pl's Introduction

Introduction

This guide describes how to integrate your Icinga installation with PagerDuty using a simple Perl-based plugin. As Icinga is a fork of Nagios, you'll use PagerDuty's built in support for Nagios for this integration.

Getting Started

If you don't already have a PagerDuty "Nagios" service, you should create one:

  • In your account, under the Services tab, click "Add New Service".
  • Enter a name for the service and select an escalation policy. Then, select "Nagios" for the Service Type.
  • Click the "Add Service" button.
  • Once the service is created, you'll be taken to the service page. On this page, you'll see the "Service key", which will be needed when you configure your Icinga server to send events to PagerDuty.

Setup

Install the necessary Perl dependencies:

For Debian, Ubuntu, and other Debian-derived systems:

aptitude install libwww-perl libcrypt-ssleay-perl

RHEL, Fedora, CentOS, and other Redhat-derived systems:

yum install perl-libwww-perl perl-Crypt-SSLeay

Download pagerduty_icinga.cfg:

wget https://raw.github.com/PagerDuty/pagerduty-icinga-pl/master/pagerduty_icinga.cfg

Open the file in your favorite editor. Enter the service key corresponding to your Nagios/Icinga service into the pager field. The service key is a 32 character string that can be found on the service's detail page.

Copy the Icinga configuration file into place and change owner:

cp pagerduty_icinga.cfg /usr/local/icinga/etc/objects
chown icinga:icinga /usr/local/icinga/etc/objects/pagerduty_icinga.cfg

Add the contact "pagerduty" to your Icinga configuration's main contact group. If you're using the default configuration, open /usr/local/icinga/etc/objects/contacts.cfg and look for the "admins" contact group. Then, simply add the "pagerduty" contact.

define contactgroup{ 
  contactgroup_name       admins
  alias                   Icinga Administrators
  members                 icingaadmin,pagerduty   ; <-- Add 'pagerduty' here.
}

Download pagerduty_icinga.pl and copy it to /usr/local/bin:

wget https://raw.github.com/PagerDuty/pagerduty-icinga-pl/master/pagerduty_icinga.pl
cp pagerduty_icinga.pl /usr/local/bin

Make sure the file is executable by Icinga

chmod 755 /usr/local/bin/pagerduty_icinga.pl

Enable environment variable macros in /usr/local/icinga/icinga.cfg (if not enabled already):

enable_environment_macros=1

Edit the icinga user's crontab:

crontab -u icinga -e

Add the following line to the crontab:

* * * * * /usr/local/bin/pagerduty_icinga.pl flush

Restart Icinga:

/etc/init.d/icinga restart

#License and Copyright Copyright (c) 2014, PagerDuty All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

  • Neither the name of [project] nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

pagerduty-icinga-pl's People

Contributors

arupchak avatar florinasavoaie avatar ienders avatar mdcollins05 avatar ryanhoskin avatar ziaunys avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pagerduty-icinga-pl's Issues

Create system packages

It would be nice if this script was packaged in order to automatically notify users of updates. Without a package users have to manually update this script or do some non-standard package management. In addition to that, this script has dependencies which could be resolved by creating a system package.

Is this still maintained?

We're currently migrating our use of the v1 REST API to v2, and I see that this script hasn't been touched since 2015. However, it looks to be using the events API, not the REST API, so it's not an immediate problem.

Thanks

How to set Priority to Icinga alerts on Pagerduty Incidents

I have integrated Icinga and Pagerduty and Icinga alerts are reported as Incidents on PD.
I want to set Priority to the Incidents from Icinga.
image

In pagerduty-icinga2.conf , I have added Priority
env = {
"ICINGA_CONTACTPAGER" = "$user.pager$"
"ICINGA_NOTIFICATIONTYPE" = "$notification.type$"
"ICINGA_SERVICEDESC" = "$service.name$"
"ICINGA_HOSTNAME" = "$host.name$"
"ICINGA_HOSTALIAS" = "$host.display_name$"
"ICINGA_SERVICESTATE" = "$service.state$"
"ICINGA_SERVICEOUTPUT" = "$service.output$"
"ICINGA_PRIORITY" = "SEV2"
}

but Priority is added as custom variable in the Incident. Please suggest.

Perl script blocking itself on multiple icinga events

When Icinga triggers multiple issues, the NotificationCommand "notify-service-by-pagerduty" fires multiples times.
One of the calls makes it, locking / blocking on file: /tmp/pagerduty/lockfile
All of the other instances of notify-service-by-pagerduty fail, with their shell script exiting on the following error:

/var/log/icinga/icinaga.log:

[2016-02-18 13:22:38 -0800] warning/PluginNotificationTask: Notification command for object 'celli.sports-it.com!apt' (PID: 15295, arguments: 'sh' '-c' '/usr/local/bin/pagerduty_icinga.pl enqueue -f pd_nagios_object=service') terminated with exit code 11, output: pagerduty_icinga[15297]: flock /tmp/pagerduty_icinga/lockfile failed: Resource temporarily unavailable
Resource temporarily unavailable at /usr/local/bin/pagerduty_icinga.pl line 221.

/var/log/syslog:

pagerduty_icinga[15297]: flock /tmp/pagerduty_icinga/lockfile failed: Resource temporarily unavailable

This happens because each icinga event triggers an enqueue on pagerduty_icinga.pl, which internally calls (or tries to call) the method 'lock_and_flush_queue'. Only one instance gets the successful locks, the others are blocked.

This is not a fatal issue. If I have my cron job setup correctly, 1 minute later, the other entries will be called when 'pagerduty_icinga.pl flush' is called.
However, this is still not ideal. The pagerduty_icinga.pl enqueue process should either only enqueue (without attempting flush, and thus blocking itself) or it should implement some passive check timeout / keepalive option in the pearl script for the 'lock_and_flush_queue' section.

These processes finish almost immediately, so a keepalive would only need to be a matter of a few seconds, after which the calls could still be allowed to fail out, there would just now be a small buffer / threshold were multiple calls could be made successively.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.