Git Product home page Git Product logo

puppet-fluentd's Introduction

puppet-fluentd

Build Status

Manage Fluentd installation and configuration with Puppet using the td-agent.

Used Modules

Todo's

  • No RedHat suport yet (THX to pranav )
  • Automatic installation of td-agent Plugins (THX to darktim )
  • Ouput copy and roundrobin to multiple stores
  • Monitor/Restart Service (THX to darktim )
  • Logrotate td-agent logs

Configuration

How to Configure a Agent to send Data to a centralised Fluentd-Server

Create a Agent

  include ::fluentd
  
  fluentd::configfile { 'apache': }
  fluentd::source { 'apache_main': 
    configfile => 'apache'
    type => 'tail',
    format => 'apache2',
    tag => 'apache.access_log',
    config => {
      'path' => '/var/log/apache2/access.log',
      'pos_file' => '/var/tmp/fluentd.pos',
    }
  }
  
  fluentd::configfile { 'syslog': }
  fluentd::source { 'syslog_main': 
    configfile => 'syslog',
    type => 'tail',
    format => 'syslog',
    tag => 'system.syslog',
    config => {
      'path' => '/var/log/syslog',
      'pos_file' => '/tmp/td-agent.syslog.pos',
    }
  }
  
  fluentd::configfile { 'forward': }
  fluentd::match { 'forward_main': 
    configfile => 'forward'
    pattern => '**',
    type => 'forward',
    servers => [
      {'host' => 'PUT_YOUR_HOST_HERE', 'port' => '24224'}
    ],
  }

creates on the agent side following files :

/etc/td-agent/
  ├── config.d
  │   ├── collector.conf
  │   ├── forward.conf
  │   └── syslog.conf
  ├── ...
  ...

Create a Collector

  include ::fluentd

  fluentd::configfile { 'forward': }
  fluentd::source { 'forward_collector': 
    configfile => 'forward'
    type => 'forward',
  #  config => {
  #    'port' => '24224',
  #    'bind' => '0.0.0.0',
  #  }
  }

  fluentd::match { 'forward_apache': 
    configfile => 'forward'
    pattern => '**',
    type => 'elasticsearch',
    config => {
      'logstash_format' => 'true',
    }
  }

creates on the agent side following files :

/etc/td-agent/
  ├── config.d
  │   └── forward.conf
  ├── ...
  ...

puppet-fluentd's People

Contributors

mms-gianni avatar mms-bruno avatar claudio-walser avatar seraf avatar awisrf avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.