Git Product home page Git Product logo

voxpupuli / puppet-splunk Goto Github PK

View Code? Open in Web Editor NEW
41.0 145.0 121.0 830 KB

Manage Splunk servers and forwarders using Puppet

Home Page: https://forge.puppet.com/puppet/splunk

License: Apache License 2.0

Ruby 57.42% Puppet 42.49% Pascal 0.09%
linux-puppet-module puppet windows-puppet-module hacktoberfest bsd-puppet-module centos-puppet-module debian-puppet-module freebsd-puppet-module redhat-puppet-module ubuntu-puppet-module

puppet-splunk's Introduction

Puppet Module For Splunk

CI Code Coverage Puppet Forge Puppet Forge - downloads Puppet Forge - endorsement Puppet Forge - scores

Table of Contents

  1. Overview
  2. Module Description - What the module does and why it is useful
  3. Setup - The basics of getting started with splunk
  4. Usage - Configuration options and additional functionality
  5. Reference - An under-the-hood peek at what the module is doing and how
  6. Limitations - OS compatibility, etc.
  7. Development - Guide for contributing to the module

Overview

This module provides a method to deploy Splunk Enterprise or Splunk Universal Forwarder with common configurations and ensure the services maintain a running state. It provides types/providers to interact with the various Splunk/Forwarder configuration files.

Module Description

This module does not configure firewall rules. Firewall rules will need to be configured separately in order to allow for correct operation of Splunk and the Splunk Universal Forwarder.

Setup

What splunk affects

  • Installs the Splunk/Forwarder package and manages their config files. It does not purge them by default.
  • The module will set up both Splunk Enterprise and Splunk Forwarder to run as the 'root' user on POSIX platforms.
  • By default, enables Splunk Enterprise and Splunk Forwarder boot-start, and uses the vendor-generated service file to manage the splunk service.

Setup Requirements

To begin using this module, use the Puppet Module Tool (PMT) from the command line to install this module:

puppet module install puppet-splunk

This will place the module into your primary module path if you do not utilize the --target-dir directive.

You can also use r10k or code-manager to deploy the module so ensure that you have the correct entry in your Puppetfile.

By default, this module will download the installation packages from https://download.splunk.com.

You can also configure offline installers, there is just a little setup needed.

First, you will need to place your downloaded splunk installers into the files directory, <module_path>/splunk/files/. If you're using r10k or code-manager you'll need to override the splunk::params::src_root parameter to point at a modulepath outside of the Splunk module because each deploy will overwrite the files.

The files must be placed according to directory structure example given below.

The expected directory structure is:

 $root_url/
 └── products/
     ├── universalforwarder/
     │   └── releases/
     |       └── $version/
     |           └── $platform/
     |               └── splunkforwarder-${version}-${build}-${additl}
     └── splunk/
         └── releases/
             └── $version/
                 └── $platform/
                     └── splunk-${version}-${build}-${additl}

A semi-populated example files directory might then contain:

$root_url/
└── products/
    ├── universalforwarder/
    │   └── releases/
    |       └── 9.2.0/
    |           ├── linux/
    |           |   ├── splunkforwarder-9.2.0-1fff88043d5f-linux-2.6-amd64.deb
    |           |   ├── splunkforwarder-9.2.0-1fff88043d5f-linux-2.6-intel.deb
    |           |   └── splunkforwarder-9.2.0-1fff88043d5f.x86_64.rpm
    |           ├── solaris/
    |           └── windows/
    |               └── splunkforwarder-9.2.0-1fff88043d5f-x64-release.msi
    └── splunk/
        └── releases/
            └── 9.2.0/
                └── linux/
                    ├── splunk-9.2.0-1fff88043d5f-linux-2.6-amd64.deb
                    ├── splunk-9.2.0-1fff88043d5f-linux-2.6-intel.deb
                    └── splunk-9.2.0-1fff88043d5f.x86_64.rpm

Second, you will need to supply the splunk::params class with three critical pieces of information.

  • The version of Splunk you are using
  • The build of Splunk you are using
  • The root URL to use to retrieve the packages

In the example given above, the version is 9.2.0, the build is 1fff88043d5f, and the root URL is puppet:///modules/splunk. See the splunk::params class documentation for more information.

Beginning with splunk

Once the Splunk packages are hosted in the users repository or hosted by the Puppet Server in the modulepath the module is ready to deploy.

Usage

If a user is installing Splunk Enterprise with packages provided from their modulepath, this is the most basic way of installing Splunk Server with default settings:

include splunk::enterprise

This is the most basic way of installing the Splunk Universal Forwarder with default settings:

class { 'splunk::params':
    server => $my_splunk_server,
}

include splunk::forwarder

Once both Splunk Enterprise and Splunk Universal Forwarder have been deployed on their respective nodes, the Forwarder is ready to start sending logs.

In order to start sending some log data, users can take advantage of the Splunkforwarder_input type. Here is a basic example of adding an input to start sending Puppet Server logs:

@splunkforwarder_input { 'puppetserver-sourcetype':
  section => 'monitor:///var/log/puppetlabs/puppetserver/puppetserver.log',
  setting => 'sourcetype',
  value   => 'puppetserver',
  tag     => 'splunk_forwarder'
}

This virtual resource will get collected by the ::splunk::forwarder class if it is tagged with splunk_forwarder and will add the appropriate setting to the inputs.conf file and refresh the service.

Setting the admin user's password

The module has the facility to set Splunk Enterprise's admin password at installation time by leveraging the user-seed.conf method described as a best practice in the Splunk docs. The way Splunk implements this prevents Puppet from managing the password in an idempotent way but makes resetting the password through the web console possible. You can also use Puppet to do a one time reset too by setting the appropriate parameters on splunk::enterprise but leaving these parameters set to true will cause corrective change on each run of the Puppet Agent.

class { 'splunk::enterprise':
  seed_password    => true,
  password_hash    => '$6$jxSX7ra2SNzeJbYE$J95eTTMJjFr/lBoGYvuJUSNKvR7befnBwZUOvr/ky86QGqDXwEwdbgPMfCxW1/PuB/IkC94QLNravkABBkVkV1',
}

Alternatively the splunk::enterprise::password::seed class can be used independently of the Puppet Agent through a Bolt Plan apply block.

Upgrade splunk and splunkforwarder packages

This module has the ability to install and upgrade the splunk and splunkforwarder packages. All you have to do is declare package_ensure => 'latest' when calling the ::splunk or ::splunk::forwarder classes.

Upgrade Example

The following code will install the 9.1.0 version of the splunk forwarder. Then comment out the 9.1.0 version and build values and uncomment the 9.2.0.1 version and build values. Running puppet again will perform the following:

  1. splunk forwarder package is upgraded
    1. splunk service is stopped as part of the package upgrade process
  2. new license agreement is automatically accepted
    1. license agreement must be accepted or the splunk service will fail to start
  3. splunk service is started
# Tell the module to get packages directly from Splunk.
class { 'splunk::params':
  version  => '9.1.0',
  build    => '1c86ca0bacc3',
  #version  => '9.2.0.1',
  #build    => 'd8ae995bf219',
  src_root => 'https://download.splunk.com',
}

# Specifying package_ensure => 'latest' will ensure that the splunk and
# splunkforwarder packages will be upgraded when you specify newer values for
# version and build.
class { 'splunk::forwarder':
  package_ensure => 'latest',
}

Reference

See in file REFERENCE.md.

Limitations

  • Upgrades are tested from Splunk 9.1.0 to 9.2.0.1.
  • New installations of splunk up to version 7.2.X are supported, but upgrades from 7.0.X to >= 7.0.X are not fully tested

Development

Learn how to get involved in this and other Vox Pupuli module development on our docs site.

Release Notes/Contributors/Etc

See the CHANGELOG.md or list of contributors.

puppet-splunk's People

Contributors

alexjfisher avatar bastelfreak avatar bmjen avatar crayfishx avatar davemcdonnell avatar dhoppe avatar ekohl avatar gcoxmoz avatar ghoneycutt avatar gregswift avatar hunner avatar juniorsysadmin avatar kenyon avatar martijndegouw avatar nanliu avatar natemccurdy avatar nicholaspearson avatar ody avatar ralfbosz avatar reidmv avatar roidelapluie avatar root-expert avatar siegy22 avatar smortex avatar tamerz avatar tragiccode avatar treydock avatar wyardley avatar ziaunys avatar zilchms avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

puppet-splunk's Issues

Error: Failed to apply catalog: Found 1 dependency cycle:

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: PE 2017.1
  • Ruby: ruby-2.0.0.648
  • Distribution: RHEL 7.3
  • Module version: 2017-01-12 - Release 5.1.0

How to reproduce (e.g Puppet code you use)

class profile::splunk_forwarder {
include ::splunk::forwarder
}

splunk::params::src_root: 'puppet:///software'
splunk::params::version: '6.5.1'
splunk::params::build: 'f74036626f0c'
splunk::forwarder::forwarder_output:
'tcpout_defaultgroup':
section: 'default'
setting: 'defaultGroup'
value: 'default-autolb-group'
tag: 'splunk_forwarder'
'tcpout_default-autolb-group':
section: 'tcpout:default-autolb-group'
setting: 'server'
value: ''
tag: 'splunk_forwarder'

What are you seeing

Error: Failed to apply catalog: Found 1 dependency cycle:
(File[/opt/staging/splunk/splunkforwarder-6.5.1-f74036626f0c-linux-2.6-x86_64.rpm] => Staging::File[splunkforwarder-6.5.1-f74036626f0c-linux-2.6-x86_64.rpm] => Package[splunkforwarder] => File[/opt/staging/splunk/splunkforwarder-6.5.1-f74036626f0c-linux-2.6-x86_64.rpm])
Try the '--graph' option and opening the resulting '.dot' file in OmniGraffle or GraphViz

What behaviour did you expect instead

Splunk Forwarder should install from the software location served from my puppet master and then outputs should be configured.

Output log

Any additional information you'd like to impart

Splunk_TA_ni/bin error

I am getting the following error on splunkd.logs

ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/Splunk_TA_nix/bin/cpu.sh" /opt/splunkforwarder/etc/apps/Splunk_TA_nix/bin/cpu.sh: line 16: /common.sh: No such file or directory

Warning unknown variable 'staged_package' when `yum` is used as package provider

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet 4.10.8:
  • Ruby: 2.1.9p490
  • Distribution: RHEL 6.7
  • Module version: 7.1.0

How to reproduce (e.g Puppet code you use)

  class {'::splunk::forwarder':
    pkg_provider => 'yum',
  }

What are you seeing

Puppet gives the following warning:

Warning: Unknown variable: 'staged_package'. at /etc/puppetlabs/code/environments/production/modules/splunk/manifests/forwarder.pp:93:28

What behaviour did you expect instead

No warning.

Output log

Any additional information you'd like to impart

Ability to manage ui-prefs.conf

It would be nice to have the ability to specify a ui-prefs.conf in order to change the default time range of every user's search to prevent causing inefficient queries on accident and remembering to change the time range for every new search

Declare a monitor resource without any option in inputs.conf

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 3.8.7
  • Ruby: ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
  • Distribution: Linux siem 4.4.0-31-generic #50~14.04.1-Ubuntu SMP Wed Jul 13 01:07:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
  • Module version: 5.0.1

How to reproduce (e.g Puppet code you use)

Try to add a monitor resource without any option in the inputs.conf (or with an empty value)

What are you seeing

The corresponding line is not added

What behaviour did you expect instead

The corresponding line should be added

Output log

Any additional information you'd like to impart

Not working:

splunk::forwarder::forwarder_input  :
    'default_foo':
        section : 'monitor://foo/bar.log'
        tag     : 'splunk_forwarder'

Not working:

splunk::forwarder::forwarder_input  :
    'default_foo':
        section : 'monitor://foo/bar.log'
        setting : 'foo'
        value   : 
        tag     : 'splunk_forwarder'

Working (but I don't want to declare any variable for this monitor entry) :

splunk::forwarder::forwarder_input  :
    'default_foo':
        section : 'monitor://foo/bar.log'
        setting : 'foo'
        value   : ''
        tag     : 'splunk_forwarder'

Working (but I don't want to declare any variable for this monitor entry):

splunk::forwarder::forwarder_input  :
    'default_foo':
        section : 'monitor://foo/bar.log'
        setting : 'foo'
        value   : 'bar'
        tag     : 'splunk_forwarder'

Allow /opt/splunkforwarder/var/run/splunk splunkd.pid to run as splunk

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet:
  • Ruby:
  • Distribution:
  • Module version:

How to reproduce (e.g Puppet code you use)

What are you seeing

I set the user under class ::splunk::forwarder to splunk but still the spunkd.pid runs as root, the customer wants this running as splunk, I made a file resource that recurse /opt/splunkforwarder to run as group and owner root, but the change happens every run with a bunch of others /opt/splunkforwarder/var/libsplunk and /opt/splunkforwarder/var/run/*

Manually the customer set chown /opt/forwarder/* to run as splunk then restarted the service and this worked

What behaviour did you expect instead

Output log

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/etc/apps/learned/local/props.conf]/owner: owner changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/etc/apps/learned/local/props.conf]/group: group changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/etc/apps/learned/local/sourcetypes.conf]/owner: owner changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/etc/apps/learned/local/sourcetypes.conf]/group: group changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/etc/apps/learned/metadata/local.meta]/owner: owner changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/etc/apps/learned/metadata/local.meta]/group: group changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/etc/passwd]/owner: owner changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/etc/passwd]/group: group changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/lib/splunk/fishbucket/4069420869]/owner: owner changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/lib/splunk/fishbucket/4069420869]/group: group changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/lib/splunk/fishbucket/rawdata/1322324208]/owner: owner changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/lib/splunk/fishbucket/rawdata/1322324208]/group: group changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/lib/splunk/fishbucket/rawdata/1322324208.old]/owner: owner changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/lib/splunk/fishbucket/rawdata/1322324208.old]/group: group changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/lib/splunk/fishbucket/rawdata/2342085248]/owner: owner changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/lib/splunk/fishbucket/rawdata/2342085248]/group: group changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/lib/splunk/fishbucket/rawdata/2342085248.old]/owner: owner changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/lib/splunk/fishbucket/rawdata/2342085248.old]/group: group changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/run/splunk/conf-mutator.pid]/owner: owner changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/run/splunk/conf-mutator.pid]/group: group changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/run/splunk/splunkd.pid]/owner: owner changed 'root' to 'splunk'

Notice: /Stage[main]/Profile::Linux_splunk/File[/opt/splunkforwarder/var/run/splunk/splunkd.pid]/group: group changed 'root' to 'splunk'

Notice: Applied catalog in 2.94 seconds

On every run

Any additional information you'd like to impart

My profile

#Linux profile to install Linux Splunk

class profile::linux_splunk {

$artifactory_host = hiera('artifactory_host')

$splunkadmsrv = hiera('splunk::deploysrv')

$splunkdir = hiera('splunk::dir')

$winsplunkport = hiera('splunk::port')

if $facts['apache_version'] {

$client_name = 'fspptuxapch'

acl { '/app/httpd/log/':

    action     => set,

    permission => [

    'group:splunk:r-x',

    'default:group:splunk:r-x',

    ],

    provider   => posixacl,

    recursive  => true,

    require    => Package['splunkforwarder'],

  }

}

else { $client_name = 'fspptux'

}



splunkforwarder_deploymentclient { 'deployment-client-disabled':

section => 'deployment-client',

setting => 'disabled',

value   => '0',

}

splunkforwarder_deploymentclient { 'deployment-client-client-name':

section => 'deployment-client',

setting => 'clientName',

value   => $client_name,

}

splunkforwarder_deploymentclient { 'deployment-server':

section => 'target-broker:deploymentServer',

setting => 'targetUri',

value   => Blank:8089',

}

class { '::splunk::params':

  version      => '6.5.2',

  build        => '67571ef4b87d',

  src_root     => "http://${artifactory_host}/artifactory/application-release-local/gov/usda/fs/busops/cio/FS_Splunk",

  server       => 'blank..com',

  splunkd_port => '8089',

}

class { '::splunk::forwarder':

splunk_user  => 'splunk',

}

file {'/opt/splunkforwarder/etc/splunk-launch.conf':

ensure  => present,

owner   => 'root',

group   => 'splunk',

recurse => false,

require => Package['splunkforwarder'],

}

file {'/opt/splunkforwarder/':

ensure  => present,

owner   => 'splunk',

group   => 'splunk',

recurse => true,

ignore   => '/opt/splunkforwarder/etc/splunk-launch.conf',

before  => File['/opt/splunkforwarder/etc/splunk-launch.conf'],

require => Package['splunkforwarder'],

}

file {'/var/log':

ensure  => present,

owner   => 'root',

group   => 'root',

mode    => "2755",

}

acl { '/var/log':

  action     => set,

  permission => [

  'group:splunk:r-x',

  'default:group:splunk:r-x',

  ],

  provider   => posixacl,

  recursive  => true,

  require    => Package['splunkforwarder'],

}

include ::splunk::forwarder

}

Erro deploying Splunk both Server and Forwarder

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 3.8.6
  • Ruby: ruby 1.8.7 (2013-06-27 patchlevel 374) [x86_64-linux]
  • Distribution: CentOS release 6.7 (Final)
  • Module version: 5.0.2

How to reproduce (e.g Puppet code you use)

What are you seeing

Error: Could not autoload puppet/type/splunk_config: /var/lib/puppet/lib/puppet/type/splunk_config.rb:7: syntax error, unexpected ':', expecting ')'
newparam(:name, namevar: true) do
^
/var/lib/puppet/lib/puppet/type/splunk_config.rb:128: syntax error, unexpected ':', expecting ')'
name: instance.name,
^
/var/lib/puppet/lib/puppet/type/splunk_config.rb:129: syntax error, unexpected ':', expecting '='
section: instance[:section],
^
/var/lib/puppet/lib/puppet/type/splunk_config.rb:130: syntax error, unexpected ':', expecting '='
setting: instance[:setting],
^
/var/lib/puppet/lib/puppet/type/splunk_config.rb:131: syntax error, unexpected kENSURE, expecting '='
ensure: :absent
^
/var/lib/puppet/lib/puppet/type/splunk_config.rb:132: syntax error, unexpected ')', expecting kEND
Error: Could not retrieve catalog from remote server: Could not intern from text/pson: Could not autoload puppet/type/splunk_config: /var/lib/puppet/lib/puppet/type/splunk_config.rb:7: syntax error, unexpected ':', expecting ')'
newparam(:name, namevar: true) do
^
/var/lib/puppet/lib/puppet/type/splunk_config.rb:128: syntax error, unexpected ':', expecting ')'
name: instance.name,
^
/var/lib/puppet/lib/puppet/type/splunk_config.rb:129: syntax error, unexpected ':', expecting '='
section: instance[:section],
^
/var/lib/puppet/lib/puppet/type/splunk_config.rb:130: syntax error, unexpected ':', expecting '='
setting: instance[:setting],
^
/var/lib/puppet/lib/puppet/type/splunk_config.rb:131: syntax error, unexpected kENSURE, expecting '='
ensure: :absent
^
/var/lib/puppet/lib/puppet/type/splunk_config.rb:132: syntax error, unexpected ')', expecting kEND
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

What behaviour did you expect instead

To be installed smoothly

Output log

Any additional information you'd like to impart

I have installed the same module on both centos 6 and 7 but with different ruby version (2.1) and it worked fine.

Add ability to upgrade universal forwarders and servers

Hi SETeam,

Thanks for creating this very useful puppet module.

I'm trying to modify the module so that it will upgrade universal forwarders to the latest version and can't get it to work right, so I was hoping that I could get help with that or get that functionality added to the module.

Thanks,

Alex

Exported resources getting automatically collected

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 4.10.6
  • Ruby: 2.0.0p648
  • Distribution: RedHat
  • Module version: 7.2.0

How to reproduce (e.g Puppet code you use)

@@splunk_server { 'clustering/master_uri':
  value => "https://${::fqdn}:8089",
}

What are you seeing

The resource is automatically realized on the node it is declared on.

What behaviour did you expect instead

According to the documentation on exported resources it should not be realized until it is collected, however even with no collectors declared in my code it is automatically being collected.

Declaring an exported resource causes that resource to be added to the catalog and marked with an “exported” flag, which prevents Puppet agent from managing the resource (unless it was collected).

Output log

Notice: /Stage[main]/Profile::Splunk::Cluster_master/Splunk_server[clustering/master_uri]/ensure: created

The splunk::forwarder::purge_inputs generates error

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 3.8.7
  • Ruby: ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
  • Distribution: Linux siem 4.4.0-31-generic #50~14.04.1-Ubuntu SMP Wed Jul 13 01:07:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
  • Module version: 5.0.1

How to reproduce (e.g Puppet code you use)

Try to enable splunk::forwarder::purge_inputs

splunk::forwarder::purge_inputs     : true

What are you seeing

An error message is generated during the provision:

Notice: Compiled catalog for puppet in environment production in 4.92 seconds
Error: /Stage[main]/Splunk::Params/Splunk_config[splunk]: Failed to generate additional resources using 'generate': can't convert nil into String
Notice: Finished catalog run in 15.65 seconds

What behaviour did you expect instead

No error message

Output log

Any additional information you'd like to impart

no parameter named 'purge_forwarder_deploymentclient'

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: Enterprise v2016.4.2
  • Ruby: 2.1.9p490 (bundled)
  • Distribution: RHEL6
  • Module version: Tried 6.3.1 and 7.0.0

How to reproduce (e.g Puppet code you use)

class profiles::splunkforwarder
{

$version = hiera('splunk::version', '6.5.0')
$build = hiera('splunk::build', '59c8927def0f')
$src_root = hiera('splunk::src_root')
$my_splunk_server = hiera('splunk::deploymentserver')

class { '::splunk::params':
server => $my_splunk_server,
version => $version,
build => $build,
src_root => $src_root,
}

include ::splunk::forwarder

What are you seeing

When trying to install just the forwarder on a host it errors out on the purge_* parameters saying they don't exist, even though they are defined in forwarder.pp

What behaviour did you expect instead

Successful puppet run with the splunk forwarder installed, configured and running.

Output log

Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: no parameter named 'purge_forwarder_deploymentclient' at /etc/puppetlabs/code/environments/karen/modules/splunk/manifests/params.pp:247 on Splunk_config[splunk] at /etc/puppetlabs/code/environments/karen/modules/splunk/manifests/params.pp:247 on node pup-karen.corp.cira.ca
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

Any additional information you'd like to impart

When I comment out the following section from the forwarder.pp file, it functions as expected:
Splunk_config['splunk'] {
purge_forwarder_deploymentclient => $purge_deploymentclient,
purge_forwarder_outputs => $purge_outputs,
purge_forwarder_inputs => $purge_inputs,
purge_forwarder_props => $purge_props,
purge_forwarder_transforms => $purge_transforms,
purge_forwarder_web => $purge_web,
}

Error on windows

Failed to install: the installation package could not be opened. Verify that the package exists and that you can access it, or contact the application vendor to verify that this is a valid windows installers package.

the installer wasn't able to be download from the puppet master.

module doesn't notify splunk restart when changing an input

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 3.8.5
  • Ruby: ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu]
  • Distribution: Ubuntu 16.04.1 LTS
  • Module version: 4.0.0

How to reproduce

class { '::splunk::forwarder': }

@splunkforwarder_input { 'source_type_example':
    section => "monitor://$logs_path/*.log",
    setting => 'sourcetype',
    value   => $name,
    tag     => "splunk_forwarder"
  }

What are you seeing

When applying a change to an instance that has already had puppet applied to it (eg. splunkforwarder already installed, then editing 'source_type_example' and running again), splunkforwarder is not restarted.

What behaviour did you expect instead

When making a chance to the resource type for splunk, splunk should be restarted

Output log

Notice: /Stage[main]/Splunk::Forwarder/File[/opt/splunkforwarder/etc/system/local/inputs.conf]/ensure: created
Notice: /Stage[main]/Splunk::Forwarder/Splunkforwarder_input[default_host]/ensure: created

No mention of a restart in the output, when running splunk list monitors the new input doesn't exist. Once I manually restart splunk splunk restart, the monitor then shows.

Strange issues during unit tests with Puppet 5.3.2

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 5.3.2
  • Ruby: 2.4.0
  • Distribution: OS X 10.11.6
  • Module version: 6.3.1

How to reproduce (e.g Puppet code you use)

Run unit tests on role class that pulls in custom types from the splunk module.

What are you seeing

This error:

Error while evaluating a Resource Statement, Could not autoload puppet/type/splunk_config: Attempt to redefine entity 'http://puppet.com/2016.1/runtime/type/splunk_authentication'.

Full output:

error during compilation: Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/splunk_config: Attempt to redefine entity 'http://puppet.com/2016.1/runtime/type/splunk_authentication'. Originally set at file:/Users/tdockendorf/puppet/osc-puppetmaster-conf/modules/splunk/lib/puppet/type/splunk_authentication.rb?line=3. at /Users/tdockendorf/puppet/osc-puppetmaster-conf/modules/splunk/manifests/params.pp:247:3 on node osc-nc167012.local

What behaviour did you expect instead

Unit tests to pass

Any additional information you'd like to impart

The unit tests work when I use ruby-2.0.0 and Puppet 3.8.6 with future parser. It's not until I switch to ruby-2.4.0 and Puppet 5.3.2 that things begin to fail.

I have two roles, one for search servers (web + search node) and one for indexers. The unit tests fail for both with same error. Unit tests for my roles that use the splunk forwarder types also fail in same way.

I have not tried the failing combination on real systems yet.

pass4SymmKey setting causes restarts of splunk

Affected Puppet, Ruby, OS and module versions/distributions

Puppet: 4.10.6
Ruby: 2.0.0p648
Distribution: RedHat
Module version: 7.2.0

How to reproduce (e.g Puppet code you use)

splunk_server { 'clustering/pass4SymmKey':
  value => "password",
}

What are you seeing

The password is set and splunk automatically encrypts it using its splunk.secret, however splunk and puppet then begin fighting over the password line, with puppet changing it back to the unencrypted password and restarting splunk (and then splunk changing it back to the encrypted password).

What behaviour did you expect instead

The password to be set and not keep changing back and forth, and splunk to not restart every run.

Any additional information you'd like to impart

Unfortunately, it seems that splunk attempts to hold their method of encrypting files very close, so implementing the encryption of the file inside of puppet may be difficult. I have had a few thoughts about generating a shasum of the password and using that to verify the password hasn't changed since last run, but even that is difficult since technically the server.secret can change and the password would then not be updated accordingly (unless puppet is also managing the server.secret).

Upgrades don't trigger license_splunkforwarder exec resource

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 4.10.6
  • Ruby:
  • Distribution: RHEL 7
  • Module version: 7.2.1

How to reproduce (e.g Puppet code you use)

class { '::splunk::params':
    server => $my_splunk_server,
}

include ::splunk::forwarder

What are you seeing

A normal upgrade of the forwarder will not trigger the license_splunkforwarder exec resource.

What behaviour did you expect instead

The license_splunkforwarder exec resource should be triggered any time the forwarder package is installed. That should include initial installs, as well as any upgrades.

Output log

Any additional information you'd like to impart

Every time the splunk forwarder package is installed, it causes the service to return to a "first time run" state. This means that the license agreement has to be agreed to before the service will start successfully.

Current

Triggering the license_splunkforwarder exec resource off the existence of the server.pem file exists prevents the exec resource from running again, unless someone decides they want to delete the server.pem file.

Suggested

Installing splunk or the forwarder always creates the ftr file. Once the license agreement has been accepted the ftr file is removed. This should make for a better trigger for the license_splunkforwarder exec resource. This way, the license gets accepted any time the package is installed.

Agent run fails: no parameter named 'purge_forwarder_deploymentclient'

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 5.1.3
  • Ruby: 2.4.1p111
  • Distribution: Debian 8
  • Module version: 7.0.0

How to reproduce (e.g Puppet code you use)

  class { '::splunk::params':
    version  => $version,
    build    => $build,
    src_root => $src_root,
  }
  include ::splunk::forwarder

puppet agent -t

What are you seeing

Error 500 on agent

What behaviour did you expect instead

Forwarder installed.

Output log

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: no parameter named 'purge_forwarder_deploymentclient' at /etc/puppetlabs/code/environments/master/modules/splunk/manifests/params.pp:258 on Splunk_config[splunk] at /etc/puppetlabs/code/environments/master/modules/splunk/manifests/params.pp:258 on node XXX

Any additional information you'd like to impart

I have been using puppet-splunk version 5.1.0 so far.
After upgrading to version 7.0.0 I get following error and can't install forwarder any more.

Splunk might be started before config is fully in place

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 5.3.3
  • Ruby: ruby 2.4.2p198 (2017-09-14 revision 59899) [x86_64-linux]
  • Distribution: CentOS Linux release 7.4.1708 (Core)
  • Module version: 7.0.0 and master

How to reproduce (e.g Puppet code you use)

class { '::splunk::params':
    server => 'localhost',
}

include ::splunk::forwarder

splunkforwarder_input { "test-index":
  section => "monitor://${title}",
  setting => 'index',
  value   => 'test',
}
splunkforwarder_input { "test-sourcetype":
  section => "monitor://${title}",
  setting => 'sourcetype',
  value   => 'test',
}

What are you seeing

Splunk gets started by puppet before the splunk config is finished. Exec['enable_splunkforwarder'] starts the splunk daemon, this might be before puppet is done writing all the config snippets. The service in splunk::virtual a bit later fails silently because splunkd is already running. It logs errors, but systemd/puppet (and in most cases, the end user 😀) are happy since the service is "running" already.

In the logs it can be seen that the test-index and test-sourcetype resources are applied after license_splunkforwarder.

What behaviour did you expect instead

Splunk running with the full config as specified through puppet.

Output log

Nov 24 08:53:33 localhost puppet-user[2616]: Compiled catalog for localhost.localdomain in environment production in 0.66 seconds
Nov 24 08:53:33 localhost puppet-user[2616]: (/Stage[main]/Archive::Staging/File[/opt/staging]/ensure) created
Nov 24 08:53:36 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/Archive[/opt/staging/splunk/splunkforwarder-7.0.0-c8a78efdd40f-linux-2.6-x86_64.rpm]/ensure) download archive from https://download.splunk.com/products/universalforwarder/releases/7.0.0/linux/splunkforwarder-7.0.0-c8a78efdd40f-linux-2.6-x86_64.rpm to /opt/staging/splunk/splunkforwarder-7.0.0-c8a78efdd40f-linux-2.6-x86_64.rpm  with cleanup
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/Package[splunkforwarder]/ensure) created
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/Splunkforwarder_input[default_host]/ensure) created
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/Splunkforwarder_output[tcpout_defaultgroup]/ensure) created
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/Splunkforwarder_output[defaultgroup_server]/ensure) created
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/Splunkforwarder_web[forwarder_splunkd_port]/ensure) created
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/File[/opt/splunkforwarder/etc/system/local/deploymentclient.conf]/ensure) created
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/File[/opt/splunkforwarder/etc/system/local/inputs.conf]/mode) mode changed '0644' to '0600'
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/File[/opt/splunkforwarder/etc/system/local/inputs.conf]/seluser) seluser changed 'unconfined_u' to 'system_u'
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/File[/opt/splunkforwarder/etc/system/local/outputs.conf]/mode) mode changed '0644' to '0600'
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/File[/opt/splunkforwarder/etc/system/local/outputs.conf]/seluser) seluser changed 'unconfined_u' to 'system_u'
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/File[/opt/splunkforwarder/etc/system/local/web.conf]/mode) mode changed '0644' to '0600'
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/File[/opt/splunkforwarder/etc/system/local/web.conf]/seluser) seluser changed 'unconfined_u' to 'system_u'
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/File[/opt/splunkforwarder/etc/system/local/limits.conf]/ensure) created
Nov 24 08:53:37 localhost puppet-user[2616]: (/Stage[main]/Splunk::Forwarder/File[/opt/splunkforwarder/etc/system/local/server.conf]/ensure) created
Nov 24 08:53:38 localhost puppet-user[2616]: (/Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns) executed successfully
Nov 24 08:53:38 localhost systemd: Reloading.
Nov 24 08:53:38 localhost systemd: Reloading.
Nov 24 08:53:38 localhost puppet-user[2616]: (/Stage[main]/Splunk::Platform::Posix/Exec[enable_splunkforwarder]/returns) executed successfully
Nov 24 08:53:38 localhost puppet-user[2616]: (/Stage[main]/Main/Splunkforwarder_input[test-index]/ensure) created
Nov 24 08:53:38 localhost puppet-user[2616]: (/Stage[main]/Main/Splunkforwarder_input[test-sourcetype]/ensure) created
Nov 24 08:53:38 localhost systemd: Reloading.
Nov 24 08:53:38 localhost systemd: Starting SYSV: Splunk indexer service...
Nov 24 08:53:38 localhost splunk: Starting Splunk...
Nov 24 08:53:39 localhost splunk: The splunk daemon (splunkd) is already running. [FAILED]
Nov 24 08:53:39 localhost systemd: Started SYSV: Splunk indexer service.
Nov 24 08:53:39 localhost puppet-user[2616]: (/Stage[main]/Splunk::Virtual/Service[splunk]/ensure) ensure changed 'stopped' to 'running'
Nov 24 08:53:39 localhost puppet-user[2616]: Applied catalog in 6.01 seconds

Any additional information you'd like to impart

There might be several ways to fix this:

  • Run a splunk stop after starting to accept the license, this can take a while (even on a fresh and mostly empty splunk install this can exceed the timeout for the puppet service provider)
  • Make sure the exec is applied after all the configs are in place. I only ran into this with splunkforwarder inputs and outputs, so arjenz@245e050 fixed it in my case, but this is probably not an exhaustive list.
Nov 24 09:14:29 localhost puppet-user[2705]: (/Stage[main]/Main/Splunkforwarder_input[test-index]/ensure) created
Nov 24 09:14:29 localhost puppet-user[2705]: (/Stage[main]/Main/Splunkforwarder_input[test-sourcetype]/ensure) created
Nov 24 09:14:32 localhost puppet-user[2705]: (/Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns) executed successfully
Nov 24 09:14:32 localhost systemd: Reloading.
Nov 24 09:14:32 localhost systemd: Reloading.
Nov 24 09:14:32 localhost puppet-user[2705]: (/Stage[main]/Splunk::Platform::Posix/Exec[enable_splunkforwarder]/returns) executed successfully
Nov 24 09:14:32 localhost systemd: Reloading.
Nov 24 09:14:32 localhost systemd: Starting SYSV: Splunk indexer service...
Nov 24 09:14:32 localhost splunk: Starting Splunk...
Nov 24 09:14:33 localhost splunk: The splunk daemon (splunkd) is already running. [FAILED]

Starting splunk for the second time still fails, but at least the config was completed before enable_splunkforwarder was run.

  • Proper systemd integration

Windows Installs Will Never Work with Default Options

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 4x
  • Ruby:
  • Distribution: PE
  • Module version: Latest

How to reproduce (e.g Puppet code you use)

Deploy splunk::forwarder to windows with default options

What are you seeing

on lines in params
These default paths will NEVER let the MSI actually install using the package manager.
as these are directly included into install arguments. please see this note on puppet docs on Package resource:

With Windows packages, note that file paths in an install option must use backslashes. (Since install options are passed directly to the installation command, forward slashes won’t be automatically converted like they are in file resources.) Note also that backslashes in double-quoted strings must be escaped and backslashes in single-quoted strings can be escaped.

if $::osfamily == 'Windows' {
$forwarder_dir = pick($forwarder_installdir, 'C:/Program Files/SplunkUniversalForwarder')
$server_dir = pick($server_installdir, 'C:/Program Files/Splunk')
} else {
$forwarder_dir = pick($forwarder_installdir, '/opt/splunkforwarder')
$server_dir = pick($server_installdir, '/opt/splunk')
}

splunkforwarder_server not working

When trying to use splunkforwarder_server type i get the following error on both ubuntu 16.04 and windows 2016

Error: /Stage[main]/Profile::Linux::Splunk::Universalforwarder/Splunkforwarder_server[sslConfig/sslPassword]: Could not evaluate: file_path must be set with splunk_config type before provider can be used
Notice: /Stage[main]/Splunk::Virtual/Service[splunk]: Dependency Splunkforwarder_server[sslConfig/sslPassword] has failures: true
Warning: /Stage[main]/Splunk::Virtual/Service[splunk]: Skipping because of failed dependencies

Here is the code i'm using

  splunkforwarder_server { 'sslConfig/sslPassword':
    value   => '$1$Bj3FSJRCLtSH',
    tag   => 'splunk_forwarder'
  }

@nicholaspearson

Cant figure out how to set the index that splunk-forwarder sends to

Hi,
Im unable to figure out how to set the indexer that this puppet-splunk module will send data to.... I've got a splunk server with a "LINUX" index - how can I configure the forwarder to send to that index rather than default? Im using the universal fwder 7.2.0

And while Im at it, thank you for a great module - this has saved me so much time!

-Matt

Warnings produced when splunk resources are in catalog

Affected Puppet, Ruby, OS and module versions/distributions

How to reproduce (e.g Puppet code you use)

Include splunk or splunk::forwarder class on a system

What are you seeing

[root@splunk ~]# puppet agent --test --noop
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Redefining rhsm_register in Puppet::Type
Info: Redefining augeasprovider in Puppet::Type
Info: Redefining shellvar in Puppet::Type
Info: Redefining datacat_fragment in Puppet::Type
Info: Redefining ini_setting in Puppet::Type
Info: Redefining firewall in Puppet::Type
Info: Redefining firewallchain in Puppet::Type
Info: Redefining foreman_resource in Puppet::Type
Info: Redefining idmapd_config in Puppet::Type
Info: Redefining gpg_key in Puppet::Type
Info: Redefining rhsm_repo in Puppet::Type
Warning: The dns_rr resource type is deprecated. Use resource_record instead
Info: Redefining file_line in Puppet::Type
Warning: The a2mod provider is deprecated, please use apache::mod instead
Info: Redefining anchor in Puppet::Type
Info: Redefining datacat_collector in Puppet::Type
Info: Caching catalog for splunk.infra.osc.edu
Info: Applying configuration version '1486415397'

What behaviour did you expect instead

Output produced by commenting out profile that includes splunk class and resources.

[root@splunk ~]# puppet agent --test --noop
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for splunk.infra.osc.edu
Info: Applying configuration version '1486415694'

Any additional information you'd like to impart

Duplicate declaration: Class[Splunk::Platform::Posix]

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 3.8.7
  • Ruby: ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
  • Distribution: Linux Ubuntu 14.04.5 LTS
  • Module version: 5.0.1

How to reproduce (e.g Puppet code you use)

Puppet code:

    include '::splunk'
    include '::splunk::forwarder'

Hiera value:

splunk::params::version: '6.5.0'
splunk::params::build: '59c8927def0f'
splunk::params::src_root: 's3://rg-infrastructure/splunk_install'
splunk::params::server: 'siem'

What are you seeing

Error message:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Class[Splunk::Platform::Posix] is already declared; cannot redeclare at /var/lib/rg_data/puppet/environments/production/modules/splunk/manifests/forwarder.pp:133 on node siem
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

What behaviour did you expect instead

No error message

Output log

Any additional information you'd like to impart

Allow Splunk/Splunkforwarder to be installed with "package" resource

This is an enhancement request.
For those using Satellite or a package manager with the splunkforwarder in their repos it would be great to be able to install the splunkforwarder with the package resource instead of bundling it with the module/in Puppet repo.
Any reason this is not supported?

Ruby load error when using splunk::forwarder on Windows

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 4.8.1 (PE 2016.5.1)
  • Ruby: 2.1.9
  • Distribution: Agent is Windows 2016, Master is RHEL 7
  • Module version: 5.1.0

How to reproduce (e.g Puppet code you use)

class { 'splunk::params':
  server => 'splunk.foo.net',
}

class { 'splunk::forwarder':
  package_source => '\\foo.net\dfs\splunk_forwarder\splunkforwarder-6.3.3-f44afce176d0-x64-release.msi'
}

What are you seeing

PS C:\Users\natemccurdy> puppet agent -t --environment splunk_agent_install --noop
Info: Using configured environment 'plunk_agent_install'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Redefining firewalld_direct_rule in Puppet::Type
Warning: C:/ProgramData/PuppetLabs/puppet/cache/lib/puppet/type/network_config.rb:6: ipaddress gem was not found
Error: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper
Error: Could not autoload puppet/type/splunk_config: Could not autoload puppet/provider/network_config/interfaces: canno
t load such file -- puppetx/filemapper
Error: Could not retrieve catalog from remote server: Could not intern from text/pson: Could not autoload puppet/type/sp
lunk_config: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

What behaviour did you expect instead

Splunk Forwarder MSI installed and configured with no error messages.

Any additional information you'd like to impart

This is a Ruby load issue that looks awfully familiar to #86

Actually, looking at it now, it seems that there just needs to be a new module version cut that includes #95 in it.

file_path not set for splunkforwarder_limits resource

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 5.5.3
  • Ruby: N/A
  • Distribution: N/A
  • Module version: v7.2.1

How to reproduce (e.g Puppet code you use)

Define a splunkforwarder_limits resource anywhere.

What are you seeing

Error: /Stage[main]/Profile_universalforwarder::Config/Splunkforwarder_limits[inputproc max_fd]: Could not evaluate: file_path must be set with splunk_config type before provider can be used

What behaviour did you expect instead

No error

Output log

Error: /Stage[main]/Profile_universalforwarder::Config/Splunkforwarder_limits[inputproc max_fd]: Could not evaluate: file_path must be set with splunk_config type before provider can be used

Any additional information you'd like to impart

Any other splunkforwarder_* resource is fine.
splunkforwarder_limits seems to be missing from the set_provider_paths function in type/splunk_config.rb around line 115.

Need a `splunkforwarder_input` defined resource type

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 2016.4.0
  • Ruby: ruby 2.1.9p490 (2016-03-30 revision 54437) [x86_64-linux]
  • Distribution: OEL 6/7
  • Module version: ce5b15ad5993cae6b72739d9f05f656ebfc5347c

What are you seeing

There are defined resource types for all of the other splunkforwarder_* configurations. However, we need one that handles limits and is placed in /opt/splunkforwarder/etc/system/local/limits.conf. It appears this one has just been missed.

What behaviour did you expect instead

It should behave exactly the same as all of the other splunkforwarder_* defined resource types.

Any additional information you'd like to impart

The other defined resource types work great.

Bump dependency constraint for the puppetlabs/inifile module

Affected Puppet, Ruby, OS and module versions/distributions

  • Module version: 7.1.0

How to reproduce (e.g Puppet code you use)

Install latest version of puppetlabs/inifile module

> puppet module install puppetlabs-inifile --version 2.2.0

Install latest version of puppet/splunk module

> puppet module install puppet-splunk --version 7.1.0

Install

What are you seeing

I see the following in my puppet server's logs

2018-03-08 05:19:15,307 WARN  [qtp1770542989-657360] [puppetserver] Puppet ModuleLoader: module 'splunk' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
   (file & line not available)
Error: Could not install module 'puppet-splunk' (???)
  No version of 'puppet-splunk' can satisfy all dependencies
    Use `puppet module install --ignore-dependencies` to install only this module

Evaluation Error: Unknown variable: 'splunk::params::forwarder_install_options'

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 4.9.2
  • Ruby: 2.3.1
  • OS: Ubuntu 16.04
  • Module version: 5.1.0

How to reproduce (e.g Puppet code you use)

  class { '::splunk::params':
    server   => hiera(my_splunk_server),
    version  => hiera(my_splunk_version),
    build    => hiera(my_splunk_build),
    src_root => hiera(my_splunk_src_root),
  }

  class { '::splunk::forwarder':
  }

What are you seeing

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Unknown variable: 'splunk::params::forwarder_install_options'.

What behaviour did you expect instead

Compile without error

Output log

Any additional information you'd like to impart

Need to put forwarder_install_options as class parameter for params and assign undef as default value or assign undef for different kernels other than Windows to avoid this compile error.

Windows File paths must be fully qualified

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet:
  • Ruby:
  • Distribution:
  • Module version:

How to reproduce (e.g Puppet code you use)

class { '::splunk::params':
        version           => '6.6.1',
        build             => 'aeae3fe0c5af',
        server            => 'splunk.acme.local',
      }
include ::splunk

What are you seeing

Puppet run throws the following error:
Failed to apply catalog: Parameter path failed on File[/opt/splunk/etc/system/local/authentication.conf]: File paths must be fully qualified, not '/opt/splunk/etc/system/local/authentication.conf' at /etc/puppetlabs/code/environments/production/modules/splunk/manifests/init.pp:221

Looks like the module maybe isn't intended to install Splunk (versus Forwarder) in Windows?

What behaviour did you expect instead

Output log

Any additional information you'd like to impart

splunk::password needs to distribute sslConfig

Currently, the splunk::password needs to also accept an $sslconfig parameter which needs to also be distributed to the forwarders /opt/splunkforwarder/etc/system/local/server.conf. Currently this is not being done and essentially when you use this class to distribute your splunk admin credentials the splunk forwarder fails to run afterwards because of ssl failing in regards to decryption/encryption

Invalid parameter key_val_separator

Hello SE Team,

I'm using this new version of your module and getting the following very unhelpful, unspecific errors.

 Error: /Stage[main]/Splunk::Forwarder/Splunkforwarder_output[tcpout_defaultgroup]:   Could not evaluate: Invalid parameter key_val_separator(:key_val_separator)
 Error: /Stage[main]/Splunk::Forwarder/Splunkforwarder_input[default_host]: Could not evaluate: Invalid parameter key_val_separator(:key_val_separator)
 Error: /Stage[main]/Splunk::Forwarder/Splunkforwarder_output[defaultgroup_server]: Could not evaluate: Invalid parameter key_val_separator(:key_val_separator)
 Error: /Stage[main]/Splunk::Platform::Posix/Splunkforwarder_input[monitor_varlog]: Could not evaluate: Invalid parameter key_val_separator(:key_val_separator)

I've checked my parameters and looked over your module and can't find anything glaringly obvious that would be causing this.

A Google search doesn't really show much. Hope you can help.

Thanks,

Alex

inheritence and version overrides

Heads up... I've not yet tracked down all of these issues here but if you classify as such:

class { 'splunk::linux_forwarder':
  splunk_ver = <some non-default value>,
}

you won't get the version specified due to the way the default version is specified in params and set in the implementation class via inheritance. I believe that the use of way inheritance is used in this module may mean there are other similar issues.

Support .conf files in non-standard locations

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: All versions
  • Distribution: All distributions
  • Module version: All versions

How to reproduce (e.g Puppet code you use)

splunk_props { 'useless title':
  path    => '/opt/splunk/etc/master-apps/_cluster/local',  # <-- would like to be able to do this
  section => 'default',
  setting => 'defaultGroup',
  value   => 'splunk_9777',
}

What are you seeing

The resource types have hardcoded paths to the .conf files and do not allow for specifying a path.

What behaviour did you expect instead

Allowing the path to be specified on a per-resource basis would allow for management of index and search head clusters.

Specifically, I would like to be able to manage the index cluster config files located at /opt/splunk/etc/master-apps/_cluster/local on the cluster master.

Any additional information you'd like to impart

Is this something that this module could support? If so, what would be the recommended way to implement this?

I've gotten this working by declaring additional resource types (splunkmaster_indexes and splunkmaster_inputs) but it feels forced. I have not been successful in implementing a path property on the base type since it seems to be a limitation of puppetlabs-inifile.

Error on using ini_setting

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 4.5.3
  • Distribution: RHEL7
  • Module version: 5.0.2

How to reproduce (e.g Puppet code you use)

  class { '::splunk::params':
    server => 'splunkmaster.domain.name'
  }

  class { '::splunk::forwarder':
    pkg_provider => 'yum',
    package_name => 'splunkforwarder',
    package_ensure => installed
  }

What are you seeing

Could not retrieve catalog from remote server: Error 400 on SERVER: Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/splunk_config: Could not autoload puppet/provider/splunk_distsearch/ini_setting: undefined method `provider' for nil:NilClass at /etc/puppetlabs/code/environments/splunk_client/modules/splunk/manifests/params.pp:244:3 on node i-26932f02

What behaviour did you expect instead

Installation of Splunk forwarder

Any additional information you'd like to impart

I'm having a hard time telling from the documentation exactly how to use this module in a scenario where the package is in a local yum repository rather than served by the puppetmaster, but I've tried to work it out from the source code. I'm not clear on why I'm getting this particular error though.

Module does not seem to work with splunkforwarder 7.X

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 4.10.6
  • Ruby: 2.0.0p648
  • Distribution: Centos
  • Module version: 7.2.0

How to reproduce (e.g Puppet code you use)

  class { '::splunk::params':
    version => '=7.1.1',
    build   => '8f0ead9ec3db',
  }

  class { '::splunk::forwarder':
    pkg_provider => 'yum',
    splunk_user  => 'splunk',
  }

What are you seeing

Notice: /Stage[main]/Splunk::Forwarder/Package[splunkforwarder]/ensure: created
Notice: /Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns: Password must contain at least:
Notice: /Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns:    * 8 total printable ASCII character(s).
Notice: /Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns: 
Notice: /Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns: This appears to be your first time running this version of Splunk.
Notice: /Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns: 
Notice: /Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns: An Admin password must be set before installation proceeds.
Notice: /Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns: tcgetattr: Inappropriate ioctl for device
Notice: /Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns: WARNING: error changing terminal modes - password will echo!
Notice: /Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns: Please enter a new password: 
Error: 'splunk start --accept-license --answer-yes' returned 1 instead of one of [0]
Error: /Stage[main]/Splunk::Platform::Posix/Exec[license_splunkforwarder]/returns: change from notrun to 0 failed: 'splunk start --accept-license --answer-yes' returned 1 instead of one of [0]

What behaviour did you expect instead

A successful install and configuration of the splunkforwarder.

Any additional information you'd like to impart

Splunk Enterprise 7.1 introduces a new password scheme for Splunk software users. This scheme includes additional settings and configuration options, which can affect how you upgrade if you use scripts to automate the upgrade process. You might need to change your upgrade scripts before performing scripted upgrades. Specifically, confirm that you do not pass any illegal arguments to the Splunk CLI for starting or restarting Splunk Enterprise during the upgrade, as this could result in a situation where Splunk Enterprise does not start after the upgrade has completed.

From http://docs.splunk.com/Documentation/Splunk/7.1.1/Installation/AboutupgradingREADTHISFIRST

Purging of inputs and outputs don't work on forwarder

Setting purge_inputs and purge_outputs to true is not actually purging unmanaged settings in those files. I have reproduced this on our forwarder on windows. We still need to investigate if this is O/S specific and also what all purging is actually broken and fix it.

Issue with services when using forwarder and indexer on the same server

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 3.8.7
  • Ruby: ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
  • Distribution: Linux siem 4.4.0-31-generic #50~14.04.1-Ubuntu SMP Wed Jul 13 01:07:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
  • Module version: 5.0.1

How to reproduce (e.g Puppet code you use)

Install both splunk forwarder and splunk server

What are you seeing

The install is ok but the corresponding services are not created properly
There is only one service in /etc/init.d with the name splunk and it corresponds to the first service that has been installed during the installation (so it might be splunk-forwarder or splunk-server depending on the order you call the 2 classes)

In my case every times I changed configuration files related to splunk server, this was the splunk forwarder which was restarted (takes me ages to understand why my new settings werent taken into account)

What behaviour did you expect instead

Puppet should manage the 2 services independently : one to restart splunk-forwarder and one to restart splunk-server

Output log

Any additional information you'd like to impart

The corresponding resources declare the same file so that couldn't work

  @exec { 'enable_splunkforwarder':
    # The path parameter can't be set because the boot-start silently fails on systemd service providers
    command => "${splunk::params::forwarder_dir}/bin/splunk enable boot-start -user ${splunk_user}",
    creates => '/etc/init.d/splunk',
    require => Exec['license_splunkforwarder'],
    tag     => 'splunk_forwarder',
    notify  => Service['splunk'],
  }

When the first exec has run, the second exec below is never executed as the '/etc/init.d/splunk' file already exists.

  @exec { 'enable_splunk':
    # The path parameter can't be set because the boot-start silently fails on systemd service providers
    command => "${splunk::params::server_dir}/bin/splunk enable boot-start -user ${splunk_user}",
    creates => '/etc/init.d/splunk',
    require => Exec['license_splunk'],
    tag     => 'splunk_server',
  }

dependency cycle error while deploying forwarder only

Hello,

I'm attempting to use your module to deploy the Splunk Forwarder only and Puppet throws a dependancy cycle error. I'm testing using Pupply apply if that makes a differenance.

Notice: Compiled catalog for <hostname> in environment production in 0.65 seconds
Error: Failed to apply catalog: Found 1 dependency cycle:
(File[/opt/staging/splunk/splunkforwarder-6.5.0-59c8927def0f-linux-2.6-x86_64.rpm] => Staging::File[splunkforwarder-6.5.0-59c8927def0f-linux-2.6-x86_64.rpm] => Package[splunkforwarder] => File[/opt/staging/splunk/splunkforwarder-6.5.0-59c8927def0f-linux-2.6-x86_64.rpm])
Try the '--graph' option and opening the resulting '.dot' file in OmniGraffle or GraphViz

Here is how I call the module:

class profiles::splunk_forwarder (
  $server  = 'splunk.domain',  
  $version = '6.5.0',  
  $build   = '59c8927def0f',
) {
  class { ::splunk::params:
    server  => "$server",
    version => "$version",
    build   => "$build"
  }
  include ::splunk::forwarder
 }

Splunk as non-root user by default for UNIX systems.

This is an enhancement request.
As part of Splunk Enterprise best practices Splunk should run as 'non-root user'. See official splunk documentation http://docs.splunk.com/Documentation/Splunk/7.0.0/Installation/RunSplunkasadifferentornon-rootuser

Code involved: ~/puppet-splunk/manifests/params.pp

 } else {
    $forwarder_dir = pick($forwarder_installdir, '/opt/splunkforwarder')
    $server_dir    = pick($server_installdir, '/opt/splunk')
    $splunk_user   = 'root'
  }

What are you seeing

splunk runs as "root" user by default. Which is against splunk best practices.

What behaviour did you expect instead

The module should create a non-root user (i.e "splunk" user) and run splunk as it.

Unable to get basic Splunk server running

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: 4.5.1 (vagrant) 4.10.8 (vmware)
  • Ruby: 2.1.9p490
  • Distribution: Ubuntu 16.04 LTS
  • Module version: v7.0.0

How to reproduce (e.g Puppet code you use)

class mailonline::splunk2 {

class { '::splunk::params':
version => '6.6.3',
src_root => 'http://somebucket.s3-website-eu-west-1.amazonaws.com',
splunkd_port => '8089',
}

include ::splunk

}

What are you seeing

Error: Failed to apply catalog: no parameter named 'section'

What behaviour did you expect instead

Notice: Applied catalog in n.nn seconds

Output log

Info: Using configured environment 'env'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Warning: Creating splunk_index via Puppet.newtype is deprecated and will be removed in a future release. Use Puppet::Type.newtype instead.
(at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:162:in `newtype')
Info: Caching catalog for
Error: Failed to apply catalog: no parameter named 'section'

Any additional information you'd like to impart

Tried it on v6.3.1 as well same issue

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.