Git Product home page Git Product logo

docker-splunk's Introduction

Docker-Splunk: Containerizing Splunk Enterprise

License  GitHub release

Welcome to the official Splunk repository of Dockerfiles for building Splunk Enterprise and Splunk Universal Forwarder images for containerized deployments.


⚠️DEPRECATION NOTICE
We are no longer releasing Debian images on Docker Hub as of May 2021 (Splunk Enterprise v8.2.0+). Red Hat images will continue to be published.


Table of Contents

  1. Purpose
  2. Quickstart
  3. Documentation
  4. Support
  5. Contributing
  6. License

Purpose

What is Splunk Enterprise?

Splunk Enterprise is a platform for operational intelligence. Our software lets you collect, analyze, and act upon the untapped value of big data that your technology infrastructure, security systems, and business applications generate. It gives you insights to drive operational performance and business results.

See Splunk Products for more information about the features and capabilities of Splunk products and how you can bring them into your organization.

What is Docker-Splunk?

This is the official source code repository for building Docker images of Splunk Enterprise and Splunk Universal Forwarder. By introducing containerization, we can marry the ideals of infrastructure-as-code and declarative directives to manage and run Splunk Enterprise.

The provisioning of these containers is handled by the Splunk-Ansible project. Refer to the Splunk-Ansible documentation and the Ansible User Guide for more details.


Quickstart

Start a single containerized instance of Splunk Enterprise with the command below, replacing <password> with a password string that conforms to the Splunk Enterprise password requirements.

$ docker run -p 8000:8000 -e "SPLUNK_PASSWORD=<password>" \
             -e "SPLUNK_START_ARGS=--accept-license" \
             -it --name so1 splunk/splunk:latest

This command does the following:

  1. Starts a Docker container using the splunk/splunk:latest image.
  2. Names the container as so1.
  3. Exposes a port mapping from the host's 8000 port to the container's 8000 port
  4. Specifies a custom SPLUNK_PASSWORD.
  5. Accepts the license agreement with SPLUNK_START_ARGS=--accept-license. This agreement must be explicitly accepted on every container or Splunk Enterprise doesn't start.

After the container starts up, you can access Splunk Web at http://localhost:8000 with admin:<password>.

To view the logs from the container created above, run:

$ docker logs -f so1

To enter the container and run Splunk CLI commands, run:

# Defaults to the user "ansible"
docker exec -it so1 /bin/bash

# Run shell as the user "splunk"
docker exec -u splunk -it so1 bash

To enable TCP 10514 for listening, run:

docker exec -u splunk so1 /opt/splunk/bin/splunk add tcp 10514 \
    -sourcetype syslog -resolvehost true \
    -auth "admin:${SPLUNK_PASSWORD}"

To install an app, run:

docker exec -u splunk so1 /opt/splunk/bin/splunk install \
	/path/to/app.tar -auth "admin:${SPLUNK_PASSWORD}"

# Alternatively, apps can be installed at Docker run-time
docker run -e SPLUNK_APPS_URL=http://web/app.tgz ...

See Deploy and run Splunk Enterprise inside a Docker container for more information.


Documentation

Visit the Docker-Splunk documentation page for full usage instructions, including installation, examples, and advanced deployment scenarios.


Support

Use the GitHub issue tracker to submit bugs or request features.

If you have additional questions or need more support, you can:

  • Post a question to Splunk Answers
  • Join the #docker room in the Splunk Slack channel. If you're a new Splunk customer you can register for Slack here
  • If you are a Splunk Enterprise customer with a valid support entitlement contract and have a Splunk-related question, you can also open a support case on the https://www.splunk.com/ support portal

See the official support guidelines for more detailed information.


Contributing

We welcome feedback and contributions from the community! See our contribution guidelines for more information on how to get involved.


License

Copyright 2018-2020 Splunk.

Distributed under the terms of our license, splunk-ansible is free and open source software.

Authors

Splunk Inc. and the Splunk Community

docker-splunk's People

Contributors

abdulk-splunk avatar adityapinglesf avatar alishamayor avatar arctan5x avatar clarkkarenl avatar dan1 avatar emxqm-11 avatar gracening avatar halr9000 avatar hendolim avatar jeff-an avatar jkoby avatar jmeixensperger avatar jmervine avatar jonathan-vega-splunk avatar jrigassio-splunk avatar lephino avatar matthewmodestino avatar melugoyal avatar michaellee-splunk avatar nwang92 avatar outcoldman avatar ryanfaircloth avatar sscholl avatar sujaykumar-splunk avatar sushantkadadi avatar thecamnelson avatar tonytamsf avatar vladdoster avatar zolvaring avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

docker-splunk's Issues

Improve security, make SPLUNK_PASSWORD not required

Exposing password using environment variable not a great idea, especially in case of Splunk for two reasons:

  1. docker inspect or if you have access to docker API, you can get an administrator password
  2. Every process inside the Splunk container will have access to the admin password. If you install the application from splunkbase with custom code, that custom code can read the admin password and do anything it wants.
  3. I am not sure if that is possible right now out of the box, but I am sure that there is some debug options, that allows you to read environment variables of Splunkd, or maybe SPL command. That means, that you can get access to the admin password.

Allow to install application directly from SplunkBase on start

Will be very useful to be able to install SplunkBase application by specifying the ID of the app, and some SplunkBase token to be able to download the app on start.

I have received this question a LOT of times for the legacy image. And hoped that new image will be able to support that.

make ansible should always pull in the latest from splunk-ansible repo

There are 2 issues from the splunk-ansible repo that fixed 2 issues for me, but then splunk-ansible was updated, the make ansible script didn't pull in the changes.

This patch should fix it

diff --git a/Makefile b/Makefile
index 34761f3..0a9bc65 100644
--- a/Makefile
+++ b/Makefile
@@ -31,6 +31,8 @@ all: splunk uf
 ansible:
        if [ -d "splunk-ansible" ]; then \
                echo "Ansible directory exists - skipping clone"; \
+               (cd splunk-ansible; git pull ;\
+               git checkout ${SPLUNK_ANSIBLE_BRANCH} ) \
        else \
                git clone https://github.com/splunk/splunk-ansible.git --branch ${SPLUNK_ANSIBLE_BRANCH}; \
        fi

docker upgrade failure, help please.

I was using a splunk 7.1 with a volume container.

I deleted the old splunk container and try to do an upgrade like this:

docker run --hostname splunk --name splunk --volumes-from=vsplunk -p 8000:8000 -d -e "SPLUNK_START_ARGS=--accept-license --seed-passwd changeme --answer-yes" -e "SPLUNK_UPGRADE" -e "DEBUG" splunk/splunk:latest

It failed. Do I need to define any other environment variables?

The error is:

WARNING: No password ENV var. Stack may fail to provision if splunk.password is not set in ENV or a default.yml

[WARNING]: * Failed to parse /opt/ansible/inventory/environ.py with script

plugin: Inventory script (/opt/ansible/inventory/environ.py) had an execution

error:

[WARNING]: * Failed to parse /opt/ansible/inventory/environ.py with ini

plugin: /opt/ansible/inventory/environ.py:16: Expected key=value host variable

assignment, got: os

[WARNING]: Unable to parse /opt/ansible/inventory/environ.py as an inventory

source

[WARNING]: No inventory was parsed, only implicit localhost is available

[WARNING]: provided hosts list is empty, only localhost is available. Note

that the implicit localhost does not match 'all'

PLAY [localhost] ***************************************************************

TASK [Gathering Facts] *********************************************************

Wednesday 10 October 2018 15:10:32 +0000 (0:00:00.051) 0:00:00.051 *****

ok: [localhost]

TASK [include_role : splunk_upgrade] *******************************************

Wednesday 10 October 2018 15:10:35 +0000 (0:00:02.829) 0:00:02.881 *****

fatal: [localhost]: FAILED! => {"msg": "The conditional check 'splunk.upgrade is defined' failed. The error was: error while evaluating conditional (splunk.upgrade is defined): 'splunk' is undefined\n\nThe error appears to have been in '/opt/ansible/site.yml': line 5, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n tasks:\n - name: Upgrade role\n ^ here\n"}

to retry, use: --limit @/opt/ansible/ansible-retry/site.retry

PLAY RECAP *********************************************************************

localhost : ok=1 changed=0 unreachable=0 failed=1

Wednesday 10 October 2018 15:10:35 +0000 (0:00:00.022) 0:00:02.904 *****

===============================================================================

Gathering Facts --------------------------------------------------------- 2.83s

include_role : splunk_upgrade ------------------------------------------- 0.02s

Allow to use wildcards for SPLUNK_LICENSE_URI

With the old version of Splunk container, I could install multiple Splunk licenses from the same folder by using a wildcard, see https://github.com/outcoldsolutions/splunk-app-boilerplate/blob/a73e6238f299c0c4c7e18177df64d30904529b5e/Makefile#L27

--env "SPLUNK_CMD=add licenses -auth admin:${SPLUNK_PASSWORD} /hack/splunk/licenses/*.lic || true" \

It will be good to have the same ability for when you target local folder with SPLUNK_LICENSE_URI, like

"SPLUNK_LICENSE_URI=/hack/splunk/licenses/*.lic"

How to maintain configuration for the Splunk instance?

I am considering running Splunk in Docker to replace my current .deb based install. My expectation is that a Docker-based setup would make it much easier to upgrade to new Splunk versions in the future. Also, I don't need a particularly fancy setup for just 1-2 GB of log volume per day.

One thing, however, I don't understand yet is what the recommended approach is to keep index data and Splunk configuration out of the running image, and at best under version control (?).

My users need to be able to install additional apps, they create dashboards and alerts and have their reports configured. Additionally, we've got custom sourcetypes and made quite some modifications to the inputs.conf, props.conf and transform.conf files. Last but not least, there's SSL certificates for the web interface, Splunk-2-Splunk and HTTP Event Collector.

Could someone point me in the right direction here? Should I just mount /opt/splunk/var and /opt/splunk/etc as volumes, and can I carry those over to new versions of the Splunk image? Is there a better way or even a recommended approach documented somewhere?

Thanks for your help!

Support installation of apps from "repository"

It would be very helpful to support installation of apps using the slim command by role so the proper and only proper components are installed. The source should be a host shared folder where the tgz files exist.

Error add new app JMS Messaging Modular Input

Hi,

i having error below after installed app JMS Messaging Modular Input from file :

Unable to initialize modular input "jms" defined inside the app "jms_ta": Introspecting scheme=jms: script running failed (exited with code 1).

Appreciate for any advice.

Thanks
Jason

Basic non-ansible container preferred

You used to have a container that was basic and simple, without using Ansible. Can you also make a version like this again? Bringing Ansible into the container kind of breaks some of the idiomatic patterns of containers, it's just not good -- and it makes it harder to bring into our own systems.

Also, will you be supporting an Alpine build soon? CentOS?

Expand supported runtimes

In the docs you say that only Docker starting from 17.x is supported.

That leaves a question for support of Kubernetes and OpenShift, where their recommended docker runtime is 1.13.1, see https://github.com/splunk/docker-splunk/blob/master/documentation/SETUP.md#supported-docker-engine-versions

And for Kubernetes, they can use different runtimes, not only Docker https://kubernetes.io/docs/setup/independent/install-kubeadm/#installing-runtime, line cri-o, containerd, etc. They all compatible with Docker image.

splunk user posses sudo rights

Connect to the splunk container
docker exec -it /bin/bash

from shell "sudo apt-get update"

while having Splunk run as non root to protect from web exploitation is important allowing sudo is a shallow control.

unable to map inputs.conf to latest forwarder container

I used to be able to do this with splunkforwarder-7.2.1-be11b2c46e23-Linux (splunk/universalforwarder@sha256:cf9b53ee182de36671de774ffecd63f3cbf3ed7cb353a50804f14151e25d12be)

docker run -it -p 9777:9777 -v /etc/admin/inputs.conf:/opt/splunkforwarder/etc/system/local/inputs.conf -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_INDEXER_URL=indexers.xxxx.com' -e 'SPLUNK_PASSWORD=xyz' splunk/universalforwarder:latest

Now with the latest, splunkforwarder-7.2.3-06d57c595b80-Linux (splunk/universalforwarder@sha256:9c99896c7fbfbb6e94689e33ca2cb621e3a49512b2322d7273be7c90774f8a42) it's throwing a permission error ansible is not able to unpack splunk files under /opt/splunkforwarder/etc. I noticed that the default user has changed from "splunk" to "ansible". What is the right way to map overrides in the container now ?

fatal: [localhost]: FAILED! => {"changed": false, "dest": "/opt", "extract_results": {"cmd": ["/bin/tar", "--extract", "-C", "/opt", "-z", "--owner=splunk", "--group=splunk", "-f", "/var/tmp/ansible-tmp-1548084479.3-222115422914359/source
"], "err": "/bin/tar: splunkforwarder/etc/log-cmdline.cfg: Cannot open: Permission denied\n/bin/tar: splunkforwarder/etc/log.cfg: Cannot open: Permission denied\n/bin/tar: splunkforwarder/etc/copyright.txt: Cannot open: Permission denied\
n/bin/tar: splunkforwarder/etc/init.d: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/init.d: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/init.d/README: Cannot open: No such file or directory\n/bin/tar:
splunkforwarder/etc/licenses: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/licenses: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/licenses/forwarder: Cannot mkdir: No such file or directory\n/bin/tar: s
plunkforwarder/etc/licenses: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/licenses/forwarder/splunkforwarder.lic: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir: Permission de
nied\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/modules/input: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir: Permission denied\n/b
in/tar: splunkforwarder/etc/modules/input/UDP: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/modules/input/UDP/config.xml: Cannot open: No su
ch file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/modules/input/structuredparsing: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/modules: C
annot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/modules/input/structuredparsing/config.xml: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir: Permission denied\n/bin/tar: splunkforw
arder/etc/modules/input/fschangemanager: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/modules/input/fschangemanager/config.xml: Cannot open:
 No such file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/modules/input/TCP: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mk
dir: Permission denied\n/bin/tar: splunkforwarder/etc/modules/input/TCP/config.xml: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/modules/inpu
t/exec: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/modules/input/exec/config.xml: Cannot open: No such file or directory\n/bin/tar: splunk
forwarder/etc/modules: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/modules/input/FIFO: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir: Permission denied\n/bin/tar: splunkfor
warder/etc/modules/input/FIFO/config.xml: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/modules: Cannot mkdir:  [snip]```

Failure setting indexers as search peers

Hi, I'm getting the following error which causes the search head container to stop.

TASK [splunk_search_head : Set all indexers as search peers] *******************
Saturday 27 October 2018 22:04:52 +0000 (0:00:03.579) 0:02:47.351 ******
fatal: [localhost]: FAILED! => {"msg": "The conditional check 'task_result.rc == 0' failed. The error was: error while evaluating conditional (task_result.rc == 0): 'task_result' is undefined"}

In my scenario I have 2 indexers and 1 search head. I've used 2idx2sh.yaml as a starting point for my file. I've attached my yaml file.

docker-compose.yml.txt

Thanks

TASK [splunk_search_head : Check Splunk instance is running] is failing

When trying the advanced example:

SPLUNK_COMPOSE=cluster_absolute_unit.yaml make sample-compose-up

The task TASK [splunk_search_head : Check Splunk instance is running] is failing after 50 retries due to admin password failure.

I've set the password in the defaults.yml according to the doc but the deployment check is failing.
Fortunately the cluster is deployed and I'm able to login using the admin credentials supplied in the defaults.yml.

When I used bash to check for Splunk container availability I did a more basic check:

wait_for_splunk_container ()
{
  port=$(docker port $1 8000)
  until $(curl --output /dev/null --silent --head --fail http://localhost:${port##*:});
  do
    printf '.'
    sleep 5
  done
  printf "\n"
}

that only looked for the Web service being available and not the ability to login which is potentially a different issue.

Cannot install `splunk/splunk:latest` on Ubuntu Server 18.04.1 and Docker available via `snap install`

This is the same underlying bug as #96 but with 100% reproducibility as of this writing.

Canonical is still shipping Docker 18.06.1-ce in the default snap repository. I intend to notify them that their Docker is out of date, but they're now five months behind so it's safe to assume that their upgrade process will take a while.

From a brand new Ubuntu Server 18.04.1 installation, after the initial round of apt update and apt upgrade, and following these steps:

$ sudo snap install docker
$ sudo docker pull splunk/splunk:latest
$ sudo docker run -d -p 8000:8000 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=<Any Password>' --name splunk-server splunk/splunk:latest

Then the following output results:

$ sudo docker ps -a
CONTAINER ID        IMAGE                  COMMAND                  CREATED             STATUS                     PORTS               NAMES
86e7ebef9874        splunk/splunk:latest   "/sbin/entrypoint.sh…"   6 seconds ago       Exited (2) 3 seconds ago                       splunk-server
$ sudo docker logs splunk-server

PLAY [Run default Splunk provisioning] *****************************************
Friday 18 January 2019  01:19:25 +0000 (0:00:00.053)       0:00:00.053 ********

TASK [Gathering Facts] *********************************************************
fatal: [localhost]: FAILED! => {"msg": "Failed to set permissions on the temporary files Ansible needs to create when becoming an unprivileged user (rc: 1, err: chown: changing ownership of '/var/tmp/ansible-tmp-1547774365.39-234000989980791/': Operation not permitted\nchown: changing ownership of '/var/tmp/ansible-tmp-1547774365.39-234000989980791/AnsiballZ_setup.py': Operation not permitted\n}). For information on working around this, see https://docs.ansible.com/ansible/become.html#becoming-an-unprivileged-user"}
        to retry, use: --limit @/opt/container_artifact/ansible-retry/site.retry

PLAY RECAP *********************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1

Friday 18 January 2019  01:19:25 +0000 (0:00:00.323)       0:00:00.376 ********
===============================================================================
Gathering Facts --------------------------------------------------------- 0.32s

Further pertinent information:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.1 LTS
Release:        18.04
Codename:       bionic
$ uname -a
Linux ubuntu-1804-docker-validation 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ docker version
Client:
 Version:           18.06.1-ce
 API version:       1.38
 Go version:        go1.10.4
 Git commit:        e68fc7a
 Built:             Mon Oct  1 14:25:31 2018
 OS/Arch:           linux/amd64
 Experimental:      false
$ snap info docker
name:      docker
summary:   Docker container runtime
publisher: canonical
contact:   [email protected]
license:   unknown
description: |
  Build and run container images with Docker.

  This build requires all files that Docker uses, such as dockerfiles, to be in $HOME. Keep files
  there for 'docker build', 'docker save' and 'docker load'. Use the 'docker-privilege' command for
  'docker run --privileged'.

  This snap is built by Canonical based on source code published by Docker, Inc. It is not endorsed
  or published by Docker, Inc.

  Docker and the Docker logo are trademarks or registered trademarks of Docker, Inc. in the United
  States and/or other countries. Docker, Inc. and other parties may also have trademark rights in
  other terms used herein.
commands:
  - docker.compose
  - docker
  - docker.help
  - docker.machine
services:
  docker.dockerd: simple, enabled, active
snap-id:      sLCsFAO8PKM5Z0fAKNszUOX0YASjQfeZ
tracking:     stable
refresh-date: today at 01:03 UTC
channels:
  stable:          18.06.1-ce   (321) 102MB -
  candidate:       18.06.1-ce   (321) 102MB -
  beta:            18.06.1-ce   (321) 102MB -
  edge:            17.03.1-ce-1 (333) 38MB  -
  17.09/stable:    –
  17.09/candidate: 17.09.1-ce   (232) 44MB  -
  17.09/beta:      17.09.1-ce   (232) 44MB  -
  17.09/edge:      17.09.1-ce   (232) 44MB  -
  17.06/stable:    17.06.2-ce   (179) 43MB  -
  17.06/candidate: 17.06.2-ce   (179) 43MB  -
  17.06/beta:      ↑
  17.06/edge:      17.06.2-ce-1 (237) 44MB  -
  17.03/stable:    17.03.2-ce-1 (159) 42MB  -
  17.03/candidate: 17.03.2-ce-1 (159) 42MB  -
  17.03/beta:      ↑
  17.03/edge:      17.03.2-ce-1 (159) 42MB  -
installed:         18.06.1-ce   (321) 102MB -

I can confirm that using the official Docker CE Ubuntu repository with Docker CE 18.09.1 results in a successful installation.

Repeatedly returning to the login page: "Your session has expired. Log in..."

With apologies if this is a Splunk (Web) issue, rather than an issue with this Splunk Docker image...

Embarrassing disclosure: Some careless fool overfed my local Windows install of Splunk too many times, so I'm now testing my Splunk dashboards entirely in Docker containers. I might be that fool; several users forward data from the same source, and I haven't bothered trying to identify which people were responsible for those particular "feeding times".

I'm starting Splunk Enterprise 7.2.0 with the Trial license in Docker containers. Sometimes, Splunk Web repeatedly returns to the login page. I've previously solved that, or thought I had, by deleting the cookies for that Splunk Web instance (using Chrome developer tools) and logging in again. However, last time this happened (a couple of days ago), that didn't work. So I restarted Chrome. That worked. Phew. It's unsettling, though. I wish I better understood why it was happening. And why deleting those cookies seemed to work before, without restarting my browser.

I should confess, in case it's significant, that I sometimes have multiple tabs open to the same and different instances of Splunk Web (different port numbers—different Docker containers—running on the same Docker host; sometimes, different hosts, too).

Any ideas?

I understand that switching Splunk inside the containers to a Splunk Free license would make this issue go away. If it's possible to use this Splunk Docker image to start a container as Splunk Free—rather than starting as Enterprise Trial, and then having to switch to a Free license—that would be useful.

Startup/restart performance degradation (10s legacy image vs 30-50s new image)

The old image takes ~10 seconds to start Splunk. The new image takes 30 seconds. Unpacking splunk.tar.gz on every start takes about 20 seconds additionally.

Because containers are running on a writable container layer without persistence, you will find these additional ~20 seconds not only on docker run, but also docker restart.

I tested on MacBook Pro with NVMe SSD, I assume that on HDD/SSD that will even more visible.

You mentioned that you reduced 30% of the image by keeping Splunk gzipped. But that does not provide any benefits because:

  • docker takes more time to start the new image,
  • only in rest you can see that image takes less space than the old image, which does not give you any benefits, considering that if you downloaded the image to the host, you probably want to start it
  • in transition docker always sends gzipped layers, so size/time of pull/push will be approximately the same

Why can't I log in to the SPLUNK platform?

I used the tag is latest .
and use this command "docker run -d -p 8000:8000 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=' splunk/splunk:latest".
The password has changed to what I know,but I use my password can't log in? what's the problem?
image

Splunk Universal Forwarder issue with 7.2.3 on Docker for Mac

As of 7.2.3 image from Dockerhub I am unable to map a forwarder app into the Splunk etc folder due to a permission problem:

splunkforwarder_1  |
splunkforwarder_1  | TASK [splunk_universal_forwarder : Install Splunk universal forwarder] *********
splunkforwarder_1  | fatal: [localhost]: FAILED! => {"changed": false, "dest": "/opt", "extract_results": {"cmd": ["/bin/tar", "--extract", "-C", "/opt", "-z", "--owner=splunk", "--group=splunk", "-f", "/var/tmp/ansible-tmp-1548625987.94-258334867164991/source"], "err": "/bin/tar: splunkforwarder/etc/apps/learned: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/learned: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/learned/metadata: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/learned: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/learned/metadata/default.meta: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/learned: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/learned/default: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/learned: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/learned/default/README: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/search: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/search: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/search/metadata: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/search: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/search/metadata/default.meta: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/search: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/search/default: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/search: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/search/default/transforms.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/search: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/search/default/restmap.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/search: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/search/default/app.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/search: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/search/default/props.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/metadata: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/metadata/default.meta: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/default: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/default/outputs.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/default/default-mode.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/default/limits.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/default/web.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/default/app.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/default/server.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/default/README: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/SplunkUniversalForwarder/default/props.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon/default: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon/default/inputs.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon/default/app.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon/default/server.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon/default/README: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon/bin: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/introspection_generator_addon/bin/collector.path: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/splunk_httpinput: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/splunk_httpinput: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/splunk_httpinput/default: Cannot mkdir: No such file or directory\n/bin/tar: splunkforwarder/etc/apps/splunk_httpinput: Cannot mkdir: Permission denied\n/bin/tar: splunkforwarder/etc/apps/splunk_httpinput/default/inputs.conf: Cannot open: No such file or directory\n/bin/tar: splunkforwarder/etc/apps: Cannot utime: Operation not permitted\n/bin/tar: Exiting with failure status due to previous errors\n", "out": "", "rc": 2}, "gid": 0, "group": "root", "handler": "TgzArchive", "mode": "0755", "msg": "failed to unpack /var/tmp/ansible-tmp-1548625987.94-258334867164991/source to /opt", "owner": "root", "size": 4096, "src": "/var/tmp/ansible-tmp-1548625987.94-258334867164991/source", "state": "directory", "uid": 0}
splunkforwarder_1  | 	to retry, use: --limit @/opt/container_artifact/ansible-retry/site.retry
splunkforwarder_1  |
splunkforwarder_1  | PLAY RECAP *********************************************************************
splunkforwarder_1  | localhost                  : ok=1    changed=0    unreachable=0    failed=1
splunkforwarder_1  |
splunkforwarder_1  | Sunday 27 January 2019  21:53:11 +0000 (0:00:03.268)       0:00:04.903 ********
splunkforwarder_1  | ===============================================================================
splunkforwarder_1  | splunk_universal_forwarder : Install Splunk universal forwarder --------- 3.27s
splunkforwarder_1  | Gathering Facts --------------------------------------------------------- 1.26s
splunkforwarder_1  | Provision role ---------------------------------------------------------- 0.13s
splunkforwarder_1  | Upgrade role ------------------------------------------------------------ 0.05s
splunkforwarder_1  | Download pre-setup playbooks -------------------------------------------- 0.05s
splunkforwarder_1  | Run pre-setup playbooks ------------------------------------------------- 0.05s
splunkforwarder_1  | Download pre-setup playbooks -------------------------------------------- 0.04s
splunkforwarder_1  | ERROR: Couldn't read "/opt/splunkforwarder/etc/splunk-launch.conf" -- maybe $SPLUNK_HOME or $SPLUNK_ETC is set wrong?
sa-demo_splunkforwarder_1 exited with code 2

As soon as I remove my app's volume mount from /opt/splunkforwarder/etc/apps everything starts up again. I presume this has to do with some permission changes made regarding the Ansible user. I suspect that my app volume mapping is creating the /opt/splunkforwarder/etc/apps directory as a different user before the ansible script is run, and ansible is unable to deal with the permission difference?

This was a very convenient method of 'installing' a forwarder app for local usage. I suppose I can build the app into my image, but I'd prefer to keep things running the way they were. Any recommendations on how to get permissions to work correctly with this scenario?

Add splunk user back to sudoers file

Please allow for optionally including the "splunk" user in the sudoers file. At the least, include a build arg, which enables it. Another option, would be to add the following lines back (removed in #72)

    && usermod -aG sudo ${SPLUNK_USER} \
    && sed -i -e 's/%sudo\s\+ALL=(ALL\(:ALL\)\?)\s\+ALL/%sudo ALL=NOPASSWD:ALL/g' /etc/sudoers \

And then include a something in the entrypoint.sh that removes them again, unless a flag is set to keep them.

Why? Under kubernetes, the /opt/splunk/etc directory is mounted with "root" as the owner. As such, splunk isn't able to install.

Add empty test_scenarios/defaults directory

In Advanced usage documentation Generating a default.yml file example

docker run splunk/splunk-debian-9:latest create-defaults > test_scenarios/defaults/default.yml

refers to a /test_secenarios/defaults directory which is not present.

Adding this empty directory would allow the examples not to fail. Just to avoid a small annoyance

splunk universal forwarder - no users exist

trying to following the documentation from: https://github.com/splunk/docker-splunk/blob/master/documentation/SETUP.md

successful
docker pull splunk/universalforwarder:latest
faiiled
docker run -d -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=1234' splunk/universalforwarder:latest

docker logs
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["/opt/splunkforwarder/bin/splunk", "enable", "listen", "9997", "-auth", "admin:1234", "--accept-license", "--answer-yes", "--no-prompt"], "delta": "0:00:00.618205", "end": "2018-10-29 16:01:44.732552", "failed_when_result": true, "msg": "non-zero return code", "rc": 24, "start": "2018-10-29 16:01:44.114347", "stderr": "No users exist. Please set up a user.", "stderr_lines": ["No users exist. Please set up a user."], "stdout": "", "stdout_lines": []}

# docker --version
Docker version 18.06.1-ce, build e68fc7a
# docker images|grep universalforwarder
splunk/universalforwarder                  latest              3b40b5f5d4a8        4 weeks ago         208MB

Recommended way to set *.conf files?

Currently I am using docker-splunk-legacy using the 7.0.3-monitor image as a standalone heavy-forwarder instance. I am creating my *.conf files (e.g., props.conf, transforms.conf, etc.) and mounting them as volumes within the heavy-forwarder container at /var/opt/splunk/etc/system/local.

I would like to use this new project. Is there a new better way to set these configuration files? I did not see anything in the ADVANCED.md doc that suggested a better way to load things like {props,transforms,inputs,outputs}.conf.

Any advice is appreciated, thank you!

Splunk standalone log error contains cleartext password

Trying to launch Splunk standalone in docker-compose displays the following error:

splunk_1 | fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["/opt/splunk/bin/splunk", "add", "licenses", "-auth", "admin:password"], "delta": "0:00:00.534752", "end": "2018-10-09 22:41:56.171553", "msg": "non-zero return code", "rc": 4, "start": "2018-10-09 22:41:55.636801", "stderr": "", "stderr_lines": [], "stdout": "missing PATH-TO-LICENSE-FILE argument: ./splunk add license [PATH-TO-FILE] ", "stdout_lines": ["missing PATH-TO-LICENSE-FILE argument: ./splunk add license [PATH-TO-FILE] "]}

Note that the line contains the arguments to the splunk command, including the auth statements (changed default password to 'password' here as a test).

First, I'm not sure why this part of the ansible script should even be executing if no license server or URL has been specified. Second, I don't think passwords should ever be printed in cleartext (I know it's being set via an environment variable, but those are only accessible to individuals with host access. The log output could be sent to an insecure monitoring setup, or to splunk itself).

Document SPLUNK_APPS_URL param

The documentation does not appear to make mention of the SPLUNK_APPS_URL parameter-- and, unfortunately, when I tested it, it did not appear to properly install any of the splunkbase app urls I supplied.

Is this intentional?

Thank you for your time

background

I have a usecase for spinning up ephemeral Splunk Heavy Forwarders inside k8 pods; their only job being to forward RFC5424 syslogs on to a corporate Splunk receiver. It would be incredibly, joyously convenient & clean for me to use this Docker image (rather than the docker-legacy 7.0.3-monitoring image) in the k8 deployment, and simply use the SPLUNK_APPS_URL param to install the RFC5424 syslog formatting add-on on the heavy forwarder.

SPLUNK_USER is root in universalforwarder

##Details :
I want to run splunk-universalforwarder with non-root user. I checked the detailed docker file and from this I thought it should be user splunk.
But when I run the docker image and check using docker exec I see the user to be root (though everything inside is with splunk:splunk permission).

I am using splunk/universalforwarder:7.1.0 image.

To reproduce :

docker run -d  -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license --seed-passwd changeme' splunk/universalforwarder:7.1.0

docker exec -it <container id> bash

OR

docker run -it  -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=splunk!!!NEW0' splunk/universalforwarder:7.1.0

# enter the password again when prompted
# After "Done", from another terminal

docker exec -it <container id> bash

You can see : root@<container id>$

Even in the documentation here it is written that user is splunk, but when I run it just runs with root.

Could you please help me here ? Thanks.

PS : I tried with splunk/universalforwarder:latest also and the results are same!

The user / perms model on develop is awkward

Hey,

I know you all are working on users, perms, etc. right now. My understanding is that on the current "develop" branch, ansible is the default user, which sudo's to splunk to set things up. This is fine, but it leaves manual interactions a bit awkward.

$ kc exec search-0 -- /opt/splunk/bin/splunk version
Splunk 7.2.1 (build be11b2c46e23)

Warning: cannot create "/opt/splunk/var/log/splunk"

Warning: cannot create "/opt/splunk/var/log/introspection"

Warning: cannot create "/opt/splunk/var/log/watchdog"
$ kubectl exec search-0 -- whoami
ansible
$ kubectl exec search-0 -- sudo -u splunk /opt/splunk/bin/splunk version
Splunk 7.2.1 (build be11b2c46e23)

This is certainly not a deal breaker, but I thought I'd raise it as an issue while you're still working things out.

Cheers!

Silent failure on building a docker container with a password that doesn't conform to the default requirements

Apparently, if you use a password that doesn't conform to the default authentication prefernces, the docker image will start up just fine, no errors or warnings, but client login will fail with error 401 "No users exist. Please set up a user."

So couple of issues:

  • The password requirement is only documented in dockerhub (https://hub.docker.com/r/splunk/splunk/), but not in github, and could be more detailed.

  • Docker should not start up without any warning logs if admin account failed to be created.

For full commands and logs, see the Stackoverflow question I opened:

https://stackoverflow.com/questions/52689758/deploying-splunk-docker-image-fails-to-login/52696725#52696725

JMS Modular Input

Hi All,

May i know how to add JMS Modular Input to Splunk docker ?
I try to mount the folder but failed.

Appreciate for any advice

Jason

How to enable receiving?

How do I start a new container with receiving enabled on port 9997?

This seems like a very common use case, yet I don't see any options to automate this. I'm simply trying to set up a demo environment with both splunk and splunk-uf containers.

For that matter, how can I set up a forwarder container to automatically forward to a standalone container?

Ansible problem after 7.2.1.

Operating System: Ubuntu 16.04
Docker Version: 18.06.1-ce

Attempted to run

docker run -d -p 8000:8000 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=12345678910' splunk/splunk:latest

and was met with this the container quickly exited. Checking docker logs provided me with:

PLAY [Run default Splunk provisioning] *****************************************
Saturday 29 December 2018  01:25:36 +0000 (0:00:00.034)       0:00:00.034 ***** 

TASK [Gathering Facts] *********************************************************
fatal: [localhost]: FAILED! => {"msg": "Failed to set permissions on the temporary files Ansible needs to create when becoming an unprivileged user (rc: 1, err: chown: changing ownership of '/var/tmp/ansible-tmp-1546046736.73-44831970175188/': Operation not permitted\nchown: changing ownership of '/var/tmp/ansible-tmp-1546046736.73-44831970175188/AnsiballZ_setup.py': Operation not permitted\n}). For information on working around this, see https://docs.ansible.com/ansible/become.html#becoming-an-unprivileged-user"}
	to retry, use: --limit @/opt/container_artifact/ansible-retry/site.retry

PLAY RECAP *********************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1   

Saturday 29 December 2018  01:25:36 +0000 (0:00:00.284)       0:00:00.318 ***** 
=============================================================================== 
Gathering Facts --------------------------------------------------------- 0.28s

This happened for tags: 7.2.2, 7.2.3, 7.2, and latest.

SPLUNK_ADD, SPLUNK_CMD, SPLUNK_APPS_URL not working

I'm trying out Splunk with Docker Compose. My environment section looks like:

environment:
  - SPLUNK_ROLE=splunk_standalone
  - SPLUNK_PASSWORD=MySuperSecurePasswordHere
  - SPLUNK_START_ARGS=--accept-license
  - SPLUNK_HOME=/opt/splunk
  - SPLUNK_APPS_URL=https://splunkbase.splunk.com/app/978/

  - SPLUNK_ADD=udp 1514 -sourcetype syslog
  - SPLUNK_ADD_1=tcp 1514 -sourcetype syslog

  - SPLUNK_CMD=add udp 1514 -sourcetype syslog

The password is set as expected, but the specified app doesn't get installed, and no inputs are added (I tried both SPLUNK_ADD and SPLUNK_CMD to add them).

Full log from docker service logs -f sinks_splunk doesn't show anything about SPLUNK_ADD, SPLUNK_CMD or SPLUNK_APPS_URL.

Need Docker container environment variables for the Deployment Server ports

If I specify a port on the host defined in the SPLUNK_DEPLOYMENT_SERVER environment variable passed to the Splunk Universal Forwarder docker container I get the error below.

fatal: [localhost]: FAILED! => {"attempts": 50, "changed": true, "cmd": ["/opt/splunkforwarder/bin/splunk", "set", "deploy-poll", "splunk-enterprise:8089:9997", "-auth", "admin:dKpNH4hB5bKvDB"], "delta": "0:00:00.454031", "end": "2018-10-08 19:32:07.967105", "msg": "non-zero return code", "rc": 22, "start": "2018-10-08 19:32:07.513074", "stderr": "Invalid host and/or port given, targetUri=splunk-enterprise:8089:9997", "stderr_lines": ["Invalid host and/or port given, targetUri=splunk-enterprise:8089:9997"], "stdout": "", "stdout_lines": []}

Appears that the Splunk Forwarded appends :9997 to the variable value.
If I don't specify the port, the Splunk Universal Forwarder connects fine to the Splunk Deployment Server.

I exchanged an email with Brian Bingham (@lephino) from Splunk. His response:

There are 2 major vars for ports: s2s_port and svc_port. To specify them you should pass in a defaults file that overrides them (in the advanced.md section). We could expose these to env vars if it’s easier than the default file.

Yes, environment variables are needed and would make the deployment easier.

My docker-compose.yaml file which defines the Splunk Forwarder service.

splunk-forwarder:
image: splunk/universalforwarder:latest
hostname: splunk-forwarder
depends_on:
- splunk-enterprise
networks:
- splunk-backend
deploy:
mode: global
ports:
- "1514:1514"
- "9997:9997"
environment:
- SPLUNK_USER=root
- SPLUNK_PASSWORD=Hidden
- SPLUNK_START_ARGS=--accept-license
- SPLUNK_FORWARD_SERVER='splunk-enterprise:9997'
- SPLUNK_FORWARD_SERVER_ARGS=-method clone
- SPLUNK_ADD_1=tcp 1514 -sourcetype syslog
- SPLUNK_DEPLOYMENT_SERVER='splunk-enterprise:8089'
volumes:
- splunk-forwarder-etc:/opt/splunk/etc
- splunk-forwarder-var:/opt/splunk/var
- /var/lib/docker/containers:/host/containers:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- /var/log:/docker/log:ro

Not able to browse apps

Hi All,

I not able to browse apps, i want to install JMS messaging modular input. Im running splunk docker in corporate network. I configured proxy. What firewall rule need to open ?

not able to browse apps

Appreciate for any advice.
Jason

Capability to specify user-prefs.conf for admin user

I see that I can add a conf in the defaults.yml, but it is pretty limited, as it writes configurations only to server/local/*.conf.
I want to be able to write a configuration for the admin user, which should go to the $SPLUNK_ETC/users/admin/user-prefs/local/user-prefs.conf, something like

[general]
default_namespace = appboilerplate
search_use_advanced_editor = true
search_line_numbers = false
search_auto_format = false
search_syntax_highlighting = dark
# tz = America/Los_Angeles

hec_enableSSL in default.yml is not working

I set hec_enableSSL to 0 in default.yml to just use http connection for this server, but when I started up the application, the option Enable SSL in Global Settings of HTTP Event Collector is still enabled. Is this parameter not support in the standalone version of 7.2.1?

default.yml is like this

---
retry_num: 50
splunk:
  app_paths:
    default: /opt/splunk/etc/apps
    httpinput: /opt/splunk/etc/apps/splunk_httpinput
    idxc: /opt/splunk/etc/master-apps
    shc: /opt/splunk/etc/shcluster/apps
  exec: /opt/splunk/bin/splunk
  group: splunk
  hec_disabled: 0
  hec_enableSSL: 0
  hec_port: 8088
  hec_token: 481673f6-444f-424c-a59e-104015849982
  home: /opt/splunk
  http_port: 8000
  idxc:
    enable: false
    label: idxc_label
    replication_factor: 3
    replication_port: 4001
    search_factor: 3
    secret: 4fCiSGT1o+w7CwiysORYgtTMHlzZVo5z
  opt: /opt
  password: my-secret-pw
  pid: /opt/splunk/var/run/splunk/splunkd.pid
  s2s_port: 9997
  shc:
    enable: false
    label: shc_label
    replication_factor: 3
    replication_port: 4001
    secret: xYLrWSz5YnGa+zwWGUrqDXOcyD828TF9
  svc_port: 8089
  user: splunk

Fatal error (ignored): missing PATH-TO-LICENSE-FILE argument

Here's an excerpt from the log from a Docker container that is based on the Splunk 7.2.0 Docker image:

TASK [splunk_common : Apply Splunk license] ************************************
Tuesday 27 November 2018  07:45:54 +0000 (0:00:00.104)       0:01:30.849 ******
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["/opt/splunk/bin/splunk", "add", "licenses", "-auth", "admin:changeme"], "delta": "0:00:03.012263", "end": "2018-11-27 07:45:57.518058", "msg": "non-zero return code", "rc": 4, "start": "2018-11-27 07:45:54.505795", "stderr": "", "stderr_lines": [], "stdout": "missing PATH-TO-LICENSE-FILE argument: ./splunk add license [PATH-TO-FILE] ", "stdout_lines": ["missing PATH-TO-LICENSE-FILE argument: ./splunk add license [PATH-TO-FILE] "]}
...ignoring

Is there a way to avoid or suppress that "fatal" error?

The Docker container starts okay, so... there's actually no real problem here. I just don't like that "fatal" error appearing in the log.

Here's why: I'm going to publish a Docker image that is based on this Splunk Docker image. In the documentation that accompanies my image, I have a troubleshooting section that asks users to send the output from docker logs. Users might notice that "fatal" error, ask about it, perhaps point that out as the potential problem. I'd prefer not to have to deal with that.

Possible alternatives to suppressing that error:

  • Would you consider changing the error message severity level so that it's not reported as "fatal"? If an error occurs that you can ignore, and then continue processing, then, by any definition I'm aware of, it's not fatal.
  • Is there a way to start a container with a Free license, rather than, as now, an Enterprise Trial license? I haven't looked into the contents of license files. Would it be possible to supply a license file that says "this is a Splunk Free license"? (Or is it the case that Splunk Free doesn't require/use a license file?)

Can't restart splunk docker after ubuntu upgrade

VM Ubuntu 16.04

I use a splunk docker on en VM ubuntu. After the upgrade, I restarted the VM but splunk docker can't start.

splunk | Friday 19 October 2018 03:09:30 +0000 (0:00:41.411) 0:00:41.565 ********
splunk | ===============================================================================
splunk | Gathering Facts -------------------------------------------------------- 41.41s
splunk | [WARNING]: provided hosts list is empty, only localhost is available. Note
splunk | that the implicit localhost does not match 'all'
splunk |
splunk |
splunk | PLAY [localhost] ***************************************************************
splunk |
splunk | TASK [Gathering Facts] *********************************************************
splunk | Friday 19 October 2018 03:09:40 +0000 (0:00:00.197) 0:00:00.198 ********
splunk | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TimeoutError: Timer expired after 10 seconds
splunk | fatal: [localhost]: FAILED! => {"changed": false, "cmd": "/bin/lsblk --list --noheadings --paths --output NAME,UUID --exclude 2", "msg": "Timer expired after 10 seconds", "rc": 257}
splunk | to retry, use: --limit @/opt/ansible/ansible-retry/site.retry
splunk |
splunk | PLAY RECAP *********************************************************************
splunk | localhost : ok=0 changed=0 unreachable=0 failed=1
splunk |
splunk | Friday 19 October 2018 03:09:59 +0000 (0:00:19.578) 0:00:19.777 ********
splunk | ===============================================================================
splunk | Gathering Facts -------------------------------------------------------- 19.58s
splunk exited with code 2

All worked well before.
Do you have any ideas?

Write configurations to the apps/{app_name}/local

Is there are any capability to write a configuration to not system/local path?

I am trying to solve simple scenario, write to the $SPLUNK_ETC/apps/splunk_instrumentation/local/telemetry.conf

[general]
optInVersionAcknowledged = 3
showOptInModal = 0

SPLUNK_ADD does not perform any actions

I'm starting Splunk for development purposes using Docker via Ansible like this:

---

- name: create volumes for splunk
  docker_volume:
    name: '{{ item }}'
  with_items:
    - opt-splunk-etc
    - opt-splunk-var

- name: create container for splunk
  docker_container:
    name: lbc-splunk
    image: splunk/splunk
    restart_policy: always
    keep_volumes: False
    recreate: True
    networks:
      - name: '{{ lbc_dedicated_network }}'
        ipv4_address: '{{ lbc_dedicated_network_splunk_address }}'
    purge_networks: yes
    published_ports:
      - 8000:8000
    env:
      SPLUNK_START_ARGS: --accept-license --answer-yes
      SPLUNK_PASSWORD: devsplunk
      SPLUNK_ADD: tcp 1514 -sourcetype log4j
      USER: root
    volumes:
      - opt-splunk-etc:/opt/splunk/etc
      - opt-splunk-var:/opt/splunk/var

but when I try to connect to Splunk - there is no TCP data input source present so I have to add it manually. I had no success calling

docker exec /opt/splunk/bin/splunk add tcp 1514 -sourcetype log4j -auth admin:devsplunk

too because I receive the following error:

stat: /opt/splunk/bin/splunk: No such file or directory

Is there any way to overcome the problem? Do I need the Developer License for Splunk to work correctly?

Remote branch the_next_century not found in upstream origin

Trying out the make all instructions today and got this error about a missing branch the_next_century

git remote -v
origin	[email protected]:splunk/docker-splunk.git (fetch)
origin	[email protected]:splunk/docker-splunk.git (push)
if [ -d "splunk-ansible" ]; then \
		echo "Ansible directory exists - skipping clone"; \
	else \
		git clone https://github.com/splunk/splunk-ansible.git --branch the_next_century; \
	fi
Cloning into 'splunk-ansible'...
fatal: Remote branch the_next_century not found in upstream origin

Docs Confusion Around Universal Forwarder

Is the Dockerhub splunk/universalforwarder image the same underlying image in this repo? The docs mention valid Universal Forwarder environment variables but SPLUNK_ROLE=splunk_universal_forwarder is not valid.

screen shot 2018-11-14 at 11 15 39 am

There are no links from Dockerhub to the Advanced README for splunk/universalforwarder. If these environment variables are valid to use with splunk/universalforwarder, would it be possible to update Dockerhub to point to the Advanced README?

See this section from splunk/splunk that links out to the Advanced README. Something like this for splunk/universalforwarder would be helpful.

screen shot 2018-11-14 at 10 46 25 am

If they aren't applicable, the Advanced README is misleading since the only valid forwarder role is splunk_heavy_forwarder and there's only one mention of Universal Forwarder in the entire README.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.