robcowart / eslog_tutorial Goto Github PK

Rob, I have been following you for a while now. Absolutely love your work.

I have run into a somewhat odd issue I am hoping you can provide some input for. I am throwing up a new ES lab stack. CentOS 7/Logstash 6.2.3 alone on this box. I tried to do a very simple Syslog pipeline and got the below error. Confused (I have used this pipeline before) I decided to just throw your tutorial pipeline in unmodified. beyond ENV_VAR paths and got the exact same error. Your elastiflow pipeline works perfectly.....

Am I missing something stupidly obvious, or has something changed recently. This is my first 6.2x install.

Everything is in /etc/logstash/syslog/*

[user@logstash logstash]$ cat pipelines.yml
# This file is where you define your pipelines. You can define multiple.
# For more information on multiple pipelines, see the documentation:
# https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html

• pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"

• pipeline.id: elastiflow
  path.config: "/etc/logstash/elastiflow/conf.d"

• pipeline.id: syslog
  path.config: "/etc/logstash/syslog/conf.d"

[user@logstash logstash]$ sudo cat /etc/systemd/system/logstash.service.d/syslog.conf
#------------------------------------------------------------------------------
# Copyright (C)2018 KOIOSSIAN
#
# The contents of this file and/or repository are subject to the Koiossian
# Public License (the "License") and may not be used or distributed except in
# compliance with the License. You may obtain a copy of the License at:
#
# http://www.koiossian.com/public/koiossian_public_license.txt
#
# Software distributed under the License is distributed on an "AS IS" basis,
# WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for
# the specific language governing rights and limitations under the License.
#
# The Original Source Code was developed by Koiossian. Portions created by
# Koiossian are Copyright (C)2018 Koiossian. All Rights Reserved.
#------------------------------------------------------------------------------

[Service]
# Syslog
Environment="ESLOG_BASE=/etc/logstash/syslog/"
Environment="ESLOG_DICTIONARY_PATH=$ESLOG_BASE/dictionaries"
Environment="ESLOG_GROK_PATTERNS_DIR=$ESLOG_BASE/patterns"
Environment="ESLOG_GEOIP_DBS_DIR=$ESLOG_BASE/geoipdbs"
Environment="ESLOG_TEMPLATE_PATH=$ESLOG_BASE/templates"
Environment="ESLOG_LOGS=$ESLOG_BASE/logs"
Environment="ESLOG_ELASTICSEARCH_HOSTS=elastic:9200"
Environment="ESLOG_ELASTICSEARCH_USER=elastic"
Environment="ESLOG_ELASTICSEARCH_PASSWORD=changeme"
Environment="ESLOG_SYSLOG_PORT=514"
Environment="ESLOG_SYSLOG_TZ=UTC"

[logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:syslog, :exception=>"LogStash::ConfigurationError", :message=>"Something is wrong with your configuration.", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/config/mixin.rb:89:in config_init'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:128:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:22:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/plugins/plugin_factory.rb:87:in plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:112:in plugin'", "(eval):255:in '", "org/jruby/RubyKernel.java:994:in eval'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:84:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in block in converge_state'", "org/jruby/RubyArray.java:1734:in each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in block in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:in block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}