logstash-plugins / logstash-input-jmx Goto Github PK

License: Apache License 2.0

Ruby 100.00%

logstash-input-jmx's Introduction

Logstash Plugin

This is a plugin for Logstash.

It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.

Documentation

Logstash provides infrastructure to automatically generate documentation for this plugin. We use the asciidoc format to write documentation so any comments in the source code will be first converted into asciidoc and then into html. All plugin documentation are placed under one central location.

For formatting code or config example, you can use the asciidoc [source,ruby] directive
For more asciidoc formatting tips, see the excellent reference here https://github.com/elastic/docs#asciidoc-guide

Need Help?

Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.

Developing

1. Plugin Developement and Testing

Code

To get started, you'll need JRuby with the Bundler gem installed.
Create a new plugin or clone and existing from the GitHub logstash-plugins organization. We also provide example plugins.
Install dependencies

bundle install

Test

Update your dependencies

bundle install

Run tests

bundle exec rspec

2. Running your unpublished Plugin in Logstash

2.1 Run in a local Logstash clone

Edit Logstash Gemfile and add the local plugin path, for example:

gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"

Install plugin

# Logstash 2.3 and higher
bin/logstash-plugin install --no-verify

# Prior to Logstash 2.3
bin/plugin install --no-verify

Run Logstash with your plugin

bin/logstash -e 'filter {awesome {}}'

At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.

2.2 Run in an installed Logstash

You can use the same 2.1 method to run your plugin in an installed Logstash by editing its Gemfile and pointing the :path to your local plugin development directory or you can build the gem and install it using:

Build your plugin gem

gem build logstash-filter-awesome.gemspec

Install the plugin from the Logstash home

# Logstash 2.3 and higher
bin/logstash-plugin install --no-verify

# Prior to Logstash 2.3
bin/plugin install --no-verify

Start Logstash and proceed to test the plugin

Contributing

All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.

Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.

It is more important to the community that you are able to contribute.

For more information about contributing, see the CONTRIBUTING file.

logstash-input-jmx's People

Contributors

Stargazers

Watchers

logstash-input-jmx's Issues

stall detection vs. jmx-input #4793

Moved from elastic/logstash#4793 and created by @boernd

Version: 2.2.2

When using the jmx-input plugin the logstash process won't stop.

Logsnippet of one stalling thread:
: stalling_threads_info=>[{ "thread_id"=>39, "name"=>"[base]<jmx", "plugin"=>nil, "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/logstash-input-jmx-2.0.2/lib/logstash/inputs/jmx.rb:356:in sleep" },

Workaround: I set the KILL_ON_STOP_TIMEOUT to 1, the --allow-unsafe-shutdown parameter did not work for me.

Offline Install jmx4r dependency

Trying to install offline and it seems to be looking for jmx4r. When I successfully install that locally first via: <logstash_dir>/vendor/jruby/bin/jruby -S gem install --local jmx4r-0.1.4.gem I continue to get the dependency error on the local install:

bin/plugin install --local logstash-input-jmx-2.0.2.gem
Validating logstash-input-jmx-2.0.2.gem
Installing logstash-input-jmx
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "jmx4r":
In Gemfile:
logstash-input-jmx (= 2.0.2) java depends on
jmx4r (>= 0) java
Could not find gem 'jmx4r (>= 0) java', which is required by gem 'logstash-input-jmx (= 2.0.2) java', in any of the sources.

not able to install logstash-input-jmx

Please post all product and debugging questions on our forum. Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.

For all general issues, please provide the following details for fast resolution:

Version: 5.0.2
Operating System: RHEL 7.2
Config File (if you have sensitive info, please remove it):
Sample Data:

bin/logstash-plugin install logstash-input-jmx
Validating logstash-input-jmx
Installing logstash-input-jmx
Error Bundler::InstallError, retrying 1/10
An error occurred while installing logstash-core-event-java (5.0.2), and Bundler cannot continue.
Make sure that gem install logstash-core-event-java -v '5.0.2' succeeds before bundling.
WARNING: SSLSocket#session= is not supported

Steps to Reproduce:

Logstash-input-jmx new feature proposal

I used the JMX plugin to capture data from a WSO2 platform nodes, reading the attributes exposed by their JMX Beans.
But not all data can be captured reading attributes, many data can be read only invoking specific methods and passing parameters.
So I investigated the JMX plugin (I know Java, C++, Delphi, ecc. but I dont know Ruby) and I saw that the plugin use a component developed by Jeff Mesnil in 2007 which interfaces the Java JMX components. This component supports attributes but also operations (methods).
So i tried to implement the operations support in JMX plugin and for some reason (I don't know Ruby!) now I am able to capture data from operations invocation.
I hope this can be a little contribution to the JMX plugin authors and possibly a new featura in the next plugin release.

CHANGES in jmx.rb:

• added in validate_configuration:
if query.has_key?("operations") && !query["operations"].respond_to?(:each)
validation_errors << BAD_TYPE_QUERY_PARAMETER % { :param => 'operations', :index => index, :expected => Enumerable, :actual => query['operations'].class }
end

• added in thread_jmx:
if query.has_key?('operations')
@logger.debug("Retrieves operations #{query['operations']} to #{jmx_object_name.object_name}")
query['operations'].each do |operation|
begin
@logger.debug("operation: #{operation[0]} params: #{operation[1].join(', ')}")
jmx_operation_value = jmx_object_name.send(operation[0], *operation[1])
if jmx_operation_value.instance_of? Java::JavaxManagementOpenmbean::CompositeDataSupport
@logger.debug('The jmx value is a composite_data one')
jmx_operation_value.each do |jmx_operation_value_composite|
@logger.debug("Get jmx value #{jmx_operation_value[jmx_operation_value_composite]} for operation #{operation}.#{jmx_operation_value_composite} to #{jmx_object_name.object_name}")
send_event_to_queue(queue, thread_hash_conf['host'], "#{base_metric_path}.#{object_name}.#{operation}.#{jmx_operation_value_composite}", jmx_operation_value[jmx_operation_value_composite])
end
else
@logger.debug("Get jmx value #{jmx_operation_value} for operation #{operation} to #{jmx_object_name.object_name}")
send_event_to_queue(queue, thread_hash_conf['host'], "#{base_metric_path}.#{object_name}.#{operation}", jmx_operation_value)
end
rescue Exception => ex
@logger.warn("Failed retrieving metrics for operation #{operation} on object #{jmx_object_name.object_name}")
@logger.warn(ex.message)
end
end
end

CHANGES in jmx configuration:
......
{
"object_name" : "java.lang:type=Threading",
"attributes" : [ "ThreadCount", "TotalStartedThreadCount", "DaemonThreadCount", "PeakThreadCount" ],
"object_alias" : "Threading"
}, {
"object_name" : "org.wso2.carbon:type=StatisticsAdmin",
"operations" : [
["getServiceRequestCount", ["MMGQueryPazienteProxy"]],
["getServiceFaultCount", ["MMGQueryPazienteProxy"]],
["getServiceResponseCount", ["MMGQueryPazienteProxy"]],
["getOperationRequestCount", ["MMGQueryPazienteProxy", "QueryPaziente"]],
["getOperationFaultCount", ["MMGQueryPazienteProxy", "QueryPaziente"]],
["getOperationResponseCount", ["MMGQueryPazienteProxy", "QueryPaziente"]]
],
"object_alias" : "StatisticsAdmin"
} ]

Operations is an array of jmx operations.
Each array element has as the first element the operation name and as the second element an array of parameters.

Graziano

Problem with "non standard" MBeans ObjectNames

I am trying to make the plug-in work with Vert.x dropwizard metrics.
The problem is that it is using weird names without specifying the type, for example:

zone.drp:name=vertx.eventbus.messages.sent
zone.drp:name=vertx.verticles

(zone.drp is my domain)

I tried many different combinations but I cannot make it work, it looks like the plugin depends on you to specify the Type so I tried even adding to the object_name something like Type=* but this does works I'm still getting the "No jmx object found"

I was able to make this work for kafka and everything else, I don't know why they named the MBeans in such an awkward way... is there anything I can do to make it work?

P.s
I tried monitoring those with zabbix jmx and also with jconsole and it works...

Improvement: Support Logstash (experimental) feature --allow-env

Logstash has the experimental feature --allow-env. This feature enables "using Environment Variables in the Configuration".

For this plugin, logstash-input-jmx, please provide support for this Logstash-feature. In special, jmxonf-files do not respect the feature (plugin configuration file does already).

Support for JMX URL

I am using your most excellent plugin with JBoss Fuse/Karaf.

Unfortunately, I cannot use the code directly, it requires some modification. Karaf has a specific URL syntax required to connect, using host and port does not work.

@logger.debug('Check if jmx connection need a user/password')
        if thread_hash_conf.has_key?('username') and thread_hash_conf.has_key?('password')
          @logger.debug("Connect to #{thread_hash_conf['host']}:#{thread_hash_conf['port']} with user #{thread_hash_conf['username']}")
          jmx_connection = JMX::MBean.connection :host => thread_hash_conf['host'],
                                                 :port => thread_hash_conf['port'],
                                                 :url => thread_hash_conf['url'],
                                                 :username => thread_hash_conf['username'],
                                                 :password => thread_hash_conf['password']

I simply added the option to pass url in from the JMX configuration file. This overrides the host and port and they are ignored by jmx4r, so more work could be done to NOT require host and port in the config if URL is provided. An example config that works is shown below.

{
  "host" : "localhost",
  "port" : 1100,
  "url"  : "service:jmx:rmi://server:44444/jndi/rmi://server:1099/karaf-root",
  "username" : "admin",
  "password": "admin",
  "alias" : "chris",
  "queries" : [...

Add support for arrays, collections and compositeData

From https://github.com/elastic/logstash-contrib/issues/128 for array and collection

It's really cool to be able to have an input plugin for JMX !
However, currently, not all data (arrays, collections) gets properly handled:
examples:
[arrays]
"metric_path" => "java.lang:type=GarbageCollector,name=ParNew.MemoryPoolNames",
"metric_value_string" => "[Ljava.lang.String;@25647d72"

[collections]:
"metric_path" => "java.lang:type=GarbageCollector,name=ParNew.LastGcInfo.memoryUsageBeforeGc",
"metric_value_string" => "{#Java::JavaUtil::Collections::UnmodifiableRandomAccessList:0x4f722930=>#Java::JavaxManagementOpenmbean::CompositeDataSupport:0x4fee370f, #Java::JavaUtil::Collections::UnmodifiableRandomAccessList:0x6091b85=>#Java::JavaxManagementOpenmbean::CompositeDataSupport:0x3fedbbdb, #Java::JavaUtil::Collections::UnmodifiableRandomAccessList:0x49669be=>#Java::JavaxManagementOpenmbean::CompositeDataSupport:0x2ef83b4c, #Java::JavaUtil::Collections::UnmodifiableRandomAccessList:0x1f12e637=>#Java::JavaxManagementOpenmbean::CompositeDataSupport:0x11c834d1, #Java::JavaUtil::Collections::UnmodifiableRandomAccessList:0x73f5504=>#Java::JavaxManagementOpenmbean::CompositeDataSupport:0x4ee67d53}"

CompositeData support was requested in https://discuss.elastic.co/t/jmx-compositedatasupport-and-array/629

metric_value_number may become integer and doubles lose fractions

If the first metric_value_number is an integer, elastic search creates the mapping type as an integer and keeps using the same type for double metrics too. This is especially very bad for the ones which are between 0 and 1, like the cpu metrics.
Maybe it can be a good idea to create 2 separate fields for numbers fields: "metric_value_long" and "metric_value_double".

Resource Leak when alias parsing fails

For all general issues, please provide the following details for fast resolution:

Version:
logstash: 2.3.1
plugin: 2.0.2
Operating System:
Linux
Config File (if you have sensitive info, please remove it):

logstash

input {
  jmx {
    path => "${MY_JMX_CONF_DIR}"
    polling_frequency => 60
    type => "jmx"
    nb_thread => 1
  }
}
filter {
  # drop any event that has a metric_string_value field.
  if ("" in [metric_value_string]) {
    drop {}
  }
  mutate {
    gsub => [
      "metric_path", "[\/\$]", "."
    ]
  }
}
output {
  graphite {
....
  }

}

jmx

{
    "host": "hostname",
    "port": 12345,
    "url": "service:jmx:rmi:///jndi/rmi://hostname:12345/jmxrmi",
    "alias": "testalias",
    "queries" : [
      {
        "object_name": "java.lang:type=*",
        "object_alias": "java.lang.${type}"
      },
      {
        "object_name": "java.lang:type=GarbageCollector,name=*",
        "object_alias": "java.lang.${type}.${name}"
      },
....

Sample Data:
Steps to Reproduce:

Description of issue

When replace_alias_object fails with the undefined method for 'split' exception, the jmx_connection.close method is never called. As a result, the input will continue to create new connections each time it runs. Using Visual VM, each thread shows up as 'JMX client heartbeat' in the threads tab. Additionally, the Threads chart continues to grow.

example exception message

{:timestamp=>"2017-03-21T15:11:16.333000-0500", :message=>"undefined method `split' for nil:NilClass", :level=>:error}

code where exception originates

  private
  def replace_alias_object(r_alias_object,object_name)
    @logger.debug("Replace ${.*} variables from #{r_alias_object} using #{object_name}")
    group_alias = @regexp_group_alias_object.match(r_alias_object)
    if group_alias
      r_alias_object = r_alias_object.gsub('${'+group_alias[1]+'}',object_name.split(group_alias[1]+'=')[1].split(',')[0])
      r_alias_object = replace_alias_object(r_alias_object,object_name)
    end
    r_alias_object
  end

The problem lies at the end of the def thread_jmx(queue_conf,queue) method

private
def thread_jmx(queue_conf,queue)
 ....
   jmx_connection.close
   rescue LogStash::ShutdownSignal
        break #free
   rescue Exception => ex
        @logger.error(ex.message)
        @logger.error(ex.backtrace.join("\n"))
    end
end

Solution

I will fix this and submit a pull request if I get the time, but I wanted to at least document this here. But, I was able to fix this locally by adding an ensure clause to the end of the rescue clauses. In the ensure clause, test to see if jmx_connection is not null, then close.

private
def thread_jmx(queue_conf,queue)
   ....
     jmx_connection.close
    rescue LogStash::ShutdownSignal
        break #free
    rescue Exception => ex
        @logger.error(ex.message)
        @logger.error(ex.backtrace.join("\n"))
    ensure
        if jmx_connection
           jmx_connection.close
    end
end

refactor shutdown sequence to use plugin.stop

Change shutdown sequence to be triggered by plugin.stop instead of ShutdownSignal exception.
Also remove any calls to: shutdown, finished, finished?, running? or terminating?
This depends on elastic/logstash#3895

Messages in queue preventing new messages coming in

I'm running both logstash 5.3 and 5.2 and I'm seeing this in the log output...

[2017-04-25T08:30:00,222][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:01,222][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:02,222][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:02,871][DEBUG][logstash.pipeline        ] Pushing flush onto pipeline
[2017-04-25T08:30:03,223][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:04,223][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:05,223][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:06,223][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:07,223][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:07,872][DEBUG][logstash.pipeline        ] Pushing flush onto pipeline
[2017-04-25T08:30:08,224][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:09,224][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:10,224][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:11,224][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:12,224][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:12,871][DEBUG][logstash.pipeline        ] Pushing flush onto pipeline
[2017-04-25T08:30:13,225][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:14,225][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:15,225][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:16,225][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:17,226][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:17,870][DEBUG][logstash.pipeline        ] Pushing flush onto pipeline
[2017-04-25T08:30:18,226][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:19,226][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:20,226][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:21,226][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:22,227][DEBUG][logstash.inputs.jmx      ] There are still 3 messages in the queue conf. Sleep 1s.
[2017-04-25T08:30:22,869][DEBUG][logstash.pipeline        ] Pushing flush onto pipeline

which is preventing new jmx messages from coming in.

my conf file looke like this:

input { 
        jmx {
          path => "/home/ec2-user/jmx-input"
          polling_frequency => 15
          nb_thread => 4
          add_field => {
                        "cluster" => "cluster1"
                }
        }

}
filter {

                ruby {  
                        code => "event.set('timejmx', event.get('@timestamp').to_f * 1000 )"
                }
}
output {
        kafka { 
                topic_id => "kafka-jmx"
                client_id => "logstash1-jmx-producer"
                bootstrap_servers => "kafka1:9092,kafka2:9092,kafka3:9092"
                acks => "0"
                key_serializer =>"org.apache.kafka.common.serialization.StringSerializer"
                codec => "json_lines"
        }

}

Whats weird is that I have this same conf setup across multiple nodes and only one of them is showing this issue. it polls JMX 3 times successfully, but then hangs on the above log output.

I have even tweaked with some logstash settings like this but still the same results:

pipeline.workers: 6
pipeline.output.workers: 3
pipeline.batch.size: 250
pipeline.unsafe_shutdown: true
queue.max_events: 500

I have stripped out the filter and also set the output to stdout and am still seeing the same behavior...

Currently the code uses double quoted string with no interpolations and/or special symbols - clean up warnings

Cleaning up warnings and code within jmx plugin to make it more readable and clear that no interpolation is happenign (literals strings etc).

undefined method `split' for nil:NilClass

Hi everyone.

We're trying to use the plugin at our company. We have various machines running logstash 2.3.4 and 2.4, but on both versions we have a problem at logstash starup that should be related to the jmx plugin:

$ tail -f /var/log/logstash/logstash.log
...
{:timestamp=>"2016-09-23T10:17:46.978000+0000", :message=>"undefined method `split' for nil:NilClass", :level=>:error}
{:timestamp=>"2016-09-23T10:17:46.978000+0000", :message=>"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-jmx-2.0.4/lib/logstash/inputs/jmx.rb:165:in `replace_alias_object'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-jmx-2.0.4/lib/logstash/inputs/jmx.rb:240:in `thread_jmx'\norg/jruby/RubyArray.java:1613:in `each'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-jmx-2.0.4/lib/logstash/inputs/jmx.rb:238:in `thread_jmx'\norg/jruby/RubyArray.java:1613:in `each'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-jmx-2.0.4/lib/logstash/inputs/jmx.rb:233:in `thread_jmx'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-jmx-2.0.4/lib/logstash/inputs/jmx.rb:321:in `run'", :level=>:error}
...

Any advice?

Thanks in advance

Bruno

Version:
- Logstash: both 2.3.4 and 2.4
- Plugin: 2.0.4
Operating System: CentOS 7
Config File:

# cat /etc/logstash/conf.d/local_input_tomcat_jmx.conf 
input {
  jmx {
    path => "/etc/logstash/jmx"
    polling_frequency => 10
    type => "jmx-tomcat"
    nb_thread => 1
    add_field => { "[@metadata][local_log]" => "" }
  }
}

Steps to Reproduce:

Install logstash from CentOS repo
Install jmx plugin
Start logstash

Use schedule instead of obsolete interval for timing

Might submit a PR if it's something the maintainer would consider merging.

Redundancy of host prefix in metric_path ?

Hi,

Is host really necessary in the metric path ?

It was not an issue for me until I tried to create a line chart in a kibana, with two buckets.

split graph bucket based on the metric_path
split line based on the host

We want to have a separate graph automatically generated per attribute, where each graph will have separate lines for each host.

If you have events (host,metric_path,time) like
(host1,host1.object.attributeA,1)
(host1,host1.object.attributeA,2)
(host2,host2.object.attributeA,1)
(host2,host2.object.attributeA,2)
(host1,host1.object.attributeB,1)
(host1,host1.object.attributeB,2)
(host2,host2.object.attributeB,1)
(host2,host2.object.attributeB,2)

then split graph on metric_path will think that it must generate 4 graphs :

host1.object.attributeA
host2.object.attributeA
host1.object.attributeB
host2.object.attributeB

while what we want is two separate graph :

object.attributeA
object.attributeB

each separate graph displaying 1 line for host1 events, and 1 line for host2 event.

Since the host value is already separatly available for filtering, what is the point of prefixing the metric_path with the host ?

I might get my solution with setting the alias as empty or something { "alias" : "" }

Doesn't support the new logstash architecture without contrib files

This input requires jmx4r gem that not provided "from the box" by logstash-1.5.0.beta1.
My workaround is install jmx4r gem to the jruby directory in logstash:

/opt/logstash/vendor/jruby/bin/jruby -S gem install --install-dir /opt/logstash/vendor/bundle/jruby/1.9 jmx4r

Plugin Currently Broken

Version: 3.0.0
Operating System: Irrelevant
Config File: Irrelevant
Sample Data: Irrelevant
Steps to Reproduce: Run version 3.0.0 of this plugin

There was an error while porting this plugin to java-style Event objects for 3.0.0. See

logstash-input-jmx/lib/logstash/inputs/jmx.rb

Lines 174 to 177 in 68b0059

 event = LogStash::Event.new 

 event('host', host) 

 event('path', @path) 

 event.set('type', @type)

When combined with the somewhat aggressive Exception handling, this results in a very easy-to-miss exception being logged at the warn level, and Logstash continues on, though no JMX results are ever received.

Connection using localhost but host is defined

Not sure what I'm doing wrong here as I'm following the instructions. I have a java app running on one server and ELK stack on a second machine. In the conf file I'm setting the connection details

{
"host" : "datahub",
"port" : 20030,
"url" : "service:jmx:rmi:///jndi/rmi://datahub:20030/jmxrmi",
"alias" : "datahub",
"queries" : [
{
"object_name" : "java.lang:type=Memory",
"object_alias" : "Memory"
}, {
"object_name" : "java.lang:type=Runtime",
"attributes" : [ "Uptime", "StartTime" ],
"object_alias" : "Runtime"
}, {
"object_name" : "java.lang:type=GarbageCollector,name=",
"attributes" : [ "CollectionCount", "CollectionTime" ],
"object_alias" : "${type}.${name}"
}, {
"object_name" : "java.nio:type=BufferPool,name=",
"object_alias" : "${type}.${name}"
}
]
}

but when I start logstash I get connection errors in the log and no messages are indexed in Elasticsearch. (Note, I have tried this with and without the url config and get the same errors)

{:timestamp=>"2016-04-19T14:55:55.218000+0000", :message=>"Loading configuration files in path", :path=>"/opt/logstash/config/jmxconf", :level=>:info, :file=>"logstash/inputs/jmx.rb", :line=>"325", :method=>"run"}
{:timestamp=>"2016-04-19T14:55:55.219000+0000", :message=>"Loading configuration from file", :file=>"logstash/inputs/jmx.rb", :level=>:debug, :line=>"330", :method=>"run"}
{:timestamp=>"2016-04-19T14:55:55.219000+0000", :message=>"Add configuration to the queue", :config=>{"host"=>"datahub", "port"=>20030, "url"=>"service:jmx:rmi:///jndi/rmi://datahub:20030/jmxrmi", "alias"=>"datahub", "queries"=>[{"object_name"=>"java.lang:type=Memory", "object_alias"=>"Memory"}, {"object_name"=>"java.lang:type=Runtime", "attributes"=>["Uptime", "StartTime"], "object_alias"=>"Runtime"}, {"object_name"=>"java.lang:type=GarbageCollector,name=", "attributes"=>["CollectionCount", "CollectionTime"], "object_alias"=>"${type}.${name}"}, {"object_name"=>"java.nio:type=BufferPool,name=", "object_alias"=>"${type}.${name}"}]}, :level=>:debug, :file=>"logstash/inputs/jmx.rb", :line=>"335", :method=>"run"}
{:timestamp=>"2016-04-19T14:55:55.220000+0000", :message=>"Wait until the queue conf is empty", :level=>:debug, :file=>"logstash/inputs/jmx.rb", :line=>"346", :method=>"run"}
{:timestamp=>"2016-04-19T14:55:55.220000+0000", :message=>"Retrieve config {"host"=>"datahub", "port"=>20030, "url"=>"service:jmx:rmi:///jndi/rmi://datahub:20030/jmxrmi", "alias"=>"datahub", "queries"=>[{"object_name"=>"java.lang:type=Memory", "object_alias"=>"Memory"}, {"object_name"=>"java.lang:type=Runtime", "attributes"=>["Uptime", "StartTime"], "object_alias"=>"Runtime"}, {"object_name"=>"java.lang:type=GarbageCollector,name=", "attributes"=>["CollectionCount", "CollectionTime"], "object_alias"=>"${type}.${name}"}, {"object_name"=>"java.nio:type=BufferPool,name=", "object_alias"=>"${type}.${name}"}]} from queue conf", :level=>:debug, :file=>"logstash/inputs/jmx.rb", :line=>"205", :method=>"thread_jmx"}
{:timestamp=>"2016-04-19T14:55:55.220000+0000", :message=>"Check if jmx connection need a user/password", :level=>:debug, :file=>"logstash/inputs/jmx.rb", :line=>"207", :method=>"thread_jmx"}
{:timestamp=>"2016-04-19T14:55:55.220000+0000", :message=>"Wait 15s (15-0(seconds wait until queue conf empty)) before to launch again a new jmx metrics collection", :level=>:debug, :file=>"logstash/inputs/jmx.rb", :line=>"355", :method=>"run"}
{:timestamp=>"2016-04-19T14:55:55.220000+0000", :message=>"Connect to datahub:20030", :level=>:debug, :file=>"logstash/inputs/jmx.rb", :line=>"216", :method=>"thread_jmx"}
{:timestamp=>"2016-04-19T14:55:55.229000+0000", :message=>"Connection refused to host: localhost; nested exception is: \n\tjava.net.ConnectException: Connection refused", :level=>:error, :file=>"logstash/inputs/jmx.rb", :line=>"297", :method=>"thread_jmx"}
{:timestamp=>"2016-04-19T14:55:55.229000+0000", :message=>"sun.rmi.transport.tcp.TCPEndpoint.newSocket(sun/rmi/transport/tcp/TCPEndpoint.java:619)\nsun.rmi.transport.tcp.TCPChannel.createConnection(sun/rmi/transport/tcp/TCPChannel.java:216)\nsun.rmi.transport.tcp.TCPChannel.newConnection(sun/rmi/transport/tcp/TCPChannel.java:202)\nsun.rmi.server.UnicastRef.invoke(sun/rmi/server/UnicastRef.java:130)\njavax.management.remote.rmi.RMIConnector.getConnection(javax/management/remote/rmi/RMIConnector.java:2430)\njavax.management.remote.rmi.RMIConnector.connect(javax/management/remote/rmi/RMIConnector.java:308)\njavax.management.remote.JMXConnectorFactory.connect(javax/management/remote/JMXConnectorFactory.java:270)\njava.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)\nRUBY.create_connection(/opt/logstash/vendor/bundle/jruby/1.9/gems/jmx4r-0.1.4/lib/jmx4r.rb:220)\nRUBY.connection(/opt/logstash/vendor/bundle/jruby/1.9/gems/jmx4r-0.1.4/lib/jmx4r.rb:137)\nRUBY.thread_jmx(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-jmx-2.0.4/lib/logstash/inputs/jmx.rb:217)\nRUBY.run(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-jmx-2.0.4/lib/logstash/inputs/jmx.rb:321)\njava.lang.Thread.run(java/lang/Thread.java:745)", :level=>:error, :file=>"logstash/inputs/jmx.rb", :line=>"298", :method=>"thread_jmx"}
{:timestamp=>"2016-04-19T14:55:55.230000+0000", :message=>"Wait config to retrieve from queue conf", :level=>:debug, :file=>"logstash/inputs/jmx.rb", :line=>"203", :method=>"thread_jmx"}

Am I missing something from my config file?

Loading configuration files... info logging

Is there any reason to info-log Loading configuration files in path [...] on every run?
I think it really belongs into the debug log as it's a lot of (not particularly useful) data filling the logfiles.
Especially with a low interval it can easily cause missing error/fatal messages.

Logstash stop processing metrics if one of the server stop responding

have configured logstash with jmx input:

input { jmx { path => "/opt/logstash/configs/jmx/" polling_frequency => 10 type => "jmx" nb_thread => 4 } }

In /opt/logstash/configs/jmx/ direcroty I have a list of servers with JMX port. In case of one of the server jmx stops responding, logstash stops processing metrics from all other servers. In the log I found this error:
{:timestamp=>"2016-10-17T19:19:44.856000+0000", :message=>"Error unmarshaling return header; nested exception is: \n\tjava.io.EOFException", :level=>:error} {:timestamp=>"2016-10-17T19:19:44.857000+0000", :message=>"sun.rmi.transport.StreamRemoteCall.executeCall(sun/rmi/transport/StreamRemoteCall.java:229)\nsun.rmi.server.UnicastRef.invoke(sun/rmi/server/UnicastRef.java:162)\njavax.management.remote.rmi.RMIConnector.getConnectionId(javax/management/remote/rmi/RMIConnector.java:384)\njavax.management.remote.rmi.RMIConnector.connect(javax/management/remote/rmi/RMIConnector.java:346)\njavax.management.remote.JMXConnectorFactory.connect(javax/management/remote/JMXConnectorFactory.java:270)\njava.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)\nJMX::MBean.create_connection(/opt/logstash-jmx/vendor/bundle/jruby/1.9/gems/jmx4r-0.1.4/lib/jmx4r.rb:220)\nJMX::MBean.create_connection(/opt/logstash-jmx/vendor/bundle/jruby/1.9/gems/jmx4r-0.1.4/lib/jmx4r.rb:220)\nJMX::MBean.create_connection(/opt/logstash-jmx/vendor/bundle/jruby/1.9/gems/jmx4r-0.1.4/lib/jmx4r.rb:216)\nJMX::MBean.create_connection(/opt/logstash-jmx/vendor/bundle/jruby/1.9/gems/jmx4r-0.1.4/lib/jmx4r.rb:216)\nJMX::MBean.connection(/opt/logstash-jmx/vendor/bundle/jruby/1.9/gems/jmx4r-0.1.4/lib/jmx4r.rb:137)\nJMX::MBean.connection(/opt/logstash-jmx/vendor/bundle/jruby/1.9/gems/jmx4r-0.1.4/lib/jmx4r.rb:137)\nRUBY.thread_jmx(/opt/logstash-jmx/vendor/bundle/jruby/1.9/gems/logstash-input-jmx-2.0.4/lib/logstash/inputs/jmx.rb:217)\nRUBY.run(/opt/logstash-jmx/vendor/bundle/jruby/1.9/gems/logstash-input-jmx-2.0.4/lib/logstash/inputs/jmx.rb:321)\njava.lang.Thread.run(java/lang/Thread.java:745)", :level=>:error}

Why logstash can`t just skip problematic server and keep on processing metrics from all other servers?

Logstash 5.x Compatibility

Hi,
I'm getting an error message when trying to install this plugin for rc1 release.

Version: 5.0.0-rc1
Steps to Reproduce: ./bin/logstash-plugin install --no-verify logstash-input-jmx
Error Msg:
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core-plugin-api":
In snapshot (Gemfile.lock):
logstash-core-plugin-api (= 2.1.16)
...

Missing data in elastic search

I just started using the JMX plugin and I'm missing some metrics in elastic search (Memory.HeapMemoryUsage.* and Runtime.StartTime).

When I use the stdout debug in logstash, I'm seeing the data but they are missing when going to elastic search.

Example data received by elastic search:

My JMX config:
{ "host": "127.0.0.1", "port": 1335, "alias": "host.name__", "queries": [{ "object_name": "java.lang:type=Memory", "object_alias": "Memory" }, { "object_name" : "java.lang:type=Threading", "attributes": ["ThreadCount"], "object_alias": "Threading" }, { "object_name": "java.lang:type=Runtime", "attributes": ["Uptime", "StartTime"], "object_alias": "Runtime" }, { "object_name": "java.lang:type=GarbageCollector,name=", "attributes": ["CollectionCount", "CollectionTime"], "object_alias": "${type}.${name}" }, { "object_name": "java.nio:type=BufferPool,name=", "object_alias": "${type}.${name}" }] }

I tried limiting the memory object to only heap memory but that didn't help. Any ideas would be helpful.

Thanks,
Cory

Host field in events

The plugin overwrite the default logstash event field "host" with the value specified in the jmx config file.

SSL support

Does logstash-input-jmx support connecting through SSL? I've tried and keep getting:

Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: non-JRMP server at remote endpoint]

Can't install the plugin

Is there any way we can install it off line?

$ ./logstash-plugin install --no-verify logstash-input-jmx
Installing logstash-input-jmx
Error Bundler::HTTPError, retrying 1/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 2/10
Could not fetch specs from https://rubygems.org/

	event = LogStash::Event.new
	event('host', host)
	event('path', @path)
	event.set('type', @type)