Git Product home page Git Product logo

log4j2-gelf's Introduction

Deprecation notice

This project has been deprecated as of 2018-01-16 and will not be maintained anymore.

If you're looking for a GELF appender for Log4j 2, we recommend using logstash-gelf by Mark Paluch.

DEPRECATED


GELF Appender for Apache Log4j 2

License Build Status Maven Central

This appender for Apache Log4j 2 logs messages to a GELF server like Graylog or logstash.

It's using the official GELF Java client to connect to a remote server.

You can specify the following parameters for the GELF appender in the log4j2.xml configuration file:

  • name
    • The reference name of the Appender
  • server (default: localhost)
    • The host name or IP address of the GELF server
  • port (default: 12201)
    • The port the GELF server is listening on
  • hostName (default: the local host name or localhost if it couldn't be detected)
    • The host name of the machine generating the logs
  • protocol (default: UDP)
    • The transport protocol to use
  • tlsEnabled (default: false)
    • Whether TLS should be enabled
  • tlsEnableCertificateVerification (default: true)
    • Whether the TLS certificate chain should be checked
  • tlsTrustCertChainFilename (default: empty)
    • A X.509 certificate chain file in PEM format for certificate verification
  • includeSource (default: true)
    • Whether the source of the log message should be included
  • includeThreadContext (default: true)
  • includeStackTrace (default: true)
    • Whether a full stack trace should be included
  • includeExceptionCause (default: false)
    • Whether the included stack trace should contain causing exceptions
  • queueSize (default: 512)
    • The size of the internally used queue
  • connectTimeout (default: 1000)
    • The connection timeout for TCP connections in milliseconds
  • reconnectDelay (default: 500)
    • The time to wait between reconnects in milliseconds
  • sendBufferSize (default: -1)
    • The size of the socket send buffer in bytes. A size of -1 deactivates the send buffer
  • tcpNoDelay (default: false)
    • Whether Nagle's algorithm should be used for TCP connections
  • tcpKeepAlive (default: false)
    • Whether to try keeping alive TCP connections.
  • Filter elements
    • A Filter to determine if the event should be handled by this Appender
  • Layout element (default: none)
    • The Layout used to format the LogEvent
  • ignoreExceptions
    • The default is true, causing exceptions encountered while appending events to be internally logged and then ignored. When set to false exceptions will be propagated to the caller, instead. Must be set to false when wrapping this Appender in a FailoverAppender.
  • Additional Fields
    • Additional fields can be added with KeyValuePair elements inside the GELF element.

log4j2.xml example

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="OFF" packages="org.graylog2.log4j2">
    <Appenders>
        <GELF name="gelfAppender" server="graylog2.example.com" port="12201" hostName="appserver01.example.com">
            <PatternLayout pattern="%logger{36} - %msg%n"/>
            <Filters>
                <Filter type="MarkerFilter" marker="FLOW" onMatch="DENY" onMismatch="NEUTRAL"/>
                <Filter type="MarkerFilter" marker="EXCEPTION" onMatch="DENY" onMismatch="ACCEPT"/>
            </Filters>
            <!-- Additional fields -->
            <KeyValuePair key="foo" value="bar"/>
            <KeyValuePair key="jvm" value="${java:vm}"/>                
        </GELF>
    </Appenders>
    <Loggers>
        <Root level="info">
            <AppenderRef ref="gelfAppender"/>
        </Root>
    </Loggers>
</Configuration>

Java code example

Logger logger = LogManager.getLogger("test");
ThreadContext.put("userId", "testUser");
logger.info("Hello World");

Using variables in the additional fields

The additional fields (KeyValuePair elements) can contain references to variables.

In order for Log4j 2.x to resolve the variable's value, the variable name must have a certain prefix depending on how the variable is provided.

Internally we're making use of Log4j's StrSubstitutor to resolve the variable's value.

This in turn is utilizing the following Log4j Lookups with the prefixes in the following list:

Prefix Documentation
ctx ContextMapLookup
bundle ResourceBundleLookup
date DateLookup
env EnvironmentLookup
java JavaLookup
jndi JndiLookup
jvmrunargs JmxRuntimeInputArgumentsLookup
map MapLookup
sd StructuredDataLookup
sys SystemPropertiesLookup

Please read up on the different variable handling in the linked Javadocs.

Example configuration with variables

<GELF name="gelfAppender" server="graylog2.example.com" port="12201" hostName="appserver01.example.com">
    <KeyValuePair key="user" value="${env:USER}"/>
    <KeyValuePair key="CLIargument" value="${sys:cliargument}"/>
    <KeyValuePair key="jvm" value="${java:vm}"/>
    <KeyValuePair key="fileEncoding" value="${sys:file.encoding}"/>
</GELF>

Versions

GELF Appender for Apache Log4j 2 Release date
1.0.0 29-Sep-2014
1.0.1 21-Oct-2014
1.0.2 09-Feb-2015
1.0.3 11-Feb-2015
1.1.0 16-Jul-2015
1.2.0 18-Jul-2016
1.2.1 11-Aug-2016
1.3.0 24-Aug-2016
1.3.1 22-Oct-2016

This appender uses GELF Java client 1.4.1.

Installation

Maven coordinates

<dependencies>
    <dependency>
        <groupId>org.graylog2.log4j2</groupId>
        <artifactId>log4j2-gelf</artifactId>
        <version>1.3.1</version>
    </dependency>
</dependencies>

License

GELF Appender for Apache Log4j 2

Copyright (C) 2014 TORCH GmbH; 2015-2018 Graylog, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

log4j2-gelf's People

Contributors

aheusingfeld avatar bruno1cruz avatar louxiu avatar rincewind99 avatar vladimirfx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

log4j2-gelf's Issues

Messages not sending

Hi,
I am experiencing some strange behavior when trying to use the UDP functionality. Basically the majority of messages are never sent. Using the code here:

for(int i=0;i<10000;i++){
            System.out.println("Logging a thing");
            logger.info("testing thing "+i);
            System.out.println("Logged a thing");

        }

and config here:

 <?xml version="1.0" encoding="UTF-8"?>
<configuration status="OFF" packages="org.graylog2.log4j2">
    <appenders>
        <GELF name="gelfAppender" server="<server>" port="4444" protocol="UDP"  hostName="<hostname>">
            <PatternLayout pattern="%logger{36} - %msg%n"/>
            <Filters>
                <MarkerFilter  marker="FLOW" onMatch="DENY" onMismatch="NEUTRAL"/>
                <MarkerFilter  marker="EXCEPTION" onMatch="DENY" onMismatch="ACCEPT"/>
            </Filters>

        </GELF>
    </appenders>
    <Loggers>
        <Root level="info">
            <AppenderRef ref="gelfAppender"/>
        </Root>
    </Loggers>
</configuration> 

I would expect all 10,000 messages to send, and a (small) number to be lost in transit. What actually happens is only 2-3k are sent, and a small number are lost in transit. I've verified what is sent/received with a packet capture on both ends.

KeyValuePair with ContextMapLookup does not resolve variable

I am trying to send a variable set at runtime through MDC as a KeyValuePair element by specifying the following element:

<KeyValuePair key="userId" value="$${ctx:userId}"/>

However, the variable cannot be resolved. In the case where I set the variable as part of the PatternLayout it is resolved fine.

What I can find out from the code in GelfAppender.java is that the KeyValuePair elements are added once at creation, but runtime variables are never resolved.

Before I dig into fixing my issue, and creating a pull request I would like to know if this is a known issue, or if there is another way to provide ContextMapLookup variables as KeyValuePair elements.

Note also that variables provided at startup e.g. ${env:APPLICATION_NAME} are resolved just fine

java.lang.NoClassDefFoundError: Could not initialize class io.netty.util.concurrent.DefaultPromise

Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class io.netty.util.concurrent.DefaultPromise
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:35)
    at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:49)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:61)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:52)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:44)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:36)
    at Main.main(Main.java:9)

With:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="debug" strict="true" name="XMLConfigTest">
    <Appenders>
        <Appender type="Console" name="STDOUT"/>
        <GELF name="GrayLog"/>
    </Appenders>
    <Loggers>
        <Root level="trace">
            <AppenderRef ref="STDOUT"/>
        </Root>
    </Loggers>
</Configuration>
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>graylog2-issue</groupId>
    <artifactId>graylog2-issue</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>

        <log4j.version>2.6.2</log4j.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>${log4j.version}</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-slf4j-impl</artifactId>
            <version>${log4j.version}</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.graylog2.log4j2</groupId>
            <artifactId>log4j2-gelf</artifactId>
            <version>1.3.0</version>
        </dependency>
    </dependencies>
</project>
import io.netty.channel.nio.NioEventLoopGroup;

import java.net.MalformedURLException;
import java.net.UnknownHostException;

public class Main {

    public static void main(String... args) throws UnknownHostException, MalformedURLException {
        NioEventLoopGroup group = new NioEventLoopGroup();
    }
}

I tried to analyze the issue, and as I understand it, it is an chicken&egg problem: at init time Netty is trying to log into gelf which uses not yet initialized Netty \o/

Log4j2 Lookups don't work

Hi guys,

First of of all, thanks for the useful plugin. I've tried to use it and seems it works very well.

Unfortunately, "additionalFields" don't work with the Log4j2 Lookups as is written in description
( I tried to use "${ctx:userId}" what I've put in ThreadContext ).

I also think that it would be great if Log4j2 Lookups could be used for "hostName" parameter.
In cloud architecture you might create one build or image and then run it on different hosts, so for sure "hostName" should be somewhat dynamic.

What do you think?

Best Regards,
Maksim

Tomcat not able to shutdown properly

Hi,

Since I've added the GelfAppender to my application, the tomcat won't stop properly.
Log4j2.xml:

<?xml version="1.0" encoding="UTF-8" ?>
<Configuration>
    <!-- Appenders and Loggers Declarations -->
    <Appenders>
        <RollingFile name="LOCALFILE" fileName="/path/to/log/file.log"
                     filePattern="/path/to/log/file-%d{yyy-MM-dd}.log">
            <PatternLayout>
                <Pattern>%d{yyyyMMddHHmmssSSS} [%-5t] - [%-5p] - [%-5c] - %m%n</Pattern>
            </PatternLayout>
            <TimeBasedTriggeringPolicy/>
        </RollingFile>
        <GELF name="GRAYLOG" server="127.0.0.1" port="12201" hostName="Maskime-MacBook-Pro.local">
            <PatternLayout>
                <Pattern>%d{yyyyMMddHHmmssSSS} [%-5t] - [%-5p] - [%-5c] - %m%n</Pattern>
            </PatternLayout>
            <Filters>
                <MarkerFilter marker="FLOW" onMatch="DENY" onMismatch="NEUTRAL"/>
                <MarkerFilter marker="EXCEPTION" onMatch="DENY" onMismatch="ACCEPT"/>
            </Filters>
            <KeyValuePair key="platform" value="technical-platform" />
            <KeyValuePair key="version" value="1.0.0-SNAPSHOT"/>
            <KeyValuePair key="env" value="dev"/>
        </GELF>
    </Appenders>

    <Loggers>
        <Root level="INFO">
            <AppenderRef ref="GRAYLOG"/>
            <AppenderRef ref="LOCALFILE"/>
        </Root>
        <!--Prevent the logs from being polluted-->
        <logger name="org.hibernate" level="INFO"/>
        <logger name="org.springframework" level="INFO"/>
    </Loggers>
</Configuration>

When calling shutdown on the tomcat, the catalina.out ends up with the following errors:

29-Nov-2016 15:16:52.657 INFO [main] org.apache.catalina.core.StandardServer.await A valid shutdown command was received via the shutdown port. Stopping the Server instance.
29-Nov-2016 15:16:52.658 INFO [main] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["http-nio-127.0.0.1-8080"]
29-Nov-2016 15:16:52.713 INFO [main] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["http-nio-8080"]
29-Nov-2016 15:16:53.769 INFO [main] org.apache.catalina.core.StandardService.stopInternal Stopping service Catalina
29-Nov-2016 15:16:53.798 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [application-name] appears to have started a thread named [gelfUdpTransport-1-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
 sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
 sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
 sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
 sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
 io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:759)
 io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:400)
 io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
 io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
 java.lang.Thread.run(Thread.java:745)
29-Nov-2016 15:16:53.799 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [application-name] appears to have started a thread named [GelfSenderThread-25] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
 java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
 org.graylog2.gelfclient.transport.GelfSenderThread$1.run(GelfSenderThread.java:93)
 java.lang.Thread.run(Thread.java:745)
29-Nov-2016 15:16:53.799 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [application-name] appears to have started a thread named [threadDeathWatcher-3-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Thread.sleep(Native Method)
 io.netty.util.ThreadDeathWatcher$Watcher.run(ThreadDeathWatcher.java:150)
 io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
 java.lang.Thread.run(Thread.java:745)
29-Nov-2016 15:16:53.800 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [application-name] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@36896e49]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@7da3fd9a]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
29-Nov-2016 15:16:53.806 INFO [main] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["http-nio-127.0.0.1-8080"]
29-Nov-2016 15:16:53.910 INFO [main] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["http-nio-8080"]
29-Nov-2016 15:16:55.053 INFO [main] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler ["http-nio-127.0.0.1-8080"]
29-Nov-2016 15:16:55.054 INFO [main] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler ["http-nio-8080"]
29-Nov-2016 15:16:55.055 SEVERE [main] org.apache.coyote.AbstractProtocol.destroy Failed to destroy end point associated with ProtocolHandler ["http-nio-8080"]
 java.io.IOException: Unknown error: 316
	at sun.nio.ch.NativeThread.signal(Native Method)
	at sun.nio.ch.ServerSocketChannelImpl.implCloseSelectableChannel(ServerSocketChannelImpl.java:292)
	at java.nio.channels.spi.AbstractSelectableChannel.implCloseChannel(AbstractSelectableChannel.java:234)
	at java.nio.channels.spi.AbstractInterruptibleChannel.close(AbstractInterruptibleChannel.java:115)
	at sun.nio.ch.ServerSocketAdaptor.close(ServerSocketAdaptor.java:137)
	at org.apache.tomcat.util.net.NioEndpoint.unbind(NioEndpoint.java:488)
	at org.apache.tomcat.util.net.AbstractEndpoint.destroy(AbstractEndpoint.java:821)
	at org.apache.coyote.AbstractProtocol.destroy(AbstractProtocol.java:534)
	at org.apache.catalina.connector.Connector.destroyInternal(Connector.java:1023)
	at org.apache.catalina.util.LifecycleBase.destroy(LifecycleBase.java:297)
	at org.apache.catalina.core.StandardService.destroyInternal(StandardService.java:589)
	at org.apache.catalina.util.LifecycleBase.destroy(LifecycleBase.java:297)
	at org.apache.catalina.core.StandardServer.destroyInternal(StandardServer.java:877)
	at org.apache.catalina.util.LifecycleBase.destroy(LifecycleBase.java:297)
	at org.apache.catalina.startup.Catalina.stop(Catalina.java:703)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:664)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:351)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:485)

Exception in thread "gelfUdpTransport-1-1" java.lang.NoClassDefFoundError: io/netty/buffer/PoolArena$1
	at io.netty.buffer.PoolArena.freeChunk(PoolArena.java:279)
	at io.netty.buffer.PoolThreadCache$MemoryRegionCache.freeEntry(PoolThreadCache.java:456)
	at io.netty.buffer.PoolThreadCache$MemoryRegionCache.free(PoolThreadCache.java:426)
	at io.netty.buffer.PoolThreadCache$MemoryRegionCache.free(PoolThreadCache.java:418)
	at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:275)
	at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:266)
	at io.netty.buffer.PoolThreadCache.free0(PoolThreadCache.java:239)
	at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:235)
	at io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.onRemoval(PooledByteBufAllocator.java:359)
	at io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.onRemoval(PooledByteBufAllocator.java:345)
	at io.netty.util.concurrent.FastThreadLocal.remove(FastThreadLocal.java:226)
	at io.netty.util.concurrent.FastThreadLocal.removeAll(FastThreadLocal.java:67)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:146)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: Illegal access: this web application instance has been stopped already. Could not load [io.netty.buffer.PoolArena$1]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1342)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1205)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1166)
	... 14 more
Caused by: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [io.netty.buffer.PoolArena$1]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1352)
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1340)
	... 16 more
Exception in thread "gelfUdpTransport-1-3" Exception in thread "gelfUdpTransport-1-5" Exception in thread "gelfUdpTransport-1-7" Exception in thread "gelfUdpTransport-1-10" Exception in thread "gelfUdpTransport-1-12" Exception in thread "gelfUdpTransport-1-8" Exception in thread "gelfUdpTransport-1-14" Exception in thread "gelfUdpTransport-1-11" Exception in thread "gelfUdpTransport-1-13" Exception in thread "gelfUdpTransport-1-9" Exception in thread "gelfUdpTransport-1-4" Exception in thread "gelfUdpTransport-1-16" Exception in thread "gelfUdpTransport-1-15" java.lang.NoClassDefFoundError: io/netty/util/concurrent/DefaultPromise$1
Exception in thread "gelfUdpTransport-1-2" 	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:435)
	at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:102)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:912)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	at java.lang.Thread.run(Thread.java:745)
Exception in thread "gelfUdpTransport-1-6" java.lang.NoClassDefFoundError: io/netty/util/concurrent/DefaultPromise$1
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:435)
	at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:102)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:912)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	at java.lang.Thread.run(Thread.java:745)
java.lang.NoClassDefFoundError: io/netty/util/concurrent/DefaultPromise$1
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:435)
	at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:102)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:912)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	at java.lang.Thread.run(Thread.java:745)
  [This one goes on and on until I kill the tomcat with a kill -9 PID command]

I'm pretty sure this comes from the appender because if I remove it the problem is solved and the tomcat exits gracefully.
Tomcat version : Apache Tomcat Version 8.0.36
Java version :

$ java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)

GelfAppender Maven dependency:

<dependency>
    <groupId>org.graylog2.log4j2</groupId>
    <artifactId>log4j2-gelf</artifactId>
    <version>1.3.1</version>
</dependency>

I hope I provided enough information for you to look into it.
Thanks for all the help that you can provide concerning this issue

FailoverAppender

Hi Team

FailoverAppender is not working when I configure GelfAppender. Please help.

Thanks
Srumith

Disable automatic MDC field creation

Hi,

I have a problem with this appender. I created the following appender configuration:

<GELF name="gelfAppender" server="XXX.XXX.XXX.XXX" port="XXX" hostName="myHost"> <PatternLayout pattern="%msg%n"/> <!-- Additional fields --> <KeyValuePair key="Sourcename" value="mySourceName"/> <KeyValuePair key="Class" value="Application"/> <KeyValuePair key="Vendor" value="myVendor"/> <KeyValuePair key="APP_Name" value="myAppName"/> <KeyValuePair key="APP_Stage" value="myStage"/> </GELF>

In the application some MDC fields are set. Now I see in Graylog that all MDC fields have automatically been created in the DB. Is there a possibility to disabled that? In your documentation I don't see an attribute for that.

There is a project under https://github.com/mp911de/logstash-gelf that offers an attribute "IncludeFullMdc" for that.

Question about additional fields in code

Hello,

We're using this library with log4j2 to send gelf messages to Graylog and it works fine. I have a question about additional fields.

I'm adding a couple of hard-coded additional fields via the XML configuration file, like environment = alpha. Is there a way to add these fields in code? We have a number of dynamic fields that we would like to set during runtime. In one project I solved this by using the objects in the org.graylog2 library such as GelfMessage:

GelfMessage gelfMessage = new GelfMessage();
Map<String, Object> additionalParameters = new Hashtable<String, Object>();
additionalParameters.put("referrer", referrer);
additionalParameters.put("duration_ms", duration);
...
...
gelfMessage.setAdditonalFields(additionalParameters);

However, this meant bypassing log4j which defeated the purpose of using a logging framework.

Is there a way to inject these additional fields into the gelf message dynamically via log4j2 library?

Thanks for your help,
Andras

Graylog http support

Is it possible to use the graylog2 http api with this log4j2 appender library?

Appender catches InterruptedException, fails to log, and clears interrupted flag.

The GelfAppender suffers from this kind of bug: https://issues.apache.org/jira/browse/LOG4J2-1049

See the fix here:

https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/appender/AsyncAppender.java#L157

2015-09-28 19:01:06,651 ERROR An exception occurred processing Appender GelfAppender org.apache.logging.log4j.core.appender.AppenderLoggingException: failed to write log event to GELF server: null
    at org.graylog2.log4j2.GelfAppender.append(GelfAppender.java:148)
    at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:99)
    at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:430)
    at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:409)
    at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:367)
    at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:112)
    at org.apache.log4j.Category.forcedLog(Category.java:346)
    at org.apache.log4j.Category.log(Category.java:433)
    at com.mchange.v2.log.log4j.Log4jMLog$Log4jMLogger.log(Log4jMLog.java:161)
    at com.mchange.v2.log.log4j.Log4jMLog$Log4jMLogger.log(Log4jMLog.java:173)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$ConnectionEventListenerImpl.doCheckinResource(C3P0PooledConnectionPool.java:640)
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$ConnectionEventListenerImpl.connectionClosed(C3P0PooledConnectionPool.java:630)
    at com.mchange.v2.c3p0.util.ConnectionEventSupport.fireConnectionClosed(ConnectionEventSupport.java:55)
    at com.mchange.v2.c3p0.impl.NewPooledConnection.fireConnectionClosed(NewPooledConnection.java:510)
    at com.mchange.v2.c3p0.impl.NewPooledConnection.markClosedProxyConnection(NewPooledConnection.java:381)
    at com.mchange.v2.c3p0.impl.NewProxyConnection.close(NewProxyConnection.java:1246)
    at org.hibernate.connection.DatasourceConnectionProvider.closeConnection(DatasourceConnectionProvider.java:97)
    at org.hibernate.jdbc.ConnectionManager.closeConnection(ConnectionManager.java:474)
    at org.hibernate.jdbc.ConnectionManager.aggressiveRelease(ConnectionManager.java:429)
    at org.hibernate.jdbc.ConnectionManager.afterTransaction(ConnectionManager.java:316)
    at org.hibernate.jdbc.JDBCContext.afterTransactionCompletion(JDBCContext.java:269)
    at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:146)
    at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:76)
    at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:512)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:392)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
    at com.sun.proxy.$Proxy173.call(Unknown Source)
    at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:273)
    at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)
    at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
    at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:268)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
    at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.InterruptedException
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219)
    at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)
    at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338)
    at org.graylog2.gelfclient.transport.AbstractGelfTransport.send(AbstractGelfTransport.java:90)
    at org.graylog2.log4j2.GelfAppender.append(GelfAppender.java:146)
    ... 44 more

using this in tomcat app causes memory leak

using latest version 1.0.3 in a tomcat 7 webapp and working fine so far. When stopping the tomcat service catalina.out reports on threads it couldn't stop:

Apr 21, 2015 1:52:32 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
Schwerwiegend: The web application [/myapp] appears to have started a thread named [nioEventLoopGroup-2-1] but has failed to stop it. This is very likely to create a
memory leak.

Apr 21, 2015 1:52:32 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
Schwerwiegend: The web application [/myapp] appears to have started a thread named [GelfSenderThread-37] but has failed to stop it. This is very likely to
create a memory leak.

Apr 21, 2015 1:52:32 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
Schwerwiegend: The web application [/myapp] created a ThreadLocal with key of type [java.lang.ThreadLocal](value [java.lang.ThreadLocal@5f9e0bcf])
and a value of type [io.netty.util.internal.InternalThreadLocalMap](value [io.netty.util.internal.InternalThreadLocalMap@1197414]) but failed to remove it when the
web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

Apr 21, 2015 1:52:32 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
Schwerwiegend: The web application [/myapp] created a ThreadLocal with key of type [java.lang.ThreadLocal](value [java.lang.ThreadLocal@5f9e0bcf]) and a value of
type [io.netty.util.internal.InternalThreadLocalMap](value [io.netty.util.internal.InternalThreadLocalMap@2e13cd9c]) but failed to remove it when the web
application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

this is proven to be only since the GelfAppender was added. It is no problem when that appender is disabled anymore.
any idea how to make the GelfAppender shutdown gracefully?

Source field

I noticed the source field only shows the hostname, is that the correct behavior? Or a bug?
I would expect the fqdn as source.

performance concern with current includeSource implementation

In current implementation:

public void append(LogEvent event) {
...
    final StackTraceElement source = event.getSource();
    if (includeSource && source != null) {    
        builder.additionalField("sourceFileName", source.getFileName());

The problem is that the event.getSource() is part of the location information that could have a performance impact according to the log4j2 doc:

If one of the layouts is configured with a location-related attribute like HTML locationInfo, or one of the patterns %C or %class, %F or %file, %l or %location, %L or %line, %M or %method, Log4j will take a snapshot of the stack, and walk the stack trace to find the location information.

This is an expensive operation: 1.3 - 5 times slower for synchronous loggers...

Should the includeSource be checked first before doing the event.getSource? like:

if (includeSource) {
    final StackTraceElement source = event.getSource();
    if (source != null) {
            builder.additionalField("sourceFileName", source.getFileName());
           ...

Thanks,
Bill

DEF: protocol TCP causing locking of appender thread

In use case, when current TCP connection is dropped (f.e. firewall)

org.graylog2.gelfclient.encoder.GelfMessageJsonEncoder - JSON encoding error
java.io.IOException: An existing connection was forcibly closed by the remote host

then the gelfTcpTranspor tread is blocking each appender thread with queue.put(message)

   public void send(final GelfMessage message) throws InterruptedException {
        LOG.debug("Sending message: {}", message.toString());
        queue.put(message);
    }

EXPECTED
the appender thread will not be blocked by transport thread .

Appenders contains an invalid element or attribute "GELF"

My project is with Ant. log4j2-gelf library from central.maven.org
Log4j v. 2.4.1

log4j2.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="ALL" packages="org.graylog2.log4j2">
    <Appenders>

        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{dd.MM.yyyy HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
        </Console>

        <GELF name="GELFAppender" server="192.168.0.17" port="12201" hostName="ddocserver">
        </GELF>

    </Appenders>
    <Loggers>

        <Logger name="testlogger" additivity="false" level="all">
            <AppenderRef ref="Console"/>
        </Logger>

        <Root level="warn">
            <AppenderRef ref="Console"/>
            <AppenderRef ref="GELFAppender"/>
        </Root>
    </Loggers>
</Configuration>

Debug output:

...
DEBUG Took 0,000498 seconds to load 0 plugins from package org.graylog2.log4j2
...
ERROR Appenders contains an invalid element or attribute "GELF"
...
ERROR Unable to locate appender "GELFAppender" for logger config "root"

elasticsearch5 example

Hi
Any chance you provide an example with elasticsearch5.1 log4j2.properties ?
thanks

Client doesn't pick up IP address change in GELF server domain name

The IP address for the GELF server's domain name was changed but the client didn't pick up the new IP address when using the UDP transport. Instead, it keeps trying to send logs to the old unreachable IP address. I believe the problem is coming from the following line of code: https://github.com/Graylog2/gelfclient/blob/cdb0241e085a9a15fdfb6dce887616a262a95be4/src/main/java/org/graylog2/gelfclient/transport/GelfUdpTransport.java#L62.

InetSocketAddress only performs resolution at construction time, and since createBootstrap(...) is just called during initialisation, IP address changes aren't picked up. The current solution is to restart the application but of course this isn't ideal. GelfMessageUdpEncoder's constructor should be changed to take in the hostname and port instead of InetSocketAddress so that they can be used to create an InetSocketAddress object in the encode(...) method.

Documentation Issue

Hi,

For the log4j2.xml exemple file, you provide the following:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="OFF" packages="org.graylog2.log4j2">
    <Appenders>
        <GELF name="gelfAppender" server="graylog2.example.com" port="12201" hostName="appserver01.example.com">
            <PatternLayout pattern="%logger{36} - %msg%n"/>
            <Filters>
                <Filter type="MarkerFilter" marker="FLOW" onMatch="DENY" onMismatch="NEUTRAL"/>
                <Filter type="MarkerFilter" marker="EXCEPTION" onMatch="DENY" onMismatch="ACCEPT"/>
            </Filters>
            <!-- Additional fields -->
            <KeyValuePair key="foo" value="bar"/>
            <KeyValuePair key="jvm" value="${java:vm}"/>                
        </GELF>
    </Appenders>
    <Loggers>
        <Root level="info">
            <AppenderRef ref="gelfAppender"/>
        </Root>
    </Loggers>
</Configuration>

But this will result with an error message with log4j2:2.6.2 on the Filters elements.
The way I got it to work is by using Filter type="" as specific class:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="OFF" packages="org.graylog2.log4j2">
    <Appenders>
        <GELF name="gelfAppender" server="graylog2.example.com" port="12201" hostName="appserver01.example.com">
            <PatternLayout pattern="%logger{36} - %msg%n"/>
            <Filters>
                <MarkerFilter marker="FLOW" onMatch="DENY" onMismatch="NEUTRAL"/>
                <MarkerFilter marker="EXCEPTION" onMatch="DENY" onMismatch="ACCEPT"/>
            </Filters>
            <!-- Additional fields -->
            <KeyValuePair key="foo" value="bar"/>
            <KeyValuePair key="jvm" value="${java:vm}"/>                
        </GELF>
    </Appenders>
    <Loggers>
        <Root level="info">
            <AppenderRef ref="gelfAppender"/>
        </Root>
    </Loggers>
</Configuration>

Next release on maven repository

When will occur the next release to maven repositories? We are expecting to use 'additionalFields' feature to log static fields.

Any example of log4j.properties format?

This is what I tried:
log4j.appender.graylog2=org.graylog2.log4j2.GelfAppender
log4j.appender.graylog2.server=mygreylogserver
log4j.appender.graylog2.hostName=myhost

But this is what I get:
java.lang.ClassNotFoundException: org.graylog2.log4j2.GelfAppender

contains an invalid element or attribute "layout"

Hi,

When startup the application, I got the error.

2016-08-04 16:02:43,257 [main] [ERROR] GELF contains an invalid element or attribute "layout"

As well, the layout is not effective in my log.

log4j2.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" packages="org.graylog2.log4j2">
    <Appenders>
        <GELF name="GelfAppender"
              protocol="TCP"
              server="xxx"
              port="xxx"
              includeSource="true"
              includeThreadContext="true"
              includeStackTrace="false"
              layout="%m%n%rEx"
              additionalFields="service=xxx,env=development" />

....

F.Y.I.
Dependencies:

slf4j: 1.7.16
log4j2: 2.4.1
log4j2-gelf: both error in 1.1.0 and 1.2.0

Every minute a log entry of "INFO GelfTcpTransport: Channel disconnected!" is written to logs

We are able to successfully configure applications with a Gel appender and verify the proper transmission of those messages to Graylog. However, every minute in our logs the message:
"INFO GelfTcpTransport: Channel disconnected!" is displayed. This is a little unreasonable across multiple apps to have this message spamming our logs every minute.

Is there any configuration we can tweak to lower the level of this message?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.