alanbarber / log4net.appender.splunk Goto Github PK

When I import the Nuget package for this library through Visual Studio 2019, I can only see versions 1.0.0 and 2.0.0, but not the latest 2.1.0.

https://www.nuget.org/packages/log4net.Appender.Splunk

We have .NET Console app that we're developling that when every it hits the response = await postEvents(token, events); it exits out of the function with out ever getting to responseCode = response.StatusCode;.
This all occurs in the HttpEvenCollectorSender.cs file, I'll post the whole function in question below:

private async Task<HttpStatusCode> PostEvents(
            List<HttpEventCollectorEventInfo> events,
            String serializedEvents)
        {
            // encode data
            HttpResponseMessage response = null;
            string serverReply = null;
            HttpStatusCode responseCode = HttpStatusCode.OK;
            try
            {
                // post data
                HttpEventCollectorHandler next = (t, e) =>
                {
                    HttpContent content = new StringContent(serializedEvents, Encoding.UTF8, HttpContentTypeMedia);
                    return httpClient.PostAsync(httpEventCollectorEndpointUri, content);
                };
                HttpEventCollectorHandler postEvents = (t, e) =>
                {
                    return middleware == null ?
                        next(t, e) : middleware(t, e, next);
                };
                response = await postEvents(token, events);
                responseCode = response.StatusCode;
                if (responseCode != HttpStatusCode.OK && response.Content != null)
                {
                    // record server reply
                    serverReply = await response.Content.ReadAsStringAsync();
                    OnError(new HttpEventCollectorException(
                        code: responseCode,
                        webException: null,
                        reply: serverReply,
                        response: response,
                        events: events
                    ));
                }
            }
            catch (HttpEventCollectorException e)
            {
                e.Events = events;
                OnError(e);
            }
            catch (Exception e)
            {                
                OnError(new HttpEventCollectorException(
                    code: responseCode,
                    webException: e,
                    reply: serverReply,
                    response: response,
                    events: events
                ));
            }
            return responseCode;
        }

I assume it's having a hard time running the function HttpEventCollectorHandler postEvents = (t, e) => { return middleware == null ? next(t, e) : middleware(t, e, next); };

I will say this function pattern I'm not familiar with, and I'm not an expert in asynchronous programming yet either.

the code was working like charm for a lot of days and suddenly when I write a small unit test to post a fake error log using Splunk appender it started to give me this Thread Being Aborted error!
I just can't figure it out.
This abort is happening on httpclient.postasync() call in the PostEvents() method in EventCollectorSender class. I can't exactly figure out which thread is being aborted!

Currently, logs sent to HttpEventCollectorSender are flushed synchronously using FlushSync(). See

I can see this is advantageous over fire and forget in that exceptions are not lost. However, if the http requests to the Splunk system hang so does the application. Moreover, I use a mock api (WireMock) and delayed the response by a significant amount of time. The entire app hangs on any logging calls. My suggestion is to use FlushAsync().

I'd be more than happy to make a pull request for this, but I currently don't have permissions to make new branches.

Thanks!
Cory

I use nb-NO culture (norwegian), which formats doubles using comma instead of dot. Splunk requires a dot as the decimal separator in the timestamp field.

In HttpEventCollectorEventInfo, the timestamp is formatted using a simple ToString("#.000"). The dot here does not actually represent a dot, but rather the decimal separator from the current culture.

To avoid this, use CultureInfo.InvariantCulture as a format provider. This uses a dot as the decimal separator.

Hi Alan
Thank you for merging my change. Could you please release it as version 2.2 I guess.
I guess it would solve the other issue that version 2.1 is not actually available in Nuget.
Thanks
Jan

#23

We are facing an issue logging into splunk using SplunkHttpEventCollector in Asp.Net Web application. We are able to log using C# console application but not using web application. Do you have any sample DotNet web application ?

We're trying to incorporate this into our product, and currently we're unable to ingest logs without setting the index for the metadata. I did not see any configurable option, nor was it obvious from the source how this could be done externally.

We are trying to use this nuget package to sent logevent to Splunk HEC.
This works fine locally on developer machine, when we deploy the asp.net WebAPI dotnet version 4.8 on azure app service
its not working, we don't see the logs in Splunk HEC.

I downloaded the project and tried to figure out what might cause the issue, I was able to identify that the issue is with httpClient
TLS issue version not setting to TLS 1.2.

Error message: The underlying connection was closed: An unexpected error occurred on a send.

I modified the httpClient to always user TLS12 and took the modified DLL and referred in WebAPI project and deployed to app service and it worked!
var handler = new HttpClientHandler();
handler.SslProtocols = System.Security.Authentication.SslProtocols.Tls12;
httpClient = new HttpClient(handler,true);

It will be very help if a configuration parameter is provided to set the TLS version.

I would like to understand use of ignoreCertificateErrors, the certificate issue with Splunk server or the application
from where the Splunk HEC is called?

Thank you

Whatever I do to the layout seems to be completely ignored when it gets into Splunk.

I need to add method, type, line, custom properties on LogicalThreadContext to the JSON that gets sent to Splunk.

Is there a way to tell this appender to pull those in?

Hello
Thank you for your good work with this Appender

Just wanted to report an issue where the path part of parameter "ServerUrl" is ignored.
In our Splunk setup it very important that the path is /services/collector/event
However, in your code it seems the path is always overridden with a constant definde in "HttpEventCollectorPath"

So we changed: HttpEventCollectorSender.cs line 160
this.httpEventCollectorEndpointUri = new Uri(uri, HttpEventCollectorPath);
We changed it to:
this.httpEventCollectorEndpointUri = uri.ToString().Contains(HttpEventCollectorPath) ? uri : new Uri(uri, HttpEventCollectorPath);

This is working for us now, do you want to implement that in your code?

Jan

Using identical appender suggested in your test.

<appender name="SplunkHTTP" type="log4net.Appender.Splunk.SplunkHttpEventCollector, log4net.Appender.Splunk">
     <ServerUrl>http://myiporhost:8088/services/collector</ServerUrl>
     <Token>mytoken</Token>
     <RetriesOnError>0</RetriesOnError>
     <threshold value="DEBUG" />
     <layout type="log4net.Layout.PatternLayout">
       <conversionPattern value="%message" />
     </layout>
   </appender>

My expectation is that the console output would remain unaffected whether or not this appender is engaged, and that application performance would be unaffected. Any help would be appreciated!