serilog-archive / serilog-sinks-loggly Goto Github PK

A Serilog event sink that writes to Loggly

License: Apache License 2.0

PowerShell 1.07% C# 98.93%

serilog-sinks-loggly's Introduction

Serilog.Sinks.Loggly

Loggly is a cloud based log management service. Create a new input and specify that you want to use a http input with JSON enabled. Use the loggly-csharp-configuration XML configuration syntax to configure the sink.

Package - Serilog.Sinks.Loggly | Platforms - .NET 4.5

var log = new LoggerConfiguration()
    .WriteTo.Loggly()
    .CreateLogger();

Properties will be sent along to Loggly. The level is sent as a category.

To use a durable logger (that will save messages locally if the connection to the server is unavailable, and resend once the connection has recovered), set the bufferBaseFilename argument in the Loggly() extension method.

var log = new LoggerConfiguration()
    .WriteTo.Loggly(bufferBaseFilename:@"C:\test\buffer")
    .CreateLogger();

This will write unsent messages to a buffer-{Date}.json file in the specified folder (C:\test\ in the example).

The method also takes a retainedFileCountLimit parameter that will allow you to control how much info to store / ship when back online. By default, the value is null with the intent is to send all persisted data, no matter how old. If you specify a value, only the data in the last N buffer files will be shipped back, preventing stale data to be indexed (if that info is no longer usefull).

The sink can also be configured from appsettings.json for .NET Standard / .NET Core applications that do not support XML configuration:

{
  "Serilog": {
    "WriteTo": [
      {
        "Name": "Loggly",
        "Args": {
          "customerToken": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
          "tags": "foo,bar"
        }
      }
    ],
    "Properties": { "Application": "SampleApp" }
  }
}

The customerToken argument is required, if you use this form of configuration. The tags argument is comma-delimited. The Application property will also be sent to Loggly and should be set appropriately.

serilog-sinks-loggly's People

Contributors

Stargazers

Watchers

serilog-sinks-loggly's Issues

LogEvents with Exceptions are not getting transmitted to Loggly, silently failing

Using this sink in an ASP.NET Core project and noticed that any of the log events that have an exception attached are not making it to Loggly. I didn't track this all the way through the loggly-csharp code but my guess is that they're not able to serialize the Exception using Newtonsoft.

Commit of a hackish work-around is here.

I think the better way to go here is to hook into the Serilog Json Formatting / Converting that handles serializing properties, but I'm not familiar enough with Serilog from the provider/sink side to know how to do that. Can someone point me in the right direction and I'll try and get that change made and a new pull request?

Sink not working

Just tried this sink with a working Loggly account but seems not working at all.
This is my configuration code:

        private void SetupLogglyConfiguration()
        {
            const string appName = "APP-NAME";
            const string customerToken = "MY-TOKEN";

            //Configure Loggly
            var config = LogglyConfig.Instance;
            config.CustomerToken = customerToken;
            config.ApplicationName = appName;
            config.Transport = new TransportConfiguration()
            {
                EndpointHostname = "XXXXX.loggly.com",
                EndpointPort = 443,
                LogTransport = LogTransport.Https
            };
            config.ThrowExceptions = true;
            config.

            //Tags
            config.TagConfig.Tags.AddRange(new ITag[]{
                new ApplicationNameTag {Formatter = "application-{0}"},
                new HostnameTag { Formatter = "host-{0}" }
            });
        }

After launch above method I create my log like this:

new LoggerConfiguration().Enrich.With(new UserLogEventEnricher(userName, $"v. {appVersionString} b{appBuildNumber}")).WriteTo.Loggly(bufferBaseFilename: @"log_buffer").CreateLogger();

But when I log something nothing appears on website side. Worst problem is that there are no error or exception at all. I tried to place a wrong token or a wrong connection port (433888888 is a little bit large number for a port) but I get no error, it does nothing at all.
What am I missing?

Edit:
Checked on app folder (in simulator) I can see the log_buffer file is created and filled with logs so the problem seems only the connection with Loggly server.

Durable sink retained file limit

The RollingFileSink in the DurableLogglySink supports the ability to set the retained file limit, but the current implementation passes in null, so it would be unlimited. Would you accept a PR that would allow this to be overridden?

Loggly sink doesn't adhere to Reliability policy of Serilog

Hello,
I recently configured my application to use loggly sink and I had error in configuration.
This error caused ArgumentNullException which prevented my application from starting.

I believe this goes against Serilog Reliability policy - https://github.com/serilog/serilog/wiki/Reliability which basically says if something fails to be configured, it should not prevent application from functioning.

I believe this sink should be updated to adhere to to Serilog reliablity policy.

Here is my exception stack trace, just for reference.

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ArgumentNullException: CustomerToken is required (Parameter 'CustomerToken')
  at at Serilog.Sinks.Loggly.LogglyConfigAdapter.ConfigureLogglyClient(LogglyConfiguration logglyConfiguration)
  at at Serilog.Sinks.Loggly.LogglySink..ctor(IFormatProvider formatProvider, Int32 batchSizeLimit, TimeSpan period, LogglyConfiguration logglyConfig, LogIncludes includes)
  at at Serilog.LoggerConfigurationLogglyExtensions.Loggly(LoggerSinkConfiguration loggerConfiguration, LogEventLevel restrictedToMinimumLevel, Int32 batchPostingLimit, Nullable`1 period, IFormatProvider formatProvider, String bufferBaseFilename, Nullable`1 bufferFileSizeLimitBytes, Nullable`1 eventBodyLimitBytes, LoggingLevelSwitch controlLevelSwitch, Nullable`1 retainedInvalidPayloadsLimitBytes, Nullable`1 retainedFileCountLimit, String customerToken, String tags, String endpointHostName, LogglyConfiguration logglyConfig, LogIncludes includes)
  --- End of inner exception stack trace ---
  at at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
  at at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
  at at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
  at at Serilog.Settings.Configuration.ConfigurationReader.CallConfigurationMethods(ILookup`2 methods, IList`1 configurationMethods, Object receiver)
  at at Serilog.Settings.Configuration.ConfigurationReader.Serilog.Settings.Configuration.IConfigurationReader.ApplySinks(LoggerSinkConfiguration loggerSinkConfiguration)
  at at Serilog.Settings.Configuration.ObjectArgumentValue.<>c__DisplayClass3_0.<ConvertTo>b__0(LoggerSinkConfiguration loggerSinkConfig)
  at at Serilog.Configuration.LoggerSinkConfiguration.Wrap(LoggerSinkConfiguration loggerSinkConfiguration, Func`2 wrapSink, Action`1 configureWrappedSink, LogEventLevel restrictedToMinimumLevel, LoggingLevelSwitch levelSwitch)
  at at Serilog.LoggerConfigurationAsyncExtensions.Async(LoggerSinkConfiguration loggerSinkConfiguration, Action`1 configure, IAsyncLogEventSinkMonitor monitor, Int32 bufferSize, Boolean blockWhenFull)
  at at Serilog.LoggerConfigurationAsyncExtensions.Async(LoggerSinkConfiguration loggerSinkConfiguration, Action`1 configure, Int32 bufferSize, Boolean blockWhenFull)
  --- End of inner exception stack trace ---
  at at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
  at at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
  at at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
  at at Serilog.Settings.Configuration.ConfigurationReader.CallConfigurationMethods(ILookup`2 methods, IList`1 configurationMethods, Object receiver)
  at at Serilog.Settings.Configuration.ConfigurationReader.ApplySinks(LoggerConfiguration loggerConfiguration)
  at at Serilog.Settings.Configuration.ConfigurationReader.Configure(LoggerConfiguration loggerConfiguration)
  at at Serilog.Configuration.LoggerSettingsConfiguration.Settings(ILoggerSettings settings)
  at at Serilog.ConfigurationLoggerConfigurationExtensions.Configuration(LoggerSettingsConfiguration settingConfiguration, IConfiguration configuration, String sectionName, DependencyContext dependencyContext)
  at at Serilog.ConfigurationLoggerConfigurationExtensions.Configuration(LoggerSettingsConfiguration settingConfiguration, IConfiguration configuration, DependencyContext dependencyContext)
  at PseApi.Startup..ctor(IConfiguration configuration) in C:\work\pse-api\PseApi\Startup.cs:24
  at at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider)
  at at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.CreateInstance(IServiceProvider provider, Type instanceType, Object[] parameters)
  at at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.GetServiceOrCreateInstance(IServiceProvider provider, Type type)
  at at Microsoft.AspNetCore.Hosting.StartupLoader.LoadMethods(IServiceProvider hostingServiceProvider, Type startupType, String environmentName)
  at at Microsoft.AspNetCore.Hosting.WebHostBuilderExtensions.<>c__DisplayClass3_0.<UseStartup>b__1(IServiceProvider sp)
  at at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)
  at at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
  at at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
  at at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
  at at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
  at at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
  at at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
  at at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
  at at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType)
  at at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
  at at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
  at at Microsoft.AspNetCore.Hosting.WebHost.EnsureStartup()
  at at Microsoft.AspNetCore.Hosting.WebHost.EnsureApplicationServices()
  at at Microsoft.AspNetCore.Hosting.WebHost.Initialize()
  at at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
  at PseApi.Program.Main(String[] args) in C:\work\pse-api\PseApi\Program.cs:18

New organization required for this repository

Hi @serilog/reviewers-loggly 👋

Via serilog/serilog#1627 - we're unbundling the serilog organization to help distribute the effort involved in managing the various Serilog sub-projects. The serilog organization will now only manage the fundamental sinks and other packages that the rest of the ecosystem builds upon.

If this package is actively maintained, it can be moved to a new organization managed by the maintainers. Otherwise, it can move to the serilog-archive parking lot, from where we hope a new community-run fork might spring from in the future 🌷 .

Let me know if you're a maintainer and keen to continue this project under a new org; otherwise, I'll shuffle things around and move this one to the archive.

Thanks!

Adding a dynamic tag to what I'm logging

It looks like when you configure Loggly, you can add tags that you want added to your messages that are getting logged:

//Configure Loggly
var config = LogglyConfig.Instance;
config.CustomerToken = customerToken;
config.ApplicationName = appName;
config.Transport = new TransportConfiguration()
{
    EndpointHostname = "logs-01.loggly.com",
    EndpointPort = 443,
    LogTransport = LogTransport.Https
};
config.ThrowExceptions = true;

//Define Tags sent to Loggly
config.TagConfig.Tags.AddRange(new ITag[]{
    new ApplicationNameTag {Formatter = "application-{0}"},
    new HostnameTag { Formatter = "host-{0}" },
    new SimpleTag { Value = "Test Tag" }   // my custom test tag
});

So now whenever I call Log.Information("..."), those tags are added to what I'm logging. But it's at a global level, it doesn't seem to be able to be changed at all, "Test Tag" is just applied to everything.

For my application, I'm receiving messages from a service bus, which can send through several different types of events. Based on the type of event, a different function is called. I'm logging information in each of these functions.

But I'd like to tag each of these with an event code, so that I can easily go into Loggly and see what was logged based on the type of events that were passed through the service bus to my application.

Is there any way to add a tag with a dynamic value at the point when I'm actually logging something?

Tags not being sent

Ignore me, I'm an idiot

Update loggly-csharp client

In the nugget package loggly-csharp client 4.6.1.33 support .net standard 1.5
And maybe it makes sense to migrate project to .csproj

Durrable Logger sends same message(s) repeatedly

One behaviour that has popped up infrequently is related to the durable Logger implementation sending the same message frequently / repeatedly to Loggly. This eventually creates a massive accumulation of messages on Loggly and many times quickly maxes out any cap in place.

We've detected the problem through the following symptoms:

We get a message from Loggly that our daily message cap has been reached or passed
looking in to the logs we see there is a large part of the graph occupied by the same message (type) - typically exceptions but not limited to.
we dig further in and find that it's the same machine sending the messages
we dig further in and find the same event timestamp across messages, generally an older timestamp

We suspect that it is the durable logger implementation we use that some how cannot update the buffer (read messages) correctly and in failing to move forward, stays and repeats the same message(s).

Looking at the buffer.bookmark file, there is a record for a byte pointer to the first buffer.json file (oldest) and many invalid-Error files in the folder.

I'll be looking in to this to fix it since it is causing problems for us, but if anyone has seen this before, any input is appreciated.

Configuration from appsettings.json

I haven't been able to find a good way to do configuration-driven setup of the Loggly sink in a .NET Core app. It doesn't look like I can specify anything like

{
  "Serilog": {
    "WriteTo": [
      {
        "Name": "Loggly",
        "Args": {
          "logglyConfig": {
            "CustomerToken": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
            "ApplicationName": "SerilogTest"
          }
        }
      }
    ]
  }
}

because that fails with System.InvalidOperationException: Argument value should be of type Action<>.. And in a .NET Core app, I don't have an app.config file, so I can't use loggly-csharp-configuration's XML-based configuration (as far as I know).

Am I missing something? Or is the current state of affairs such that I need to do programmatic configuration of Loggly in a .NET Core application?

The Loggly sink automatically change the LogLevel when the LogLevelSwitching is active.

The Loggly sink automatically change the LogLevelSwich to the debug. It should be updated from Loggly and if the feature is not possible it shouldn't change the value at all.

Here is the related code from HttpLogShipper

        async Task OnTick()
        {
            LogEventLevel? minimumAcceptedLevel = LogEventLevel.Debug;

            try
            {
              ...........
            }
            catch (Exception ex)
            {
                SelfLog.WriteLine("Exception while emitting periodic batch from {0}: {1}", this, ex);
                _connectionSchedule.MarkFailure();
            }
            finally
            {
                lock (_stateLock)
                {
                    _controlledSwitch.Update(minimumAcceptedLevel);

                    if (!_unloading)
                        SetTimer();
                }
            }
        }

LogContext.PushProperty not included in the log output

The setting of the LogContext is not included in the message.
When I do the following

using (LogContext.PushProperty("app", "my-app"))
{
     log.Debug("Debug Message at {time}", DateTime.Now);
}

I see next (raw) message arrive at Loggly:
{"Message":"Debug Message at 02/18/2019 11:41:02","time":"02/18/2019 11:41:02","Level":"Debug","timestamp":"2019-02-18T11:41:02.3620794+00:00"}
I'm missing the "app" property from the message, expected:
{"Message":"Debug Message at 02/18/2019 11:41:02","time":"02/18/2019 11:41:02","app":"my-app","Level":"Debug","timestamp":"2019-02-18T11:41:02.3620794+00:00"}

Will it work on dotnet core 2.1 environment for sending logs to Loggly?

Hi,

Just want to confirm if this library will work on sending logs to Loggly when the environment is dotnet core 2.1 SDK?

Best,
S

serilog-archive / serilog-sinks-loggly Goto Github PK

serilog-sinks-loggly's Introduction

Serilog.Sinks.Loggly

serilog-sinks-loggly's People

Contributors

Stargazers

Watchers

Forkers

serilog-sinks-loggly's Issues

Recommend Projects

Recommend Topics

Recommend Org