elmahio / elmah.io.elasticsearch Goto Github PK

It would be great if we could define an Elasticsearch index pattern like 'elmah-${yyyy.MM.dd}' and use the ErrorDocument Time property as the date to format. A nice example of this can be found in https://github.com/ReactiveMarkets/NLog.Targets.ElasticSearch.

Is there any interest in something like this?

These have breaking changes.

https://www.elastic.co/blog/ga-release-of-nest-2-0-our-dot-net-client-for-elasticsearch

https://github.com/elmahio/Elmah.Io.ElasticSearch/blob/master/Elmah.Io.ElasticSearch/Elmah.Io.ElasticSearch.nuspec

Hello,

I just wanted to know why the environment name is not being used to filter the errors to show on the elmah page? This is really annoying if you use the same index in the same elasticsearch cluster for different environments, but with the same application name.

Upgrading to nest 1.x is important, it has breaking changes that are needed in Elastic Search 1.x.

• .NET Framework -> 4.0 to 4.6.1
• ElasticSearch.Net -> 1.4.2 to 1.7.1
• Newtonsoft.Json -> 6.0.8 to 7.0.1 (updating to 8.x caused errors)
• NEST -> 1.4.1 to 1.7.1

Update the nuget configuration to specify new minimum requirement for these.

Currently the fields "message", "type", and "source" are string fields which the ES indexer will split. This is great when searching for individual words (i.e. how many errors happened with the phrase "was not found"), but this will not allow you to do an aggregation on all alerts with the message ""The controller for path '/foo' was not found or does not implement IController."

In order to do aggregations on the fields they must be a multi-fields in ES.

update to the latest versions of ES nuget packages.

Hi there,

Just wondering when you think you might upgrade to the NEST 2.0 binaries?

Thanks!
Annie

1. Ensure support for .NET 4.6.1
2. Support C# 6 syntax
3. Support CI build on branches that publish -prerelease candidates to Nuget

move things around a little in the source files and in the solution explorer.

• all source code moved into src folder
• testing project moved into testing folder
• example projects moved into examples folder
• can we remove the nuget folder and add an appveyor explicit command for it?

Create release notes, update readme

Hi,

Is there any plans to support Elasticsearch 5.x?

Thanks!

Optionally pass in the configuration information into the singleton

users may want to read this field as a whole, or split it up. Give them the ability to do both.

The HostName field should be marked as not_analyzed in ES so that when searching and aggregating it doesn't split on spaces, dashes, underscores, etc.

@ThomasArdal I'll take what you did in Elmah.Io and create a pull request shortly.

1. create release notes that say that 2.0 requires ES 2.0 or higher
2. update CI info on landing page
3. only publish to nuget.org on master and on release/*

When someone installs the nuget package, add the follwoing connection string:

<add name="ElmahElasticSearch" connectionString="Nodes=http://localhost:9200;DefaultIndex=elmah;" />

Also update the configuration section with the correct connection string name including:

1. application name
2. environment name
3. customer name

  <elmah>
    <errorLog
        type="Elmah.Io.ElasticSearch.ElasticSearchErrorLog, Elmah.Io.ElasticSearch, Version=1.0.0.0, Culture=neutral"
        connectionStringName="ElmahElasticSearch"
        applicationName="ElmahElasticSearchSampleWebsite"
        environmentName="development"
        customerName="sample customer"
        />
  </elmah>

The latest NuGet version (3.0.0.69) includes a folder for .Net 4.6 but the assembly is targeting 4.7.2.
I have projects that target the 4.6.1 and what happens is that the package is installed but then it won't compile.

ES 2.0 has a number of breaking changes. Ensure that this product still works

Currently, when the Elmah index is setup in ElasticSearch the following defaults are overridden:

Replicas = 0
Shards = 1
merge.policy.merge_factor = 10 --note: this is the Elastic Default
search.slowlog.threshold.fetch.warn = 1s

In a production environment where there is more than 1 node setting replicas = 0 means that a copy of the data would not exist on the other nodes in the cluster. This is not ideal.

ElasticSearch support recommends leaving all of these at the default values and only tweaking them if necessary in production.

http://nest.azurewebsites.net/elasticsearch-net/cluster-failover.html

Hi,
I've ran into a problem with basic setup,

<connectionStrings>
    <add name="ElmahElasticSearch" connectionString="http://foo.bar:9200" />
  </connectionStrings>

<elmah>
    <!--
        See http://code.google.com/p/elmah/wiki/SecuringErrorLogPages for 
        more information on remote access and securing ELMAH.
    -->
    <security allowRemoteAccess="false" />
    <errorLog type="Elmah.Io.ElasticSearch.ElasticSearchErrorLog, Elmah.Io.ElasticSearch" connectionStringName="ElmahElasticSearch" defaultIndex="elmah" />
  </elmah>

And I'm getting following exception.

System.ArgumentOutOfRangeException was unhandled by user code
HResult=-2146233086
Message=startIndex cannot be larger than length of string.
Parameter name: startIndex
Source=mscorlib
ParamName=startIndex
StackTrace:
at System.String.Substring(Int32 startIndex, Int32 length)
at System.String.Substring(Int32 startIndex)
at Elmah.Io.ElasticSearch.ElasticSearchErrorLog.GetDefaultIndexFromConnectionString(String connectionString) in z:\Builds\work\999c9445caaa3f48\Elmah.Io.ElasticSearch\ElasticSearchErrorLog.cs:line 159
at Elmah.Io.ElasticSearch.ElasticSearchErrorLog.GetDefaultIndex(IDictionary config, String connectionString) in z:\Builds\work\999c9445caaa3f48\Elmah.Io.ElasticSearch\ElasticSearchErrorLog.cs:line 146
at Elmah.Io.ElasticSearch.ElasticSearchErrorLog.InitElasticSearch(IDictionary config) in z:\Builds\work\999c9445caaa3f48\Elmah.Io.ElasticSearch\ElasticSearchErrorLog.cs:line 113
at Elmah.Io.ElasticSearch.ElasticSearchErrorLog..ctor(IDictionary config) in z:\Builds\work\999c9445caaa3f48\Elmah.Io.ElasticSearch\ElasticSearchErrorLog.cs:line 24
InnerException:

6.0.8

Today each call to Elmah results in a new ElasticClient, which is not recommended at all.

Put the latest build and NUGET info into the readme file.

Not sure if this is a real problem or not, but I wanted to document it for investigation.

Adapt the following standards:

1. time field is "@timestamp"
2. all string fields are multi-fields.

The build server is not yet configured to support C# 6, so for now we're going to remove those instances

A common usecase for Elmah logging is multiple customers and environment in a single ES index. This could be supported with the applicationName field, but it may be better to store these in ES as separate, distinct fields.

It's free for open source project. It will automagically detect all the tests of the project. You can also add a badge with build status.

suggested priv setup:

# Elmah
elmah:
  indices:
    'elmah':
      privileges: read, index, indices:admin/get, indices:admin/exists, indices:admin/mapping/put, create_index

In ElasticClientSingleton.cs PropertiesDescriptor method, multi field suffix doesn't create raw field when index is newly created.

Here is the elmah index mapping:

{
   "elmah": {
      "mappings": {
         "error": {
            "properties": {
               "@timestamp": {
                  "type": "date",
                  "format": "dateOptionalTime"
               },
               "applicationName": {
                  "type": "string"
               },
               "detail": {
                  "type": "string"
               },
               "environmentName": {
                  "type": "string"
               },
               "errorXml": {
                  "type": "string"
               },
               "hostName": {
                  "type": "string"
               },
               "message": {
                  "type": "string"
               },
               "source": {
                  "type": "string"
               },
               "statusCode": {
                  "type": "long"
               },
               "type": {
                  "type": "string"
               }
            }
         }
      }
   }
}

Would you be opposed to reverting .Net versions from 4.7.2 to 4.6.1? It looks like it was updated across the board in this commit, and the nuspec was fixed here to go along with it.

The updates don't seem to actually require 4.7.2 however (only based on a local build succeeding, I could be wrong), so I'm wondering if reverting could be done to expand compatibility with those of us who aren't on the latest and greatest :)

I already using Elmah to send email with errors.

I tested the sample and works fine.

When i configured to ElasticSearch in my webapi project the output:

GET elmah/error/_search { "query": { "match_all": {} } }

this result

{ "took": 1, "timed_out": false, "_shards": { "total": 5, "successful": 5, "skipped": 0, "failed": 0 }, "hits": { "total": 4, "max_score": 1, "hits": [ { "_index": "elmah", "_type": "error", "_id": "3dox_WQBVZNQ__bDh_oa", "_score": 1, "_source": { "query": { "match_all": {} } } }, { "_index": "elmah", "_type": "error", "_id": "_search_shards", "_score": 1, "_source": { "query": { "match_all": {} } } }, { "_index": "elmah", "_type": "error", "_id": "3Nox_WQBVZNQ__bDafq5", "_score": 1, "_source": { "query": { "match_all": {} } } }, { "_index": "elmah", "_type": "error", "_id": "_percolate", "_score": 1, "_source": { "query": { "match_all": {} } } } ] } }