hariom282538 / graylog-content-pack-haproxy-json Goto Github PK

Hi, install went well, many thanks, but the haproxy_statusCode from the following line is not parsed sucessfully:

funk haproxy[26784]: {haproxy_clientIP:193.174.118.209,haproxy_clientPort:51222,haproxy_dateTime:27/Nov/2018:21:44:43.568,haproxy_frontendNameTransport:fe_whs_mail~,haproxy_backend:be_WebMail,haproxy_serverName:mail.koellers.net,haproxy_Tw:0,haproxy_Tc:5,haproxy_Tt:444,haproxy_bytesRead:1425,haproxy_terminationState:--,haproxy_actconn:1,haproxy_FrontendCurrentConn:1,haproxy_backendCurrentConn:0,haproxy_serverConcurrentConn:0,haproxy_retries:0,haproxy_srvQueue:0,haproxy_backendQueue:0,haproxy_backendSourceIP:192.168.10.1,haproxy_backendSourcePort:60200,haproxy_statusCode:404,haproxy_serverIP:192.168.1.40,haproxy_serverPort:443,haproxy_frontendIP:94.79.171.94,haproxy_frontendPort:443,haproxy_capturedRequestHeaders:{mail.koellers.net},haproxy_httpRequest:GET /sdfsdfsdfs/erwer HTTP/1.1}

The "Manage Stream Rules" states:

Field haproxy_statusCode must be greater than 399
Field haproxy_statusCode must be smaller than 500

No Idea what's wrong here.

Does it work with Graylog 3.1?

Followed all the instructions but I get blank messages

This is my rsyslog.d/49-graylog.conf

$template GRAYLOGRFC5424,"<%PRI%>%PROTOCOL-VERSION% %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\$
local0.=info -/var/log/haproxy/haproxy.log;GRAYLOGRFC5424
local0.=info @graylog:12211;GRAYLOGRFC5424
& stop

And here is what I get in the haproxy.log (IPs and server names scrubbed)

<134>0 2020-02-16T14:18:54+00:00 11891690394b haproxy 1 - -  X.X.X.X:31494 [16/Feb/2020:14:18:54.095] external~ apache/apache 0/0/0/4/4 200 10618 - - ---- 1/1/0/1/0 0/0 {XXXXXXXXX|Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36||Y.Y.Y.Y} "GET /index.htm?=& HTTP/1.1"$

I had an error installing content-pack.json

error messages:
Installing content pack failed with status: Error:cannot post http://X.X.X.X:9000/api/system/content_packs/xxxxxx/installations(500) could not install content pack

Thanks for the hard work you put into the project!
I am receiving gl2_processing_error

Could not apply extractor <Extract JSON fields (15adfb62-d840-4a6f-828f-a35daf2a3cd4)> - Unexpected character ('.' (code 46)): Expected space separating root-level values
at [Source: (String)"IP_ADDRESS:PORT [20/Oct/2022:20:09:52.033] vfrontend~ vbackend/backend-n2 0/0/0/2/2 200 781 - - ---- 2/1/0/0/0 0/0 {DOMAIN|Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) Gecko/20100101 Firefox/106.0|DOMAIN/URL.html|} "GET /URL_2.html HTTP/1.1"$"; line: 1, column: 9].

Tried loading the log format file from the issue #4 but did no help, debugging deeper now

Hello, I am missing the fields
haproxy_clientIP_country_code
haproxy_clientIP_geolocation
The geo map is set up and I have also set up a lookup table (with data adapter and cache), what do I have to do to get the two fields to come?

Thanks a lot

With Elastic version 7.x
this command

$ curl -X PUT -d @'graylog-custom-mapping.json' -H 'Content-Type: application/json' 'http://localhost:9200/_template/graylog-custom-mapping?pretty'
Dosent work anymore for the graylog-custom-mapping.json

{
"template": "graylog_*",
"mappings" : {
"message" : {
"properties" : {
"haproxy_Tc" : {
"type" : "long"
},
"haproxy_Tt" : {
"type" : "long"
},
"haproxy_Tw" : {
"type" : "long"
},
"haproxy_bytesRead" : {
"type" : "long"
}
}
}
}
}

It is necessary to remove this line
"message" : {
And to remove the corresponding curly bracket
}

The working graylog-custom-mapping.json
is the following

{
"template": "graylog_*",
"mappings" : {

        "properties" : {
            "haproxy_Tc" : {
                "type" : "long"
            },
            "haproxy_Tt" : {
                "type" : "long"
            },
            "haproxy_Tw" : {
                "type" : "long"
            },
            "haproxy_bytesRead" : {
                "type" : "long"
            }
        }

}

}

Thanks for your work :)

Info: https://xyzcoder.github.io/elasticsearch/nest/2019/04/12/es-70-and-nest-mapping-error.html

I want to install this content pack on graylog version v3.0.2 but I did not goes well, I want to know that it is compatible with version 3.0.2 or not?

Do you have plans upgrade it to work with Graylog5 ?