sipcapture / pastash Goto Github PK

View Code? Open in Web Editor NEW

101.0 12.0 27.0 6.04 MB

pastaʃ'ʃ = Spaghetti I/O Event Data Processing, Interpolation, Correlation and beyond :spaghetti:

Home Page: http://sipcapture.io

License: Apache License 2.0

JavaScript 98.59% Shell 0.70% Yacc 0.65% Dockerfile 0.03% Procfile 0.03%

logstash elasticsearch zeromq logs node redis hep netflow amqp parser

pastash's Issues

paStash docker and Audiocodes optional filter

Hi,

I have tried to build up-to-date docker image for paStash and it launches ok with the given Dockerfile. However, if I want to add the Audiocodes SBC filter pastash_sonus.conf in the /config, I'll get the following:

[Thu, 28 Jan 2021 07:54:44 GMT] NOTICE Starting pastash 1.0.71
[Thu, 28 Jan 2021 07:54:44 GMT] INFO Max http socket 100
[Thu, 28 Jan 2021 07:54:44 GMT] INFO Loading config files from : /config
[Thu, 28 Jan 2021 07:54:44 GMT] INFO Files loaded from directory, 3 urls found
[Thu, 28 Jan 2021 07:54:44 GMT] INFO Loading config : 6 urls
[Thu, 28 Jan 2021 07:54:44 GMT] INFO Initializing output HEP/EEP Server
[Thu, 28 Jan 2021 07:54:44 GMT] INFO Start output to HEP udp to 127.0.0.1:9060
[Thu, 28 Jan 2021 07:54:44 GMT] INFO Initializing filter AddHost
[Thu, 28 Jan 2021 07:54:44 GMT] INFO Initializing filter AddTimestamp
[Thu, 28 Jan 2021 07:54:44 GMT] INFO Initializing filter AddVersion
Loading npm module... @pastash/filter_app_audiocodes
[Thu, 28 Jan 2021 07:54:44 GMT] ERROR Unable to load urls from command line
[Thu, 28 Jan 2021 07:54:44 GMT] ERROR TypeError: Cannot read property 'create' of undefined
at LogstashAgent.configure (/usr/src/app/lib/agent.js:143:21)
at LogstashAgent. (/usr/src/app/lib/agent.js:215:10)
at /usr/src/app/node_modules/async/dist/async.js:3096:16
at replenish (/usr/src/app/node_modules/async/dist/async.js:998:17)
at iterateeCallback (/usr/src/app/node_modules/async/dist/async.js:983:17)
at /usr/src/app/node_modules/async/dist/async.js:958:16
at LogstashAgent. (/usr/src/app/lib/agent.js:265:5)
at LogstashAgent. (/usr/src/app/lib/agent.js:220:7)
at LogstashAgent. (/usr/src/app/lib/agent.js:156:9)
at FilterAddVersion. (/usr/src/app/lib/lib/base_filter.js:36:5)

I have installed all sorts of dependencies into the image trying to solve the issue, but no luck so far. Is this something I can fix in the build process or some other error? Thanks!

Splitting Modules from Core

@ALL paStash is growing a number of modules, and risks looking a pinch too monolithic for the possibilities NodeJS offers. Is anyone willing to help out moving a few modules (and their dependencies) outside of the core? Any help, suggestion or contribution is well appreciated!

paStash installation failed

hi
system version：debian 10
Install this way ： sudo npm install -g @pastash/pastash --unsafe-perm

error log：

Using simply a grok output an error

Installed the app globaly and LOKI running locally

at first had this

input {
  file {
    path => "./logs/*.log"
  }
}

output {
  loki {
    host => localhost
    port => 3100
    path => "/loki/api/v1/push"
  }
}

and each time I insert logs I get [Wed, 20 Jan 2021 16:34:15 GMT] ERROR error Wrong HTTP Post return code: 400

So I taught maybe I need to parse better each line with this :

filter { 
  grok { 
    match => '%{TIMESTAMP_ISO8601:timestamp}: %{GREEDYDATA:message}' 
  }
}

Now I only get this error of loading module.

Loading npm module... @pastash/filter_grok
[Wed, 20 Jan 2021 16:16:13 GMT] ERROR Unable to load urls from command line
[Wed, 20 Jan 2021 16:16:13 GMT] ERROR TypeError: Cannot read property 'create' of undefined
    at LogstashAgent.configure (C:\Users\LeChat\AppData\Roaming\npm\node_modules\@pastash\pastash\lib\agent.js:143:21)
    at LogstashAgent.<anonymous> (C:\Users\LeChat\AppData\Roaming\npm\node_modules\@pastash\pastash\lib\agent.js:215:10)
    at C:\Users\LeChat\AppData\Roaming\npm\node_modules\@pastash\pastash\node_modules\async\dist\async.js:3096:16
    at replenish (C:\Users\LeChat\AppData\Roaming\npm\node_modules\@pastash\pastash\node_modules\async\dist\async.js:998:17)
    at iterateeCallback (C:\Users\LeChat\AppData\Roaming\npm\node_modules\@pastash\pastash\node_modules\async\dist\async.js:983:17)
    at C:\Users\LeChat\AppData\Roaming\npm\node_modules\@pastash\pastash\node_modules\async\dist\async.js:958:16
    at LogstashAgent.<anonymous> (C:\Users\LeChat\AppData\Roaming\npm\node_modules\@pastash\pastash\lib\agent.js:265:5)
    at LogstashAgent.<anonymous> (C:\Users\LeChat\AppData\Roaming\npm\node_modules\@pastash\pastash\lib\agent.js:220:7)
    at LogstashAgent.<anonymous> (C:\Users\LeChat\AppData\Roaming\npm\node_modules\@pastash\pastash\lib\agent.js:156:9)
    at FilterAddVersion.<anonymous> (C:\Users\LeChat\AppData\Roaming\npm\node_modules\@pastash\pastash\lib\lib\base_filter.js:36:5)

I dont have the easier time using that application. what I am missing

Audiocodes SBC syslog errors

Hi,

When using the audiocodes pastash plugin, I'm getting the following errors:

Wed, 03 Mar 2021 20:47:14 GMT] NOTICE Starting pastash 1.0.71
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Max http socket 100
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Loading config file : /home/peter/pastash/sbc-stack01.conf
[Wed, 03 Mar 2021 20:47:14 GMT] INFO File loaded, 3 urls found
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Loading config : 6 urls
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Initializing output HEP/EEP Server
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Start output to HEP udp to 127.0.0.1:9060
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Initializing filter AddHost
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Initializing filter AddTimestamp
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Initializing filter AddVersion
Loading npm module... @pastash/filter_app_audiocodes
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Initializing filter AppAudiocodes
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Initialized App Audiocodes SysLog to SIP/HEP parser
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Initializing input Udp
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Start listening on udp 0.0.0.0:925
[Wed, 03 Mar 2021 20:47:14 GMT] INFO Config loaded.
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Exception has been catch, it' a bug
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Please submit an issue on https://github.com/sipcapture/pastash
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Exception: SyntaxError: Invalid regular expression: /[SID=(?.?):(?.?):(?.?)]/: Invalid group
at FilterAppAudiocodes.process (/usr/lib/node_modules/@pastash/filter_app_audiocodes/filter_app_audiocodes.js:114:71)
at FilterAppAudiocodes. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:21:24)
at emitOne (events.js:96:13)
at FilterAppAudiocodes.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:27:18)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR SyntaxError: Invalid regular expression: /[SID=(?.?):(?.?):(?.?)]/: Invalid group
at FilterAppAudiocodes.process (/usr/lib/node_modules/@pastash/filter_app_audiocodes/filter_app_audiocodes.js:114:71)
at FilterAppAudiocodes. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:21:24)
at emitOne (events.js:96:13)
at FilterAppAudiocodes.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:27:18)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Exception has been catch, it' a bug
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Please submit an issue on https://github.com/sipcapture/pastash
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Exception: SyntaxError: Invalid regular expression: /[SID=(?.?):(?.?):(?.?)]/: Invalid group
at FilterAppAudiocodes.process (/usr/lib/node_modules/@pastash/filter_app_audiocodes/filter_app_audiocodes.js:114:71)
at FilterAppAudiocodes. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:21:24)
at emitOne (events.js:96:13)
at FilterAppAudiocodes.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:27:18)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR SyntaxError: Invalid regular expression: /[SID=(?.?):(?.?):(?.?)]/: Invalid group
at FilterAppAudiocodes.process (/usr/lib/node_modules/@pastash/filter_app_audiocodes/filter_app_audiocodes.js:114:71)
at FilterAppAudiocodes. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:21:24)
at emitOne (events.js:96:13)
at FilterAppAudiocodes.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:27:18)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Exception has been catch, it' a bug
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Please submit an issue on https://github.com/sipcapture/pastash
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Exception: SyntaxError: Invalid regular expression: /[SID=(?.?):(?.?):(?.?)]/: Invalid group
at FilterAppAudiocodes.process (/usr/lib/node_modules/@pastash/filter_app_audiocodes/filter_app_audiocodes.js:114:71)
at FilterAppAudiocodes. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:21:24)
at emitOne (events.js:96:13)
at FilterAppAudiocodes.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:27:18)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR SyntaxError: Invalid regular expression: /[SID=(?.?):(?.?):(?.?)]/: Invalid group
at FilterAppAudiocodes.process (/usr/lib/node_modules/@pastash/filter_app_audiocodes/filter_app_audiocodes.js:114:71)
at FilterAppAudiocodes. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:21:24)
at emitOne (events.js:96:13)
at FilterAppAudiocodes.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:27:18)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Exception has been catch, it' a bug
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Please submit an issue on https://github.com/sipcapture/pastash
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR Exception: SyntaxError: Invalid regular expression: /[SID=(?.?):(?.?):(?.?)]/: Invalid group
at FilterAppAudiocodes.process (/usr/lib/node_modules/@pastash/filter_app_audiocodes/filter_app_audiocodes.js:114:71)
at FilterAppAudiocodes. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:21:24)
at emitOne (events.js:96:13)
at FilterAppAudiocodes.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:27:18)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
[Wed, 03 Mar 2021 20:47:18 GMT] ERROR SyntaxError: Invalid regular expression: /[SID=(?.?):(?.?):(?.?)]/: Invalid group
at FilterAppAudiocodes.process (/usr/lib/node_modules/@pastash/filter_app_audiocodes/filter_app_audiocodes.js:114:71)
at FilterAppAudiocodes. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:21:24)
at emitOne (events.js:96:13)
at FilterAppAudiocodes.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
at FilterAddVersion. (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:27:18)
at emitOne (events.js:96:13)
at FilterAddVersion.emit (events.js:188:7)
^C[Wed, 03 Mar 2021 20:47:26 GMT] INFO SIGINT received.
[Wed, 03 Mar 2021 20:47:26 GMT] INFO Closing listening udp 0.0.0.0:925
[Wed, 03 Mar 2021 20:47:26 GMT] INFO Closing output to HEP udp to 127.0.0.1:9060
[Wed, 03 Mar 2021 20:47:26 GMT] INFO Closing agent
[Wed, 03 Mar 2021 20:47:26 GMT] INFO Quitting.

Running on CentOS 7. Installed Homer with the standard install script. Audiocodes CE on Azure with firmware 7.20CO.256.016

Pastash unable to install with Node Version 16.

Hi,

I am trying to install Pastash on Centos using Node Version 16 but got this error:

[root@localhost ~\]# npm install --unsafe-perm -g @pastash/pastash
npm WARN deprecated [email protected]: This module relies on Node.js's internals and will break at some point. Do not use it, and update to [email protected].
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)
npm WARN deprecated [email protected]: no longer maintained
npm WARN deprecated [email protected]: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.

insert host name to ip addresses

we are attempting to use an SQLite filter to capture incoming IP addresses in the index and then add a name to each IP address from the database.

the IPs are custom and not related to DNS. we want to take IPs and attach our client's hostnames.
the list in the DB will be updated frequently.

is there any way to achieve that (reverse DNS is not fit for this task)

sqlite {
db => '/home/folder/userdata_qa.db'
query => 'SELECT service from ip_list WHERE ip_number=(?) limit 1;'
source_field => 'ip_v4'
target_field => 'host_name'
}

this is not working well for us.

prefix date and time output

hi, i am trying to output my index like this :

output {
  elasticsearch {
    host => 0.0.0.0
    port => 9200
    bulk_limit => 1000
    bulk_timeout => 100
    index_prefix => "netflow-%{+YYYY-MM-dd.HH}"
    data_type => netflow
  }
}

but i don't receive any index in the elastic.

it works only when i put only the name "netflow"
index_prefix => netflow

Cisco UCM CMR filter

in reference to #70 i want to add further information to homer by retrieving data of cisco CMR files.

the structure is as follwos:

column name | type | example

Hep split via

Hi,
I tried to use this example do duplicate some udp hep to both servers :

https://github.com/sipcapture/paStash/wiki/Example:-HEP-Relay

But cannot obtain what i expected. :

Config :

`input {
udp {
host => 172.16.0.112
port => 9060
}
}

output {
udp {
host => 172.16.0.95
port => 9060
}
udp {
host => 172.16.0.131
port => 9060
}
}`

Tcpdump checks :

Received 1030 hep packets > 172.16.0.112.9060
Sent 76 hep packets > 172.16.0.131.9060
Sent 894 hep packets > 172.16.0.95.9060

Can you confirm if i'm going the good way and is even feasible ?

Thanks

app_cisco (Cisco ISR) filter plugin DNS lookup error

I am testing out the app_cisco filter for Cisco ISR devices.
I am running pastash as a container built using the following Dockerfile:

FROM node:17-alpine

RUN npm install --unsafe-perm -g @pastash/pastash @pastash/filter_app_audiocodes @pastash/filter_app_cisco

EXPOSE 514
CMD [ "pastash", "--config_file=/pastash.conf" ]

The pastash docker-compose service config is:

pastash:
    image: custom_pastash:17-alpine
    container_name: custom_pastash
    volumes:
      - ./pastash.conf:/pastash.conf
    ports:
      - "514:514/udp"

This is the pastash.config:

input {
  udp {
    host => 0.0.0.0
    port => 514
    type => syslog
  }
}
filter {
        if [udp_port] == 514  {
                multiline {
                  start_line_regex => /^<\d+?>\d+:\s(\*|)[A-Za-z]{3}\s{1,2}\d{1,2}\s\d{2}:\d{2}:\d{2}\.\d+.*ccsipDisplayMsg:/
                }
                app_cisco{
                  debug => true
                }
        }
}
output {
        hep {
          host => '1.2.3.4'
          port => 9060
          hep_id => 2223
          hep_type => 1
          hep_protocol => 6
        }
}

I am able to successfully send SIP messages as Syslog.

On pastash logs however I see these kind of errors:

Error: getaddrinfo ENOTFOUND sip-du-a-as.pstnhub.microsoft.com:5061
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:72:26) {
  errno: -3008,
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'sip-du-a-as.pstnhub.microsoft.com:5061'
}
undefined

This particular Cisco ISR device is connecting to Microsoft Teams.
They send the identity of their SIP Proxies by using the port as well.
It looks like pastash is trying to resolve the names with the ports included and this of course fails.

Is this working as designed or could I add something in the configuration to address this?

Many thanks for the help here.

ERROR Exception has been catch, it' a bug

hello.

if you need more info, let me know.

[Mon, 13 Jul 2020 16:28:44 GMT] ERROR Exception has been catch, it' a bug
[Mon, 13 Jul 2020 16:28:44 GMT] ERROR Please submit an issue on https://github.com/sipcapture/pastash
[Mon, 13 Jul 2020 16:28:44 GMT] ERROR Exception: TypeError: Object.entries is not a function
    at LokiPost.process (/usr/lib/node_modules/@pastash/output_loki/output_loki.js:78:33)
    at LokiPost.<anonymous> (/usr/lib/node_modules/@pastash/pastash/lib/lib/base_output.js:21:14)
    at emitOne (events.js:96:13)
    at LokiPost.emit (events.js:188:7)
    at LogstashAgent.<anonymous> (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:273:14)
    at emitOne (events.js:96:13)
    at LogstashAgent.emit (events.js:188:7)
    at LogstashAgent.<anonymous> (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:18:10)
    at emitOne (events.js:96:13)
    at FilterAddVersion.emit (events.js:188:7)

pastash_loki.conf

input {
    file {
        path => "/var/log/*.log"
    }
}

output {
    loki {
        host => "loki.my.domain"
        port => 80
        path => "/loki/api/v1/push"
    }
}

started pastash with pastash --config_file=/root/pastash_loki.conf

root@elastic-01:~# node -v
v6.17.1

imagestore

Audiocodes SBC syslog

Hi, I'am trying to send "syslog" coming from Audiocodes SBC to Homer 7.
What i did so far is a configuration file for pastash:

`input {
udp {
host => 0.0.0.0
port => 514
type => syslog
}
}

output {
stdout{}
if [rcinfo] != 'undefined' {
hep {
host => '10.160.21.80'
port => 9069
hep_id => 2222
hep_type => 1
}
}
}`

and i am sending the following output to Homer 7:

[STDOUT] { "message": "<133>[S=1455266] [SID=b9027c:24:167717] (N 5828845) AcSIPDialog(#1532): Handling GENERAL_RESPONSE_REQ in state DialogInitiated \n(N 5828846) States: (#1532)AcSIPDialog[DialogInitiated->DialogConnected] \n(N 5828847) ---- Outgoing SIP Message to 52.114.76.76:12544 from SIPInterface #2 (Teams) TLS TO(#156) SocketID(182) ---- \nSIP/2.0 200 OK \nVia: SIP/2.0/TLS 52.114.76.76:5061;branch=z9hG4bK5e9ae5e1 \nFrom: <sip:sip-du-a-eu.pstnhub.microsoft.com:5061>;tag=d652f27d-dba2-466e-b4a6-c0b6db2fd363 \nTo: <sip:10.160.111.51>;tag=1c345880336 \nCall-ID: 7ac9a01a-9624-41b8-970d-c903c5b24cd5 \nCSeq: 1 OPTIONS \nContact: <sip:sbc.domain.com:5061;transport=tls> \nServer: SBC Lab/v.7.20A.260.012 \nContent-Length: 0 \n \n \n(N 5828848) AcSIPDialog(#1532): Handling DIALOG_DISCONNECT_REQ in state DialogConnected \n(N 5828849) States: (#1532)AcSIPDialog[DialogConnected->DialogDisconnected] \n(N 5828850) RELEASE_ACK_EV: (#117)SIPSBCDialogLeg -> (#23)SBCDialog[Disconnecting->Disconnected] \n -> (#17)SBCEndPoint[Releasing->Released] \n -> (#8)SBCController[Disconnecting->Disconnected] \n -> (#16)SBCEndPoint[Releasing->Released] \n -> (#90)SBCDialog[Disconnecting->Disconnected] \n -> (#113)SIPSBCDialogLeg[Deallocated] \n [Time:17-12@17:08:32.551]", "host": "10.160.21.20", "udp_port": "514", "type": "syslog", "@timestamp": "2020-12-17T15:08:39.891Z", "@version": "1" } [STDOUT] { "message": "<133>[S=1455267] [SID=b9027c:24:167717] (N 5828851) SIPAppMngr::GetControlIPAddress - Near NAT translation found for SIP Interface 2. Translated IP Address 82.185.88.164:5061 \n(N 5828852) States: (#117)SIPSBCDialogLeg[Deallocated] \n(N 5828853) Discarding event SBC_ROUTING_DONE_EV. Receiver is invalid (#127) \n(N 5828854) States: (#57)SBCRoutesIterator[Deallocated] \n (#127)SBCFeature[Deallocated] \n (#8)SBCController[Deallocated] \n(N 5828855) CAC: Remove SBC Outgoing Other, IPG 1 (Teams): 0, SRD 0 (DefaultSRD): 0, SipIF 2 (Teams): 0 \n(N 5828856) States: (#90)SBCCall[Deallocated] \n(N 5828857) CAC: Remove SBC Incoming Other, IPG 1 (Teams): 0, SRD 0 (DefaultSRD): 0, SipIF 2 (Teams): 0 \n(N 5828858) States: (#23)SBCCall[Deallocated] \n [Time:17-12@17:08:32.552]", "host": "10.160.21.20", "udp_port": "514", "type": "syslog", "@timestamp": "2020-12-17T15:08:39.892Z", "@version": "1" } [STDOUT] { "message": "<135>[S=1455268] [BID=b9027c:24] _DnsCallback: end query recieved _GetInterfaceIndexByCtx(ctx) 1, q->qtyp 1 q->name sip-du-a-as.pstnhub.microsoft.com, p.dnsp_ttl 2 [File:DnsApi_Linux.cpp Line:1390] [Time:17-12@17:08:32.560]", "host": "10.160.21.20", "udp_port": "514", "type": "syslog", "@timestamp": "2020-12-17T15:08:39.896Z", "@version": "1" } [STDOUT] { "message": "<135>[S=1455269] [BID=b9027c:24] _DnsUpdateCacheEntryAddrInfo: update addr for query sip-du-a-as.pstnhub.microsoft.com [File:DnsApi_Linux.cpp Line:360] [Time:17-12@17:08:32.560]", "host": "10.160.21.20", "udp_port": "514", "type": "syslog", "@timestamp": "2020-12-17T15:08:39.896Z", "@version": "1" } [STDOUT] { "message": "<135>[S=1455270] [BID=b9027c:24] sip-du-a-as.pstnhub.microsoft.com resolved to 52.114.7.24 [File:DnsApi_Linux.cpp Line:313] [Time:17-12@17:08:32.560]", "host": "10.160.21.20", "udp_port": "514", "type": "syslog", "@timestamp": "2020-12-17T15:08:39.897Z", "@version": "1" }

However I cannot see anything on Homer server.
Any help?

Regards

Audiocodes Plugin: ERROR TypeError: Cannot read property '1' of null

Hi @lmangani

I did some tests after our first conversation today, here my setup:
Audiocodes VE 7.20A.258.367
nodejs: v14.17.0
heplify-server 1.54

What I expect: I would like to see the traffic in Homer7 from the Audiocodes SBC between my SIP-Proxy and MS-Teams.
Is this possible?

First confusion:
https://github.com/sipcapture/paStash/wiki/Example:-AUDIOCODES-Syslog says:
"Supports 7.20A.260.012 (or lower) and 7.20A.256.511 (or higher)."
https://github.com/sipcapture/paStash/blob/next/plugins/filters/app_audiocodes/app_audiocodes.md says:
"Supports 7.20A.260.012 (or higher) and 7.20A.256.511 (or lower)."

Is my Version now supported or not?

paStash runs on the same Server as Homer7 and heplify-server, here my config (IP's changed):

input {
  udp {
    host => 192.168.1.100
    port => 514
    type => syslog
  }
}

filter {
  app_audiocodes{
    localip => 192.168.1.101
    autolocal => true
    localport => 5060
    debug => true
    version => '7.20A.256.511'
  }
}

output {
  if [rcinfo] != 'undefined' {
        hep {
          host => '127.0.0.1'
          port => 9060
          hep_id => 2222
        }
  }
}

I see many of these:

[Tue, 08 Jun 2021 14:08:02 GMT] ERROR TypeError: Cannot read property '1' of null
    at FilterAppAudiocodes.process (/usr/local/lib/node_modules/@pastash/filter_app_audiocodes/filter_app_audiocodes.js:103:64)
    at FilterAppAudiocodes.<anonymous> (/usr/local/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:21:24)
    at FilterAppAudiocodes.emit (events.js:376:20)
    at FilterAddVersion.<anonymous> (/usr/local/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
    at FilterAddVersion.emit (events.js:376:20)
    at FilterAddVersion.<anonymous> (/usr/local/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:27:18)
    at FilterAddVersion.emit (events.js:376:20)
    at FilterAddTimestamp.<anonymous> (/usr/local/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
    at FilterAddTimestamp.emit (events.js:376:20)
    at FilterAddTimestamp.<anonymous> (/usr/local/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:27:18)
DEBUG <134>[S=5605569] [BID=5dd74d:40]  Trace 0-4: 6bda42 GetTraceBack(+0x4b) 14a4b0b SystemError(+0x27b) 14a6219 IsrMain(+0x1f0) 7f40b5669aa1 pthread_getattr_np(+0x91) 7f40b43b5c4d hol_append(+0x2bd)  [Time:08-06@14:08:00.447]
[Tue, 08 Jun 2021 14:08:02 GMT] ERROR Exception has been catch, it' a bug
[Tue, 08 Jun 2021 14:08:02 GMT] ERROR Please submit an issue on https://github.com/sipcapture/pastash
[Tue, 08 Jun 2021 14:08:02 GMT] ERROR Exception: TypeError: Cannot read property '1' of null

How to debug further?

Regards

following issue : trying to use sqlite filter #209

Hello,

I'm trying to use sqlite to add the service name according to the Destination port.
the problem is that it looks like the system takes the port number from the first packet and don't update it.

in this example from the log, even though ports are changing all the time the port remains 443

pastash config:

sqlite {
db => '/home/******/userdata_qa.db'
query => 'SELECT service_name from port_list WHERE port_number=(?) limit 1;'
source_field => 'l4_dst_port'
target_field => 'service'
}

sqlite config

sqlite> PRAGMA table_info(port_list);
0|id|INTEGER|0||1
1|port_number|long|0||0
2|service|TEXT|1||0

sqlite> SELECT * from port_list;
1|80|http
2|53|dns
3|123|ntp
4|80|http
5|443|https
6|1194|ovpn
7|8883|mqtt

Installation fails on Debian 10

Hi there!
I would like to test paStash with the Audiocodes Plugin, but I am not able to build it. There is some error with the zmq module and amqplib. Also installed libzmq3-dev, but did not help:

Here the command I used to install paStash:

npm install --unsafe-perm -g @pastash/pastash

Here is the error output:

make: *** [zmq.target.mk:103: Release/obj.target/zmq/binding.o] Error 1
make: Leaving directory '/usr/lib/node_modules/@pastash/pastash/node_modules/zmq/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:191:23)
gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Linux 4.19.0-16-amd64
gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /usr/lib/node_modules/@pastash/pastash/node_modules/zmq
gyp ERR! node -v v10.24.1
gyp ERR! node-gyp -v v5.1.0
gyp ERR! not ok 
npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">=0.8 <6 || ^6"} (current: {"node":"10.24.1","npm":"6.14.12"})
npm WARN notsup Not compatible with your version of node/npm: [email protected]
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/@pastash/pastash/node_modules/zmq):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] install: `node-gyp rebuild`
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1

Any help on this?

Pastash installation errors

Hi,

I'm trying to install pastash on a SLES 12.3 and having no luck with it.

npm -v: 6.13.6
nvm alias default node
default -> node (-> v12.14.1)

` npm install --unsafe-perm -g @pastash/pastash
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: This module relies on Node.js's internals and will break at some point. Do not use it, and update to [email protected].
/root/.nvm/versions/node/v12.14.1/bin/pastash -> /root/.nvm/versions/node/v12.14.1/lib/node_modules/@pastash/pastash/bin/pastash

[email protected] install /root/.nvm/versions/node/v12.14.1/lib/node_modules/@pastash/pastash/node_modules/oniguruma
node-gyp rebuild

make: Entering directory '/root/.nvm/versions/node/v12.14.1/lib/node_modules/@pastash/pastash/node_modules/oniguruma/build'
CC(target) Release/obj.target/oniguruma/deps/onig/regcomp.o
CC(target) Release/obj.target/oniguruma/deps/onig/regenc.o
CC(target) Release/obj.target/oniguruma/deps/onig/regerror.o
CC(target) Release/obj.target/oniguruma/deps/onig/regexec.o
CC(target) Release/obj.target/oniguruma/deps/onig/regext.o
CC(target) Release/obj.target/oniguruma/deps/onig/reggnu.o
CC(target) Release/obj.target/oniguruma/deps/onig/regparse.o
CC(target) Release/obj.target/oniguruma/deps/onig/regposerr.o
CC(target) Release/obj.target/oniguruma/deps/onig/regposix.o
CC(target) Release/obj.target/oniguruma/deps/onig/regsyntax.o
CC(target) Release/obj.target/oniguruma/deps/onig/regtrav.o
CC(target) Release/obj.target/oniguruma/deps/onig/regversion.o
CC(target) Release/obj.target/oniguruma/deps/onig/st.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/ascii.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/big5.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/cp1251.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/euc_jp.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/euc_kr.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/euc_tw.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/gb18030.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_1.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_2.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_3.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_4.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_5.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_6.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_7.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_8.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_9.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_10.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_11.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_13.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_14.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_15.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/iso8859_16.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/koi8.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/koi8_r.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/mktable.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/sjis.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/unicode.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/utf16_be.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/utf16_le.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/utf32_be.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/utf32_le.o
CC(target) Release/obj.target/oniguruma/deps/onig/enc/utf8.o
AR(target) Release/obj.target/oniguruma.a
COPY Release/oniguruma.a
CXX(target) Release/obj.target/onig_scanner/src/onig-result.o
CXX(target) Release/obj.target/onig_scanner/src/onig-reg-exp.o
CXX(target) Release/obj.target/onig_scanner/src/onig-scanner.o
../src/onig-scanner.cc: In static member function ‘static void OnigScanner::Init(v8::Localv8::Object)’:
../src/onig-scanner.cc:13:82: error: no matching function for call to ‘v8::FunctionTemplate::GetFunction()’
target->Set(Nan::New("OnigScanner").ToLocalChecked(), tpl->GetFunction());
^
../src/onig-scanner.cc:13:82: note: candidate is:
In file included from /root/.cache/node-gyp/12.14.1/include/node/node.h:63:0,
from ../../nan/nan.h:54,
from ../src/onig-scanner.h:4,
from ../src/onig-scanner.cc:1:
/root/.cache/node-gyp/12.14.1/include/node/v8.h:5995:46: note: v8::MaybeLocalv8::Function v8::FunctionTemplate::GetFunction(v8::Localv8::Context)
V8_WARN_UNUSED_RESULT MaybeLocal GetFunction(
^
/root/.cache/node-gyp/12.14.1/include/node/v8.h:5995:46: note: candidate expects 1 argument, 0 provided
../src/onig-scanner.cc: In static member function ‘static Nan::NAN_METHOD_RETURN_TYPE OnigScanner::FindNextMatchSync(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/onig-scanner.cc:42:85: error: no matching function for call to ‘v8::Value::ToObject()’
OnigString* onigString = node::ObjectWrap::Unwrap(info[0]->ToObject());
^
../src/onig-scanner.cc:42:85: note: candidates are:
In file included from /root/.cache/node-gyp/12.14.1/include/node/node.h:63:0,
from ../../nan/nan.h:54,
from ../src/onig-scanner.h:4,
from ../src/onig-scanner.cc:1:
/root/.cache/node-gyp/12.14.1/include/node/v8.h:2576:44: note: v8::MaybeLocalv8::Object v8::Value::ToObject(v8::Localv8::Context) const
V8_WARN_UNUSED_RESULT MaybeLocal ToObject(
^
/root/.cache/node-gyp/12.14.1/include/node/v8.h:2576:44: note: candidate expects 1 argument, 0 provided
In file included from /root/.cache/node-gyp/12.14.1/include/node/v8-internal.h:14:0,
from /root/.cache/node-gyp/12.14.1/include/node/v8.h:25,
from /root/.cache/node-gyp/12.14.1/include/node/node.h:63,
from ../../nan/nan.h:54,
from ../src/onig-scanner.h:4,
from ../src/onig-scanner.cc:1:
/root/.cache/node-gyp/12.14.1/include/node/v8.h:2590:31: note: v8::Localv8::Object v8::Value::ToObject(v8::Isolate*) const
Local ToObject(Isolate* isolate) const);
^
/root/.cache/node-gyp/12.14.1/include/node/v8config.h:311:3: note: in definition of macro ‘V8_DEPRECATED’
declarator attribute((deprecated(message)))
^
/root/.cache/node-gyp/12.14.1/include/node/v8.h:2590:31: note: candidate expects 1 argument, 0 provided
Local ToObject(Isolate* isolate) const);
^
/root/.cache/node-gyp/12.14.1/include/node/v8config.h:311:3: note: in definition of macro ‘V8_DEPRECATED’
declarator attribute((deprecated(message)))
^
../src/onig-scanner.cc: In constructor ‘OnigScanner::OnigScanner(v8::Localv8::Array)’:
../src/onig-scanner.cc:61:45: warning: ‘v8::Localv8::Value v8::Object::Get(uint32_t)’ is deprecated (declared at /root/.cache/node-gyp/12.14.1/include/node/v8.h:3461): Use maybe version [-Wdeprecated-declarations]
Nan::Utf8String utf8Value(sources->Get(i));
^
../src/onig-scanner.cc: In member function ‘v8::Localv8::Value OnigScanner::FindNextMatchSync(OnigString*, v8::Localv8::Number)’:
../src/onig-scanner.cc:91:98: warning: ‘bool v8::Object::Set(v8::Localv8::Value, v8::Localv8::Value)’ is deprecated (declared at /root/.cache/node-gyp/12.14.1/include/node/v8.h:3402): Use maybe version [-Wdeprecated-declarations]
result->Set(Nan::New("index").ToLocalChecked(), Nan::New(bestResult->Index()));
^
../src/onig-scanner.cc:92:118: warning: ‘bool v8::Object::Set(v8::Localv8::Value, v8::Localv8::Value)’ is deprecated (declared at /root/.cache/node-gyp/12.14.1/include/node/v8.h:3402): Use maybe version [-Wdeprecated-declarations]
result->Set(Nan::New("captureIndices").ToLocalChecked(), CaptureIndicesForMatch(bestResult.get(), source));
^
../src/onig-scanner.cc: In static member function ‘static v8::Localv8::Value OnigScanner::CaptureIndicesForMatch(OnigResult*, OnigString*)’:
../src/onig-scanner.cc:108:85: warning: ‘bool v8::Object::Set(v8::Localv8::Value, v8::Localv8::Value)’ is deprecated (declared at /root/.cache/node-gyp/12.14.1/include/node/v8.h:3402): Use maybe version [-Wdeprecated-declarations]
capture->Set(Nan::New("index").ToLocalChecked(), Nan::New(index));
^
../src/onig-scanner.cc:109:92: warning: ‘bool v8::Object::Set(v8::Localv8::Value, v8::Localv8::Value)’ is deprecated (declared at /root/.cache/node-gyp/12.14.1/include/node/v8.h:3402): Use maybe version [-Wdeprecated-declarations]
capture->Set(Nan::New("start").ToLocalChecked(), Nan::New(captureStart));
^
../src/onig-scanner.cc:110:88: warning: ‘bool v8::Object::Set(v8::Localv8::Value, v8::Localv8::Value)’ is deprecated (declared at /root/.cache/node-gyp/12.14.1/include/node/v8.h:3402): Use maybe version [-Wdeprecated-declarations]
capture->Set(Nan::New("end").ToLocalChecked(), Nan::New(captureEnd));
^
../src/onig-scanner.cc:111:106: warning: ‘bool v8::Object::Set(v8::Localv8::Value, v8::Localv8::Value)’ is deprecated (declared at /root/.cache/node-gyp/12.14.1/include/node/v8.h:3402): Use maybe version [-Wdeprecated-declarations]
capture->Set(Nan::New("length").ToLocalChecked(), Nan::New(captureEnd - captureStart));
^
../src/onig-scanner.cc:112:33: warning: ‘bool v8::Object::Set(uint32_t, v8::Localv8::Value)’ is deprecated (declared at /root/.cache/node-gyp/12.14.1/include/node/v8.h:3411): Use maybe version [-Wdeprecated-declarations]
captures->Set(index, capture);
^
onig_scanner.target.mk:123: recipe for target 'Release/obj.target/onig_scanner/src/onig-scanner.o' failed
make: *** [Release/obj.target/onig_scanner/src/onig-scanner.o] Error 1
make: Leaving directory '/root/.nvm/versions/node/v12.14.1/lib/node_modules/@pastash/pastash/node_modules/oniguruma/build'
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/root/.nvm/versions/node/v12.14.1/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack at ChildProcess.emit (events.js:223:5)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
gyp ERR! System Linux 4.4.143-94.47-default
gyp ERR! command "/root/.nvm/versions/node/v12.14.1/bin/node" "/root/.nvm/versions/node/v12.14.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /root/.nvm/versions/node/v12.14.1/lib/node_modules/@pastash/pastash/node_modules/oniguruma
gyp ERR! node -v v12.14.1
gyp ERR! node-gyp -v v5.0.5
gyp ERR! not ok

[email protected] install /root/.nvm/versions/node/v12.14.1/lib/node_modules/@pastash/pastash/node_modules/zmq
node-gyp rebuild

make: Entering directory '/root/.nvm/versions/node/v12.14.1/lib/node_modules/@pastash/pastash/node_modules/zmq/build'
CXX(target) Release/obj.target/zmq/binding.o
../binding.cc:28:17: fatal error: zmq.h: No such file or directory
#include <zmq.h>
^
compilation terminated.
zmq.target.mk:109: recipe for target 'Release/obj.target/zmq/binding.o' failed
make: *** [Release/obj.target/zmq/binding.o] Error 1
make: Leaving directory '/root/.nvm/versions/node/v12.14.1/lib/node_modules/@pastash/pastash/node_modules/zmq/build'
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/root/.nvm/versions/node/v12.14.1/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack at ChildProcess.emit (events.js:223:5)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
gyp ERR! System Linux 4.4.143-94.47-default
gyp ERR! command "/root/.nvm/versions/node/v12.14.1/bin/node" "/root/.nvm/versions/node/v12.14.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /root/.nvm/versions/node/v12.14.1/lib/node_modules/@pastash/pastash/node_modules/zmq
gyp ERR! node -v v12.14.1
gyp ERR! node-gyp -v v5.0.5
gyp ERR! not ok
npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">=0.8 <6 || ^6"} (current: {"node":"12.14.1","npm":"6.13.6"})
npm WARN notsup Not compatible with your version of node/npm: [email protected]
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: lumberjack-protocol@git://github.com/bpaquet/node-lumberjack-protocol.git (node_modules/@pastash/pastash/node_modules/lumberjack-protocol):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Error: Error while executing:
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: undefined ls-remote -h -t git://github.com/bpaquet/node-lumberjack-protocol.git
npm WARN optional SKIPPING OPTIONAL DEPENDENCY:
npm WARN optional SKIPPING OPTIONAL DEPENDENCY:
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: spawn git EACCES
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/@pastash/pastash/node_modules/oniguruma):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] install: node-gyp rebuild
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/@pastash/pastash/node_modules/zmq):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] install: node-gyp rebuild
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1

@pastash/[email protected]
updated 1 package in 19.778s
`

PgSQL output doesn't reopen connection on error threshold breach

We've been running the PgSQL output plugin for a while in production, and occasionally it attempts to insert duplicate IDs somehow. This shouldn't be a problem for our specific application, except that these primary key conflicts increase the output plugin's error count, and when the error threshold (a hard-coded >= 10) is reached, the plugin should close & reopen the connection and clear the error count.

The problem appears to be related to the node-postgres calls being made to restart that connection. Our logs end up filling up with the following error:

Error: Client has already been connected. You cannot reuse a client.
at Client._connect (/usr/lib/node_modules/@pastash/output_pgsql/node_modules/pg/lib/client.js:92:17)
at /usr/lib/node_modules/@pastash/output_pgsql/node_modules/pg/lib/client.js:302:10
at new Promise ()
at Client.connect (/usr/lib/node_modules/@pastash/output_pgsql/node_modules/pg/lib/client.js:301:10)
at Query.callback (/usr/lib/node_modules/@pastash/output_pgsql/output_pgsql.js:103:36)
at Query.handleError (/usr/lib/node_modules/@pastash/output_pgsql/node_modules/pg/lib/query.js:138:19)
at /usr/lib/node_modules/@pastash/output_pgsql/node_modules/pg/lib/client.js:544:13
at processTicksAndRejections (internal/process/task_queues.js:79:11)

It appears that we should be spinning up a new Client() after calling end() rather than attempting pool.client.end(); pool.client.connect();: brianc/node-postgres#1352

filtering netflow fields

Hi, I am using NetFlow to receive data from a port. I am trying to filter the data and output certain fields to Elasticsearch.

This is the information that I am trying to read off :
"last_switched",
"first_switched",
"in_pkts",
"input_snmp",
"output_snmp",
"src_tos",
"ipv4_next_hop",
"dst_mask",
"src_mask",
"tcp_flags",
"in_dst_mac",
"out_src_mac",
"postNATSourceIPv4Address",
"postNATDestinationIPv4Address",
"postNAPTSourceTransportPort",
"postNAPTDestinationTransportPort",
"fsId",

but didn't find any way to filter the incoming data.
can someone pls guide me on how can i achieve this?

HEP relay/forwarder not working

Hi,

I'm trying to set up paStash as a HEP relat/forwarder so that it will send HEP packets (from rtpproxy 2.2.0) to two separate Homer servers.

I am following this guide:

https://github.com/sipcapture/paStash/wiki/Example:-HEP-Relay

and have configured it to run as a service, as per:

https://github.com/sipcapture/paStash/wiki/pastash-service#running-as-systemd-service

I am using the following paStash config:

input {
  udp {
    host => 127.0.0.1
    port => 9060
  }
}

output {
  udp {
    host => <Homer #1>
    port => 9060
  }
  udp {
    host => <Homer #1>
    port => 9060
  }
}

Note that I'm using UDP as:

rtpproxy only seems to work with HEP using UDP
My Homer servers are currently working with HEP using UDP.

My node version is v16.14.1 running on a CentOS 7.9 machine.

pastash seems to be working in that it receives the input and sends an output to both Homer server IPs.

The original UDP HEP packet contents is:

HEP3...................................	.
bE.....
.
..	6............
.e.......&c61af2ecfc784480a93934fbf9c3a27f.....
m........
..+.......{
 "ssrc": 4199153814,
 "sender_information": {
  "ntp_timestamp_sec": 3857727066,
  "ntp_timestamp_usec": 511629389,
  "rtp_timestamp": 32000,
  "packets": 175,
  "octets": 28000
 },
 "type": 200,
 "report_blocks": [
  {
   "source_ssrc": 0,
   "fraction_lost": 1,
   "packets_lost": 1,
   "highest_seq_no": 22704,
   "ia_jitter": 78,
   "lsr": 0,
   "dlsr": 0
  }
 ],
 "report_count": 1
}

However, the format of the output does not seem correct. When I capture the packets and view the output UDP stream, the contents are:

{
	"message": "HEP3\u0002\u000F\u0000\u0000\u0000\u0001\u0000\u0007\u0002\u0000\u0000\u0000\u0002\u0000\u0007\u0011\u0000\u0000\u0000\u0007\u0000\b......\u0000\u0000\u0000\b\u0000\b...\u0000\u0000\u0000\t\u0000\nbE......\u0000\u0000\u0000\n\u0000\n\u0000\u0002\t6\u0000\u0000\u0000\u000B\u0000\u0007\u0005\u0000\u0000\u0000\f\u0000\n\u0000e\u0000\u0000\u0000\u0000\u0000\u0011\u0000&c61af2ecfc784480a93934fbf9c3a27f\u0000\u0000\u0000\u0003\u0000\nm.........\u0000\u0000\u0000\u0004\u0000\n...\f+...\u0000\u0000\u0000\u000F\u0001...{\n \"ssrc\": 4199153814,\n \"sender_information\": {\n  \"ntp_timestamp_sec\": 3857727066,\n  \"ntp_timestamp_usec\": 511629389,\n  \"rtp_timestamp\": 32000,\n  \"packets\": 175,\n  \"octets\": 28000\n },\n \"type\": 200,\n \"report_blocks\": [\n  {\n   \"source_ssrc\": 0,\n   \"fraction_lost\": 1,\n   \"packets_lost\": 1,\n   \"highest_seq_no\": 22704,\n   \"ia_jitter\": 78,\n   \"lsr\": 0,\n   \"dlsr\": 0\n  }\n ],\n \"report_count\": 1\n}",
	"host": "127.0.0.1",
	"udp_port": "9060",
	"@timestamp": "2022-03-31T14:51:06.160Z",
	"@version": "1"
}

So it seems that paStash is converting the message and encoding it in a JSON format.

I expect paStash to be relaying/forwarding the HEP UDP contents untouched and relay it on to the destination Homer server IP.

Amazon MSK SASL/SCRAM auth method support

Today, Pastash is using the following Kafka client (https://www.npmjs.com/package/kafka-node/v/3.0.1), which supports only SASL/PLAIN auth method.
This is limiting Pastash usage with Amazon MSK Kafka service, which uses SASL/SCRAM auth method.

Please implement a new Kafka plugin in Pastash based on (https://kafka.js.org/), which supports SASL/SCRAM.

SDP issues and etc with Audiocodes filter

I would like to open this issue to discuss some known bugs/problems.

@spady7 could you please share with me logs, so I can recheck why SDP for you is not getting parsed. I will need logs for such kind of INVITE? I'm interested in one line what starts with reassembled line and has that INVITE what you see in homer without SDP, or you can share full file. If you want you can try to fix it by yourself, I think if you will update filter_app_audiocodes.js line 119 to following:

regex = /(.*)---- Incoming SIP Message from (.*) to SIPInterface #[0-99] $(.*)$ (.*) TO.*--- #012(.*)(.*)/g;

And line 156 to following:

regex = /(.*)---- Outgoing SIP Message to (.*) from SIPInterface #[0-99] $(.*)$ (.*) TO.*--- #012(.*)(.*)/g;

It will do the trick, not sure how this is correct. :) Maybe @lmangani will be against it.

Basically old regex saves to 5th group everything before #012 #012, but SDP are exactly after #012 #012

I'm trying to use in pastash filter clone from logstash, so something like this I tried:

input {
  udp {
    host => 0.0.0.0
    port => 10514
    type => syslog
    tags => ["10514"]
  }
  udp {
    host => 0.0.0.0
    port => 10515
    type => syslog
    tags => ["10515"]
  }
}

filter {
clone {
    add_tag => [ "copy" ]
  }
  if "10514" in [tags] {
  app_audiocodes {
    debug => true
    autolocal => true
    qos => true
    localip => 10.43.56.13
  }
  }
  if "10515" in [tags] {
  app_audiocodes {
    debug => true
    autolocal => true
    qos => true
    localip => 10.43.156.157
  }
  }
}

output {
  if "copy" not in [tags] {
    if [rcinfo] != 'undefined' {
        hep {
          host => '10.10.151.163'
          port => 9060
          hep_id => 100
          hep_type => 1
        }
    }
  } else {
    stdout {}
 }
}

Idea behind this is to send everything to Homer and to save all logs unchanged additionally somewhere so we can dig those logs if something get lost, because this is still quite experimental. I plan to save this to graylog, so sending out to syslog/graylog will be ideal, nevertheless stdout will work too, because I run pastash in docker and I can use docker driver for sending out logs to graylog. @lmangani do you know why clone do not works? Any betters ideas how to mirror completely everything to graylog?

Sometimes I see in Homer IP 10.2.0.4, but not one what I configured with localip. Here some logs:

DEBUG <133>[S=432471] [SID=4bc9e3:2:11204]  SIP/2.0 404 Not Found
FROM: +123456<sip:[email protected]>;tag=1c352922656
TO: <sip:[email protected]>
CSEQ: 1 INVITE
CALL-ID: [email protected]
VIA: SIP/2.0/TLS stams1.myappapp.net:5061;branch=z9hG4bKac871490062
REASON: Q.850;cause=1;text="cf264671-c7af-43cc-acc4-579a0589fe10;RNL"
CONTENT-LENGTH: 0
ALLOW: INVITE,ACK,OPTIONS,CANCEL,BYE,NOTIFY
SERVER: Microsoft.PSTNHub.SIPProxy v.2021.1.15.7 i.USEA.1

 [Time:21-01@15:47:23.269]
reassembled line <133>[S=432470] [SID=4bc9e3:2:11204]  (N   361291) ---- Incoming SIP Message from 52.114.132.46:5061 to SIPInterface #0 (SIPInterface_0) TLS TO(#2005) SocketID(1) ----  SIP/2.0 404 Not Found #012FROM: +123456<sip:[email protected]>;tag=1c352922656 #012TO: <sip:[email protected]> #012CSEQ: 1 INVITE #012CALL-ID: [email protected] #012VIA: SIP/2.0/TLS stams1.myappapp.net:5061;branch=z9hG4bKac871490062 #012REASON: Q.850;cause=1;text="cf264671-c7af-43cc-acc4-579a0589fe10;RNL" #012CONTENT-LENGTH: 0 #012ALLOW: INVITE,ACK,OPTIONS,CANCEL,BYE,NOTIFY #012SERVER: Microsoft.PSTNHub.SIPProxy v.2021.1.15.7 i.USEA.1 #012 #012
[Thu, 21 Jan 2021 15:47:22 GMT] ERROR SESSION SID 11204
[STDOUT] {
  "payload": "SIP/2.0 404 Not Found \r\nFROM: +123456<sip:[email protected]>;tag=1c352922656 \r\nTO: <sip:[email protected]> \r\nCSEQ: 1 INVITE \r\nCALL-ID: [email protected] \r\nVIA: SIP/2.0/TLS stams1.myappapp.net:5061;branch=z9hG4bKac871490062 \r\nREASON: Q.850;cause=1;text=\"cf264671-c7af-43cc-acc4-579a0589fe10;RNL\" \r\nCONTENT-LENGTH: 0 \r\nALLOW: INVITE,ACK,OPTIONS,CANCEL,BYE,NOTIFY \r\nSERVER: Microsoft.PSTNHub.SIPProxy v.2021.1.15.7 i.USEA.1\r\n\r\n",
  "rcinfo": {
    "type": "HEP",
    "version": 3,
    "payload_type": "SIP",
    "ip_family": 2,
    "protocol": 6,
    "proto_type": 1,
    "correlation_id": "[email protected]",
    "srcIp": "52.114.132.46",
    "srcPort": "5061",
    "dstIp": "10.2.0.4",
    "dstPort": "5061",
    "time_sec": 1611244042,
    "time_usec": 696,
    "captureId": "100",
    "capturePass": "MyHep"
  }
}
DEBUG <133>[S=432473] [SID=4bc9e3:2:11204]  (N   361293) ---- Outgoing SIP Message to 52.114.132.46:5061 from SIPInterface #0 (SIPInterface_0) TLS TO(#2005) SocketID(1) ---- [Time:21-01@15:47:23.269]
[Thu, 21 Jan 2021 15:47:22 GMT] ERROR SESSION SID 11204
[Thu, 21 Jan 2021 15:47:22 GMT] ERROR failed parsing Outgoing SIP. Cache on!
DEBUG <133>[S=432472] [SID=4bc9e3:2:11204]  (N   361292) AcSIPCall(#2987): Handling 404 in state Proceeding [Time:21-01@15:47:23.269]
reassembled line <133>[S=432473] [SID=4bc9e3:2:11204]  (N   361293) ---- Outgoing SIP Message to 52.114.132.46:5061 from SIPInterface #0 (SIPInterface_0) TLS TO(#2005) SocketID(1) ----  (N   361292) AcSIPCall(#2987): Handling 404 in state Proceeding
[Thu, 21 Jan 2021 15:47:22 GMT] ERROR SESSION SID 11204
[STDOUT] {
  "payload": "(N   361292) AcSIPCall(#2987): Handling 404 in state Proceeding\r\n\r\n",
  "rcinfo": {
    "type": "HEP",
    "version": 3,
    "payload_type": "SIP",
    "ip_family": 2,
    "protocol": 17,
    "proto_type": 1,
    "correlation_id": "11204",
    "srcIp": "10.2.0.4",
    "srcPort": "5061",
    "dstIp": "52.114.132.46",
    "dstPort": "5061",
    "time_sec": 1611244042,
    "time_usec": 697,
    "captureId": "100",
    "capturePass": "MyHep"
  }
}
DEBUG <133>[S=432474] [SID=4bc9e3:2:11204]  ACK sip:[email protected] SIP/2.0
Via: SIP/2.0/TLS stams1.myappapp.net:5061;alias;branch=z9hG4bKac871490062
Max-Forwards: 70
From: 123456 <sip:[email protected]>;tag=1c352922656
To: <sip:[email protected]>
Call-ID: [email protected]
CSeq: 1 ACK
Contact: <sip:stams1.myappapp.net:5061;transport=tls>
User-Agent: Mediant SW/v.7.20A.258.271
Content-Length: 0

 [Time:21-01@15:47:23.269]
[Thu, 21 Jan 2021 15:47:22 GMT] ERROR SESSION SID 11204

Sometimes I see this "Java error logs" and this do not seems right:

reassembled line <133>[S=1858983] [SID=8999b9:10:48575]  (N  1549767) ---- Incoming SIP Message from 52.114.132.46:6152 to SIPInterface #0 (SIPInterface_0) TLS TO(#1954) SocketID(52) ----  (N  1549766) (#404)Route found (0), Route by Address, IP Group 2 -> 1 (Teams -> myapp), Url:internal:0;
[Thu, 21 Jan 2021 15:47:44 GMT] ERROR SESSION SID 48575
[Thu, 21 Jan 2021 15:47:44 GMT] ERROR Conditional error Error: In / not in right args must be an array : undefined
    at Object.exports.compute (/usr/local/lib/node_modules/@pastash/pastash/lib/lib/condition_evaluator.js:89:13)
    at FilterAppAudiocodes.BaseComponent.processMessage (/usr/local/lib/node_modules/@pastash/pastash/lib/lib/base_component.js:260:33)
    at FilterAppAudiocodes.<anonymous> (/usr/local/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:20:16)
    at FilterAppAudiocodes.emit (events.js:198:13)
    at FilterAppAudiocodes.<anonymous> (/usr/local/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
    at FilterAppAudiocodes.emit (events.js:198:13)
    at FilterAppAudiocodes.<anonymous> (/usr/local/lib/node_modules/@pastash/pastash/lib/lib/base_filter.js:27:18)
    at FilterAppAudiocodes.emit (events.js:198:13)
    at FilterAddVersion.<anonymous> (/usr/local/lib/node_modules/@pastash/pastash/lib/agent.js:260:14)
    at FilterAddVersion.emit (events.js:198:13)

Filter for Cisco CUBE

I can see various filters have been written for other systems, Avaya SM, Sonos etc. Would there be any appetite to write one for syslog SIP debug output from a Cisco CUBE?

I raised the question on the Homer Google group and was suggested to post the question here.

Thanks

trying to use sqlite filter

hi,
i am trying to create a DB that compares incoming IP from netflow and compare it with the customer name on the DB

i instilled the filter :
npm install @pastash/filter_sqlite

and when i tried to run the command with the filter i got an error related to the filter module missing:
log1.txt

this is my conf:
conf.txt

any suggestions?

Add TCP Support for output Plugin `HEP`

pastash can only send hep over UDP. Can we please add support for TCP?

loki output multitenancy (X-Scope-OrgID)

hello.

i believe requests to the Loki API should include an HTTP header (X-Scope-OrgID) identifying the tenant for the request.

https://github.com/grafana/loki/blob/master/docs/operations/multi-tenancy.md

Journald Log support

Hi,

is the a way to get the journald logs using paStash? I have not found any input plugin for it and search on Journal(d) did not provide any results either.

Filter installation fails when running npm as root user

Please add the following command to the wiki page for HSP/CDR generation:

npm -g config set user root

See log snippet below for an example of such an issue when attempting to install @pastash/filter_app_hsp:

=======

root@poc-cdc-hepic:/usr/lib# npm -g install @pastash/filter_app_hsp

[email protected] install /usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3
node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using needle for node-pre-gyp https download
node-pre-gyp WARN Pre-built binaries not installable for [email protected] and [email protected] (node-v72 ABI, glibc) (falling back to source compile with node-gyp)
node-pre-gyp WARN Hit error EACCES: permission denied, mkdir '/usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3/lib/binding'
gyp WARN EACCES current user ("nobody") does not have permission to access the dev dir "/root/.cache/node-gyp/12.20.0"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3/.node-gyp"
gyp WARN install got an error, rolling back install
gyp WARN install got an error, rolling back install
gyp ERR! configure error
gyp ERR! stack Error: EACCES: permission denied, mkdir '/usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3/.node-gyp'
gyp ERR! System Linux 4.19.0-12-amd64
gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3/lib/binding/node-v72-linux-x64/node_sqlite3.node" "--module_name=node_sqlite3" "--module_path=/usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3/lib/binding/node-v72-linux-x64" "--napi_version=7" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v72"
gyp ERR! cwd /usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3
gyp ERR! node -v v12.20.0
gyp ERR! node-gyp -v v5.1.0
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute '/usr/bin/node /usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3/lib/binding/node-v72-linux-x64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3/lib/binding/node-v72-linux-x64 --napi_version=7 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v72' (1)
node-pre-gyp ERR! stack at ChildProcess. (/usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack at ChildProcess.emit (events.js:314:20)
node-pre-gyp ERR! stack at maybeClose (internal/child_process.js:1022:16)
node-pre-gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
node-pre-gyp ERR! System Linux 4.19.0-12-amd64
node-pre-gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3
node-pre-gyp ERR! node -v v12.20.0
node-pre-gyp ERR! node-pre-gyp -v v0.11.0
node-pre-gyp ERR! not ok
Failed to execute '/usr/bin/node /usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3/lib/binding/node-v72-linux-x64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/usr/lib/node_modules/@pastash/filter_app_hsp/node_modules/sqlite3/lib/binding/node-v72-linux-x64 --napi_version=7 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v72' (1)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: node-pre-gyp install --fallback-to-build
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2020-12-02T20_08_40_035Z-debug.log

Cisco UCM CDR filter

Hi,
I want to heplify Cisco UCM CDRs (currently version 12.5, but afaik the structure didn't change in the last years).
The plain-text-files have the following structure:

column-name | type | example

app_audiocodes Error: Cannot find module 'filters/filter_app_audiocodes'

Hi,
I have followed the documentation and installed the app_audiocodes filter. However, when I try to run the paStash with the recommended config file it does not work. In debug mode paStash displays the following error:

/opt/pastash$ sudo pastash --config_file=/opt/pastash/audiocodes.conf --log_level debug
[Sat, 04 May 2024 15:43:23 GMT] INFO Changing log_level debug
[Sat, 04 May 2024 15:43:23 GMT] NOTICE Starting pastash 1.0.80
[Sat, 04 May 2024 15:43:23 GMT] INFO Max http socket 100
[Sat, 04 May 2024 15:43:23 GMT] INFO Loading config file : /opt/pastash/audiocodes.conf
[Sat, 04 May 2024 15:43:23 GMT] INFO File loaded, 4 urls found
[Sat, 04 May 2024 15:43:23 GMT] INFO Loading config : 7 urls
[Sat, 04 May 2024 15:43:23 GMT] DEBUG Loading urls [
'filter://add_host://',
'filter://add_timestamp://',
'filter://add_version://',
'input://udp://?host=0.0.0.0&port=514&type=syslog',
'filter://app_audiocodes://?version=7.20A.256.396&debug=false&autolocal=true&ini=%2Ftmp%2Faudiocodes.ini',
'output://stdout://',
'output://hep://?host=127.0.0.1&port=9060&hep_id=2222&dynamic_eval=%7B%22false_clauses%22%3A%5B%5D%2C%22true_clause%22%3A%7B%22op%22%3A%22!%3D%22%2C%22left%22%3A%7B%22field%22%3A%22rcinfo%22%7D%2C%22right%22%3A%7B%22value%22%3A%22undefined%22%7D%7D%7D'
]
[Sat, 04 May 2024 15:43:23 GMT] DEBUG Initializing module output
[Sat, 04 May 2024 15:43:23 GMT] INFO Initializing output Stdout
[Sat, 04 May 2024 15:43:23 GMT] DEBUG Initializing module output
[Sat, 04 May 2024 15:43:23 GMT] INFO Initializing output HEP/EEP Server
[Sat, 04 May 2024 15:43:23 GMT] INFO Start output to HEP udp to 127.0.0.1:9060
[Sat, 04 May 2024 15:43:23 GMT] DEBUG Initializing module filter
[Sat, 04 May 2024 15:43:23 GMT] INFO Initializing filter AddHost
[Sat, 04 May 2024 15:43:23 GMT] DEBUG Initializing module filter
[Sat, 04 May 2024 15:43:23 GMT] INFO Initializing filter AddTimestamp
[Sat, 04 May 2024 15:43:23 GMT] DEBUG Initializing module filter
[Sat, 04 May 2024 15:43:23 GMT] INFO Initializing filter AddVersion
[Sat, 04 May 2024 15:43:23 GMT] DEBUG Initializing module filter
[Sat, 04 May 2024 15:43:23 GMT] DEBUG no local module found filters filter_app_audiocodes Error: Cannot find module 'filters/filter_app_audiocodes'
Require stack:

/usr/local/lib/node_modules/@pastash/pastash/lib/agent.js
/usr/local/lib/node_modules/@pastash/pastash/bin/pastash
at Module._resolveFilename (node:internal/modules/cjs/loader:1039:15)
at Module._load (node:internal/modules/cjs/loader:885:27)
at Module.require (node:internal/modules/cjs/loader:1105:19)
at require (node:internal/modules/cjs/helpers:103:18)
at LogstashAgent.configure (/usr/local/lib/node_modules/@pastash/pastash/lib/agent.js:133:17)
at LogstashAgent. (/usr/local/lib/node_modules/@pastash/pastash/lib/agent.js:215:10)
at /usr/local/lib/node_modules/@pastash/pastash/node_modules/async/dist/async.js:3113:16
at replenish (/usr/local/lib/node_modules/@pastash/pastash/node_modules/async/dist/async.js:1014:17)
at /usr/local/lib/node_modules/@pastash/pastash/node_modules/async/dist/async.js:1019:9
at eachLimit$1 (/usr/local/lib/node_modules/@pastash/pastash/node_modules/async/dist/async.js:3199:24) {
code: 'MODULE_NOT_FOUND',
requireStack: [
'/usr/local/lib/node_modules/@pastash/pastash/lib/agent.js',
'/usr/local/lib/node_modules/@pastash/pastash/bin/pastash'
]
}
Loading npm module... @pastash/filter_app_audiocodes
[Sat, 04 May 2024 15:43:23 GMT] INFO Initializing filter AppAudiocodes
[Sat, 04 May 2024 15:43:23 GMT] INFO Initialized App Audiocodes SysLog to SIP/HEP parser
[Sat, 04 May 2024 15:43:23 GMT] INFO Reading INI file to resolver... /tmp/audiocodes.ini
[Sat, 04 May 2024 15:43:23 GMT] INFO INI Loaded undefined Interfaces
[Sat, 04 May 2024 15:43:23 GMT] INFO INI Loaded undefined SIP Profiles
[Sat, 04 May 2024 15:43:23 GMT] DEBUG Initializing module input
[Sat, 04 May 2024 15:43:23 GMT] INFO Initializing input Udp
[Sat, 04 May 2024 15:43:23 GMT] INFO Start listening on udp 0.0.0.0:514
[Sat, 04 May 2024 15:43:23 GMT] INFO Config loaded.
^C[Sat, 04 May 2024 15:46:25 GMT] INFO SIGINT received.
[Sat, 04 May 2024 15:46:25 GMT] INFO Closing listening udp 0.0.0.0:514
[Sat, 04 May 2024 15:46:25 GMT] INFO Closing stdout
[Sat, 04 May 2024 15:46:25 GMT] INFO Closing output to HEP udp to 127.0.0.1:9060
[Sat, 04 May 2024 15:46:25 GMT] INFO Closing agent
[Sat, 04 May 2024 15:46:25 GMT] INFO Quitting.

I use the following config file:

input {
udp {
host => 0.0.0.0
port => 514
type => syslog
}
}

filter {
app_audiocodes{
#version => '7.40A.100.114'
version => '7.20A.256.396'
debug => false
autolocal => true
ini => '/tmp/audiocodes.ini'
}
}

output {
stdout{}
if [rcinfo] != 'undefined' {
hep {
host => '127.0.0.1'
port => 9060
hep_id => 2222
}
}
}

The INI file is in the /tmp folder.
Here is my npm and node version:
node -v
v18.13.0
npm -v
9.2.0
myrcin@ZabbixSRV:/opt/pastash$

Also during the installation of the app_audiocodes I got the following warnings:

What am I doing wrong? Even though, paStash seems to be running, it does not send any data. Any help is highly appreciated.

syslog input filtering not working [podman]

Hello and thank you so much for this amazing tool.

I have brought up PaStash in a container alongside cLoki and Grafana containers, problem is, even though the hostnames can be seen in the log messages and the host label is working, all of the host label values get changed to the container's IP address.

Here is my recipe:

input {
  tcp {
    host => 0.0.0.0
    port => 514
    type => syslog
  }
}

filter {
  if [type] == syslog {
    regex {
      builtin_regex => syslog
    }
    syslog_pri {}
  }
}

output {
  loki {
    host => localhost
    port => 3100
    path => "/loki/api/v1/push"
  }
}

Output: (For all hosts regardless of their name)

In addition, I'd appreciate some guidance on how can I create more labels and/or group some log messages for instance if I have several services related to DHCP, I want to have a label called dhcp that whenever I select it I can see all log messages related to all services under dhcp as well as seeing them individually via the syslog_program label.

I'm almost certain that this is not an actual issue but in fact, I'm just a newbie and do not know how to properly use this amazing tool, hence I'm asking for some guidance, examples, and suggestions.

Make HEP network setting configurable

I am trying to send freeswitch CDRs as HEP Log to Homer. But I am unable to set HEP SRC IP and SRC Port with respect to IPs in the CDRs so That I can see Log Correclty in the flow.

Pastash Configuration

input {
      file {
        path => "/home/hrhashmi/FREESWITCH/var/log/freeswitch/cdr-csv/Master.csv"
      }
    }
 filter {
   csv {
    headers => ['caller_id_name', 'caller_id_number', 'source_ip', 'source_port', 'destination_number', 'context', 'start_stamp', 'answer_stamp', 'end_stamp', 'duration', 'billsec', 'hangup_cause', 'uuid', 'bleg_uuid', 'accountcode','codec','bleg_codec']
 }
  compute_field {
    field => correlation_id
    value => "#{uuid}"
  }
  compute_field {
    field => s_ip
    value => "#{source_ip}"
  }
  compute_field {
    field => s_port
    value => "#{source_port}"
  }
}
    output {
      hep {
        host => debian10.hbvoice.local
        port => 9063
        hep_id => 2022
        hep_type => 100
        hep_cid => '#{correlation_id}'
        hep_protocol => 17
        src_ip => '#{s_ip}'
        src_port => '#{s_port}'
        dst_ip => 192.168.0.167
      }
      stdout{}
    }

Pasttash Logs (it does not print hep packet.)


[Wed, 08 May 2024 11:37:05 GMT] DEBUG Event received for /home/hrhashmi/FREESWITCH/var/log/freeswitch/cdr-csv/Master.csv : change Master.csv
[Wed, 08 May 2024 11:37:05 GMT] DEBUG File /home/hrhashmi/FREESWITCH/var/log/freeswitch/cdr-csv/Master.csv changed
[Wed, 08 May 2024 11:37:05 GMT] DEBUG Launch reading on 20 , current_index 2832072
[Wed, 08 May 2024 11:37:05 GMT] DEBUG Read from 20 : 212 bytes
[Wed, 08 May 2024 11:37:05 GMT] DEBUG CSV Data! Row {
  caller_id_name: 'Hamid R. Hashmi',
  caller_id_number: '+923009682285',
  source_ip: '192.168.0.110',
  source_port: '64874',
  destination_number: '+97123456789',
  context: 'public',
  start_stamp: '2024-05-08 16:37:01',
  answer_stamp: '',
  end_stamp: '2024-05-08 16:37:05',
  duration: '4',
  billsec: '0',
  hangup_cause: 'ORIGINATOR_CANCEL',
  uuid: '0d51a83875da4ea38dcbc1328191ddbe',
  bleg_uuid: '',
  accountcode: '',
  codec: 'PCMA',
  bleg_codec: 'PCMA'
}
[STDOUT] {
  "caller_id_name": "Hamid R. Hashmi",
  "caller_id_number": "+923009682285",
  "source_ip": "192.168.0.110",
  "source_port": "64874",
  "destination_number": "+97123456789",
  "context": "public",
  "start_stamp": "2024-05-08 16:37:01",
  "answer_stamp": "",
  "end_stamp": "2024-05-08 16:37:05",
  "duration": "4",
  "billsec": "0",
  "hangup_cause": "ORIGINATOR_CANCEL",
  "uuid": "0d51a83875da4ea38dcbc1328191ddbe",
  "bleg_uuid": "",
  "accountcode": "",
  "codec": "PCMA",
  "bleg_codec": "PCMA",
  "correlation_id": "0d51a83875da4ea38dcbc1328191ddbe",
  "s_ip": "192.168.0.110",
  "s_port": "64874"
}

I can see a comment here which says that the Network part is to be Done.

Issue with Janus Events in Loki

Hello,

I am using the qxip/pastash-loki docker image and the configuration described in https://github.com/sipcapture/paStash/wiki/Example:-Janus-Events to store Janus Events into Loki.

However, it seems that the Loki labels created when I store the data are not the same with the ones shown in examples.

Attached you can find a screenshot of the labels created in my system. Useful labels such as emitter, handle_id, opaque_id, etc., are not created. How can I solve this issue?

Thank you in advance,
George

any performance report?

no Audiocodes SBC syslog seen

Hi,

I using the audiocodes pastash plugin, but i cant see any Messages in Homer.

Homer is running on CentOS 7.
Installed Homer with the standard install script.
Audiocodes VE on VM Ware with firmware 7.20A.258.119

Hier is my pastash config file

nput {
udp {
host => 0.0.0.0
port => 514
type => syslog
}
}

filter {
app_audiocodes{
debug => true
autolocal => true
version => '7.20A.258.119'
}
}

output {
stdout {}
if [rcinfo] != 'undefined' {
hep {
host => '127.0.0.1'
port => 9060
hep_id => 2222
}
}
}

Wenn i running this command pastash --config_file=/home/centos/Audiocodes/pastash_audiocodes.conf --log_level debug i becom the folloing Messages:
pastash --config_file=/home/centos/Audiocodes/pastash_audiocodes.conf --log_level debug
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Changing log_level debug
[Fri, 26 Nov 2021 10:05:01 GMT] NOTICE Starting pastash 1.0.75
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Max http socket 100
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Loading config file : /home/centos/Audiocodes/pastash_audiocodes.conf
[Fri, 26 Nov 2021 10:05:01 GMT] INFO File loaded, 3 urls found
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Loading config : 6 urls
[Fri, 26 Nov 2021 10:05:01 GMT] DEBUG Loading urls [
'filter://add_host://',
'filter://add_timestamp://',
'filter://add_version://',
'input://udp://?host=0.0.0.0&port=514&type=syslog',
'filter://app_audiocodes://',
'output://hep://?host=127.0.0.1&port=9060&hep_id=2222&dynamic_eval=%7B%22false_clauses%22%3A%5B%5D%2C%22true_clause%22%3A%7B%22op%22%3A%22!%3D%22%2C%22left%22%3A%7B%22field%22%3A%22rcinfo%22%7D%2C%22right%22%3A%7B%22value%22%3A%22undefined%22%7D%7D%7D'
]
[Fri, 26 Nov 2021 10:05:01 GMT] DEBUG Initializing module output
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Initializing output HEP/EEP Server
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Start output to HEP udp to 127.0.0.1:9060
[Fri, 26 Nov 2021 10:05:01 GMT] DEBUG Initializing module filter
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Initializing filter AddHost
[Fri, 26 Nov 2021 10:05:01 GMT] DEBUG Initializing module filter
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Initializing filter AddTimestamp
[Fri, 26 Nov 2021 10:05:01 GMT] DEBUG Initializing module filter
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Initializing filter AddVersion
[Fri, 26 Nov 2021 10:05:01 GMT] DEBUG Initializing module filter
[Fri, 26 Nov 2021 10:05:01 GMT] DEBUG no local module found filters filter_app_audiocodes Error: Cannot find module 'filters/filter_app_audiocodes'
Require stack:

/usr/lib/node_modules/@pastash/pastash/lib/agent.js
/usr/lib/node_modules/@pastash/pastash/bin/pastash
at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
at Function.Module._load (node:internal/modules/cjs/loader:778:27)
at Module.require (node:internal/modules/cjs/loader:1005:19)
at require (node:internal/modules/cjs/helpers:102:18)
at LogstashAgent.configure (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:133:17)
at LogstashAgent. (/usr/lib/node_modules/@pastash/pastash/lib/agent.js:215:10)
at /usr/lib/node_modules/@pastash/pastash/node_modules/async/dist/async.js:3096:16
at replenish (/usr/lib/node_modules/@pastash/pastash/node_modules/async/dist/async.js:998:17)
at iterateeCallback (/usr/lib/node_modules/@pastash/pastash/node_modules/async/dist/async.js:983:17)
at /usr/lib/node_modules/@pastash/pastash/node_modules/async/dist/async.js:958:16 {
code: 'MODULE_NOT_FOUND',
requireStack: [
'/usr/lib/node_modules/@pastash/pastash/lib/agent.js',
'/usr/lib/node_modules/@pastash/pastash/bin/pastash'
]
}
Loading npm module... @pastash/filter_app_audiocodes
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Initializing filter AppAudiocodes
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Initialized App Audiocodes SysLog to SIP/HEP parser
[Fri, 26 Nov 2021 10:05:01 GMT] DEBUG Initializing module input
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Initializing input Udp
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Start listening on udp 0.0.0.0:514
[Fri, 26 Nov 2021 10:05:01 GMT] INFO Config loaded.
[Fri, 26 Nov 2021 10:05:02 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:02 GMT] ERROR failed parsing Outgoing SIP. Cache on!
[Fri, 26 Nov 2021 10:05:03 GMT] ERROR failed parsing Outgoing SIP. Cache on!
[Fri, 26 Nov 2021 10:05:03 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:06 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:06 GMT] ERROR failed parsing Outgoing SIP. Cache on!
[Fri, 26 Nov 2021 10:05:06 GMT] ERROR failed parsing Outgoing SIP. Cache on!
[Fri, 26 Nov 2021 10:05:06 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:06 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:06 GMT] ERROR failed parsing Outgoing SIP. Cache on!
[Fri, 26 Nov 2021 10:05:06 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:07 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:07 GMT] ERROR failed parsing Outgoing SIP. Cache on!
[Fri, 26 Nov 2021 10:05:13 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:13 GMT] ERROR failed parsing Outgoing SIP. Cache on!
[Fri, 26 Nov 2021 10:05:15 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:15 GMT] ERROR failed parsing Outgoing SIP. Cache on!
[Fri, 26 Nov 2021 10:05:15 GMT] ERROR failed parsing Outgoing SIP. Cache on!
[Fri, 26 Nov 2021 10:05:15 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:15 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:15 GMT] INFO !!!!!!!!!!!!!! DEBUG MEDIA [
'<142>[S=61]',
'MEDIA_END',
'[email protected]',
'9f41ae:50:41179',
'AUDIO',
'g711Alaw64k',
'20',
'192.168.12.245',
'7068',
'195.185.37.60',
'32112',
'362',
'363',
'3913228979',
'-1',
'127',
'127',
'127',
'127',
'46',
'NO_TRANSCODING'
] 21
[Fri, 26 Nov 2021 10:05:15 GMT] ERROR missing media parameters [
'<142>[S=61]',
'MEDIA_END',
'[email protected]',
'9f41ae:50:41179',
'AUDIO',
'g711Alaw64k',
'20',
'192.168.12.245',
'7068',
'195.185.37.60',
'32112',
'362',
'363',
'3913228979',
'-1',
'127',
'127',
'127',
'127',
'46',
'NO_TRANSCODING'
]
[Fri, 26 Nov 2021 10:05:15 GMT] INFO !!!!!!!!!!!!!! DEBUG MEDIA [
'<142>[S=63]',
'MEDIA_END',
'[email protected]',
'9f41ae:50:41179',
'AUDIO',
'g711Alaw64k',
'20',
'192.168.12.245',
'6076',
'192.168.12.111',
'32514',
'363',
'362',
'2913200721',
'1227887156',
'127',
'127',
'127',
'127',
'46',
'NO_TRANSCODING'
] 21
[Fri, 26 Nov 2021 10:05:15 GMT] ERROR missing media parameters [
'<142>[S=63]',
'MEDIA_END',
'[email protected]',
'9f41ae:50:41179',
'AUDIO',
'g711Alaw64k',
'20',
'192.168.12.245',
'6076',
'192.168.12.111',
'32514',
'363',
'362',
'2913200721',
'1227887156',
'127',
'127',
'127',
'127',
'46',
'NO_TRANSCODING'
]
[Fri, 26 Nov 2021 10:05:15 GMT] ERROR failed parsing Incoming SIP. Cache on!
[Fri, 26 Nov 2021 10:05:15 GMT] ERROR failed parsing Outgoing SIP. Cache on!
^C[Fri, 26 Nov 2021 10:05:21 GMT] INFO SIGINT received.
[Fri, 26 Nov 2021 10:05:21 GMT] INFO Closing listening udp 0.0.0.0:514
[Fri, 26 Nov 2021 10:05:21 GMT] INFO Closing output to HEP udp to 127.0.0.1:9060
[Fri, 26 Nov 2021 10:05:21 GMT] INFO Closing agent
[Fri, 26 Nov 2021 10:05:21 GMT] INFO Quitting.
[root@centtos7 centos]# pastash --config_file=/home/centos/Audiocodes/pastash_audiocodes.conf --log_level debug
[Fri, 26 Nov 2021 10:15:31 GMT] INFO Changing log_level debug
[Fri, 26 Nov 2021 10:15:31 GMT] NOTICE Starting pastash 1.0.75
[Fri, 26 Nov 2021 10:15:31 GMT] INFO Max http socket 100
[Fri, 26 Nov 2021 10:15:31 GMT] INFO Loading config file : /home/centos/Audiocodes/pastash_audiocodes.conf
[Fri, 26 Nov 2021 10:15:31 GMT] ERROR Unable to load config file /home/centos/Audiocodes/pastash_audiocodes.conf
[Fri, 26 Nov 2021 10:15:31 GMT] ERROR Error: Processing error for file /home/centos/Audiocodes/pastash_audiocodes.conf : Error: Parse error on line 13:
...on => 7.20A.258.119}}output { if [
----------------------^
Expecting 'SET', got 'STOP'
at /usr/lib/node_modules/@pastash/pastash/lib/lib/file_loader.js:42:16
at FSReqCallback.readFileAfterClose [as oncomplete] (node:internal/fs/read_file_context:68:3)
[root@centtos7 centos]# pastash --config_file=/home/centos/Audiocodes/pastash_audiocodes.conf
[Fri, 26 Nov 2021 10:15:43 GMT] NOTICE Starting pastash 1.0.75
[Fri, 26 Nov 2021 10:15:43 GMT] INFO Max http socket 100
[Fri, 26 Nov 2021 10:15:43 GMT] INFO Loading config file : /home/centos/Audiocodes/pastash_audiocodes.conf
[Fri, 26 Nov 2021 10:15:43 GMT] ERROR Unable to load config file /home/centos/Audiocodes/pastash_audiocodes.conf
[Fri, 26 Nov 2021 10:15:43 GMT] ERROR Error: Processing error for file /home/centos/Audiocodes/pastash_audiocodes.conf : Error: Parse error on line 13:
...on => 7.20A.258.119}}output { if [
----------------------^
Expecting 'SET', got 'STOP'
at /usr/lib/node_modules/@pastash/pastash/lib/lib/file_loader.js:42:16
at FSReqCallback.readFileAfterClose [as oncomplete] (node:internal/fs/read_file_context:68:3)
[root@centtos7 centos]# pastash --config_file=/home/centos/Audiocodes/pastash_audiocodes.conf
[Fri, 26 Nov 2021 10:18:02 GMT] NOTICE Starting pastash 1.0.75
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Max http socket 100
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Loading config file : /home/centos/Audiocodes/pastash_audiocodes.conf
[Fri, 26 Nov 2021 10:18:02 GMT] INFO File loaded, 3 urls found
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Loading config : 6 urls
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Initializing output Stdout
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Initializing output HEP/EEP Server
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Start output to HEP udp to 127.0.0.1:9060
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Initializing filter AddHost
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Initializing filter AddTimestamp
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Initializing filter AddVersion
Loading npm module... @pastash/filter_app_audiocodes
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Initializing filter AppAudiocodes
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Initialized App Audiocodes SysLog to SIP/HEP parser
[Fri, 26 Nov 2021 10:18:02 GMT] INFO Config loaded.
^C[Fri, 26 Nov 2021 10:18:59 GMT] INFO SIGINT received.
[Fri, 26 Nov 2021 10:18:59 GMT] INFO Closing stdout
[Fri, 26 Nov 2021 10:18:59 GMT] INFO Closing output to HEP udp to 127.0.0.1:9060
[Fri, 26 Nov 2021 10:18:59 GMT] INFO Closing agent
[Fri, 26 Nov 2021 10:18:59 GMT] INFO Quitting.

Syslog via TCP not synonymous to syslog via UDP

When I try to use TCP (not TLS yet) to deliver syslog to a paStash node, it just barfs all over the place, and cannot re-assemble things very well at all.

When I switch over to UDP - everything plays nicely, regexp also. So all things being equal, TCP doesn't play nicely at the receiving end. I enable stdout and just see a mish-mash of bad re-assembly.

It boils down to the multi-line filter.

UDP: multi-line splits cleanly where it should, based on my 'start-lines'.
TCP: multi-line trips up all over the place.

If we want to try to blame my device for doing something wrong on TCP, we would be pointing the finger at Rsyslogd, which the device uses. So we're up against a few decades of battle-tested code, which, as I see the packets and data upon closer inspection, look fully compliant.

I'm also a bit surprised to see things like trim() this and this in both UDP and TCP (which, to me, is either a bug or bad assumptions, but this is an aside). This now needs to be worked around in every 'app'. Especially since syslog rows can contain indented space:

0000  3c 33 30 3e 4a 75 6c 20  31 38 20 31 37 3a 33 30   <30>Jul  18 17:30
0010  3a 31 31 20 6e 65 74 6c  6f 67 67 65 72 3a 20 20   :11 netl ogger:  
0020  20 20 20 20 20 20 20 20  20 20 20 20 0a                         .

Here is a slice of where I follow a UDP stream in Wireshark containing the above:

<30>Jul 18 17:30:11 netlogger: Debug: sipfw: StartRegistration: InitiateStandaloneRequest
<30>Jul 18 17:30:11 netlogger: Debug: sipfw: Connection 1 (0x5589cdfbbdb0) reused
<30>Jul 18 17:30:11 netlogger: Debug: sipfw: NEW_TRANSACTION CLIENT 128697980 REGISTER 0x5589cdff4030
<30>Jul 18 17:30:11 netlogger: Info: sipfw: send sf (0x5589cdff4030) to 127.0.0.1:5060 via 127.0.0.1:5060 UDP connection 1:
<30>Jul 18 17:30:11 netlogger:              
<30>Jul 18 17:30:11 netlogger:              REGISTER sip:redacted.com SIP/2.0
<30>Jul 18 17:30:11 netlogger:              Via: SIP/2.0/UDP 127.0.0.1:5060;branch=z9hG4bK0f4d5546961c521e97ebd21da0b5390a
<30>Jul 18 17:30:11 netlogger:              Max-Forwards: 70

Here's my recipe. Ignore the commented out variants - I've been trying different things to no avail.

input {
  udp {
    host => 0.0.0.0
    port => 514
    type => syslog
    }
#  tcp {
#    host => 0.0.0.0
#    port => 514
#    type => syslog_tcp
#    unserializer => raw
#    }

}

filter {
  #recipe for syslog to SIP packet re-assembly
  if [type] == syslog {
    regex {
      regex => /^<(\S+)>(\S+\s+\S+\s+\d+:\d+:\d+)(\s\S+[^:]){1,3}(:.*|:)$/
      fields =>   "syslog_priority,timestamp,syslog_program,message"
      date_format => "MMM DD HH:mm:ss Z"
    }
    syslog_pri {}
   }
  if [type] == syslog_tcp {
     multiline {
        start_line_regex => /\n/
       }
    regex {
#      regex => /<(\S+)>(\S+\s+\S+\s+\d+:\d+:\d+)(\s\S+[^:]){1,2}(:.*|:)\n/
      regex => /^(.*)$/
#      fields =>   "syslog_priority,timestamp,syslog_program,message"
      regex_flags => m
    }
    syslog_pri {}
  }

    multiline {
     start_line_regex => /^.*sipfw:\s+(send|recv).*via.*connection\s\d+:/
     max_delay => 10
     regex_flags => i
    }

    #app to parse Syslog -> SIP
    app_secret {}
#  }
}
output {
# uncomment to observe your results
# stdout {}
 hep {
    host => heplify-server
    port => 9060
    hep_id => 2001
    hep_type => 100
 }
}

Tell me what you need and I'll do my best to provide it.

I'm basically at commit 4917bd7

What gives?

Audiocodes SBC ERROR failed parsing Outgoing SIP. Cache on!

Hi, tried the paStash with Audiocodes SBC thing to get syslogs to Homer, but no luck so far. When I redirect the syslog from Audiocodes Mediant SW SBC to paStash IP, I see the traffic incoming and at the same time massive amount of

ERROR failed parsing Outgoing SIP. Cache on!

as paStash output and nothing going over to the 9060 Homer port. All settings default from the guide. Any idea what that error means?

Thanks for the help!

UDP input/output issue

Hello,
on the WIKI on HEP forwarding https://github.com/sipcapture/paStash/wiki/Example:-HEP-Relay there an example with UDP in and UDP out, also an example with UDP-> AMQP. AMQP -> UDP. I have tried both with similar problems.

I am trying to get UDP-> AMQP. AMQP -> UDP working but I have a feeling the UDP in/out issue will solve the issue with AMQP.

Here is an outline of what I have tried or looked at:

heplify -------UDP. ------>. heplify Server. Data in Homer

Once I put in paStash with UDP in and out based on example nothing shows up in homer

heplify---UDP--> paStatsh --UDP----> heplify Server nothing in Homer

I have replaced paStash with Samplicator. https://github.com/sleinen/samplicator and I get data in homer. Samplicator is a simple UDP packet replicator.

heplify---UDP--> samplicator --UDP----> heplify Server Data in homer... but does not help on moving to AMQP. It just shows the path and firewall rules are good.

Environment: Cent 7 latest. node v14.15.5 also tried v10.x, v15.8.0. same issues.
simple udp in udp out config.

input {
  udp {
    host => 192.168.176.11
    port => 9060
  }
}

output {
  udp {
    host => 192.168.176.8
    port => 9060
  }
}

192.168.176.43 heplify
192.168.176.11. paStash
192.168.176.8 Heplify-server

I see packet flow in and out of paStash. pcap is attached.

The LUA Wireshark dissector
pa-Stash-udp-udp-hep.pcap.zip
does not decode what is going out of paStash(packets sourced from 192.168.176.11 to 192.168.176.8). The packet capture was from the server where paStash running 192.168.176.11

sipcapture / pastash Goto Github PK

pastash's Issues

pastash config:

sqlite config

Recommend Projects

Recommend Topics

Recommend Org