Cisco UCM CDR filter about pastash HOT 18 OPEN

Hi Andre - Are those materialized as individual files?

from pastash.

he file is simple csv, that means the example given looks in csv like this:

1,2,28377,36951964,1593547211,2,36951964,152200202,"+4912345678","",0,16,4,152200202,23618,2,20,0,0,0,0,0,0,"0","0",36951967,2,36951967,152200202,"+4923456789","+4923456789","",0,0,4,152200202,23320,2,20,0,0,0,0,0,0,"0","0",1593547211,1593547987,"+4912345678","dbfc3109-c95c-4a46-96fe-d94d89e20cde","PSTN_OUT","","PSTN_OUT","",776,"CUBE","CUBE",12,17,0,18,0,146,0,"StandAloneCluster",18,"","",0,"",2,2,0,0,64,64,"","","","10.100.10.99","10.100.10.99",0,0,0,0,0,0,0,0,0,0,0,0,1,"216DF07ABA4311EAB627B490D23A7537",1,"4AF15980000100000009151F0212640A",0,0,0,"","",5,"","","","","","","","",0,0,"","","","","","","","",0,0,0,"+!","+!","","","","","fe17729285175238945d2e0c49e9929c","b78e7565083b5488b59e1cdc3ba7f58c"

from pastash.

first line of each csv contains column names, second line column types, line 3 ff contains data.

from pastash.

Hi Andre - Are those materialized as individual files?

no, datasets are grouped by time - in our setup each hour a new file is created containing all CDRs / CMRs for that timeframe (one file for CDRs, one for CMRs).

from pastash.

wait up - i'm confused. if its a CSV file each hour, how can the values be spread on multiple lines? it seems unlikely.
from the example, it seems there must be a fixed header on the first line of the file, and each cdr to be one full line after.

Could you please provide the first 3-4 lines of any such csv file?

from pastash.

sorry for confusion. here the first 4 lines of a cdr file of 2020 june 30th 21:13:

"cdrRecordType","globalCallID_callManagerId","globalCallID_callId","origLegCallIdentifier","dateTimeOrigination","origNodeId","origSpan","origIpAddr","callingPartyNumber","callingPartyUnicodeLoginUserID","origCause_location","origCause_value","origPrecedenceLevel","origMediaTransportAddress_IP","origMediaTransportAddress_Port","origMediaCap_payloadCapability","origMediaCap_maxFramesPerPacket","origMediaCap_g723BitRate","origVideoCap_Codec","origVideoCap_Bandwidth","origVideoCap_Resolution","origVideoTransportAddress_IP","origVideoTransportAddress_Port","origRSVPAudioStat","origRSVPVideoStat","destLegIdentifier","destNodeId","destSpan","destIpAddr","originalCalledPartyNumber","finalCalledPartyNumber","finalCalledPartyUnicodeLoginUserID","destCause_location","destCause_value","destPrecedenceLevel","destMediaTransportAddress_IP","destMediaTransportAddress_Port","destMediaCap_payloadCapability","destMediaCap_maxFramesPerPacket","destMediaCap_g723BitRate","destVideoCap_Codec","destVideoCap_Bandwidth","destVideoCap_Resolution","destVideoTransportAddress_IP","destVideoTransportAddress_Port","destRSVPAudioStat","destRSVPVideoStat","dateTimeConnect","dateTimeDisconnect","lastRedirectDn","pkid","originalCalledPartyNumberPartition","callingPartyNumberPartition","finalCalledPartyNumberPartition","lastRedirectDnPartition","duration","origDeviceName","destDeviceName","origCallTerminationOnBehalfOf","destCallTerminationOnBehalfOf","origCalledPartyRedirectOnBehalfOf","lastRedirectRedirectOnBehalfOf","origCalledPartyRedirectReason","lastRedirectRedirectReason","destConversationId","globalCallId_ClusterID","joinOnBehalfOf","comment","authCodeDescription","authorizationLevel","clientMatterCode","origDTMFMethod","destDTMFMethod","callSecuredStatus","origConversationId","origMediaCap_Bandwidth","destMediaCap_Bandwidth","authorizationCodeValue","outpulsedCallingPartyNumber","outpulsedCalledPartyNumber","origIpv4v6Addr","destIpv4v6Addr","origVideoCap_Codec_Channel2","origVideoCap_Bandwidth_Channel2","origVideoCap_Resolution_Channel2","origVideoTransportAddress_IP_Channel2","origVideoTransportAddress_Port_Channel2","origVideoChannel_Role_Channel2","destVideoCap_Codec_Channel2","destVideoCap_Bandwidth_Channel2","destVideoCap_Resolution_Channel2","destVideoTransportAddress_IP_Channel2","destVideoTransportAddress_Port_Channel2","destVideoChannel_Role_Channel2","IncomingProtocolID","IncomingProtocolCallRef","OutgoingProtocolID","OutgoingProtocolCallRef","currentRoutingReason","origRoutingReason","lastRedirectingRoutingReason","huntPilotPartition","huntPilotDN","calledPartyPatternUsage","IncomingICID","IncomingOrigIOI","IncomingTermIOI","OutgoingICID","OutgoingOrigIOI","OutgoingTermIOI","outpulsedOriginalCalledPartyNumber","outpulsedLastRedirectingNumber","wasCallQueued","totalWaitTimeInQueue","callingPartyNumber_uri","originalCalledPartyNumber_uri","finalCalledPartyNumber_uri","lastRedirectDn_uri","mobileCallingPartyNumber","finalMobileCalledPartyNumber","origMobileDeviceName","destMobileDeviceName","origMobileCallDuration","destMobileCallDuration","mobileCallType","originalCalledPartyPattern","finalCalledPartyPattern","lastRedirectingPartyPattern","huntPilotPattern","origDeviceType","destDeviceType","origDeviceSessionID","destDeviceSessionID"
INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,VARCHAR(50),VARCHAR(128),INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,VARCHAR(64),VARCHAR(64),INTEGER,INTEGER,INTEGER,INTEGER,VARCHAR(50),VARCHAR(50),VARCHAR(128),INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,VARCHAR(64),VARCHAR(64),INTEGER,INTEGER,VARCHAR(50),UNIQUEIDENTIFIER,VARCHAR(50),VARCHAR(50),VARCHAR(50),VARCHAR(50),INTEGER,VARCHAR(129),VARCHAR(129),INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,VARCHAR(50),INTEGER,VARCHAR(2048),VARCHAR(50),INTEGER,VARCHAR(32),INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,VARCHAR(32),VARCHAR(50),VARCHAR(50),VARCHAR(64),VARCHAR(64),INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,VARCHAR(32),INTEGER,VARCHAR(32),INTEGER,INTEGER,INTEGER,VARCHAR(50),VARCHAR(50),INTEGER,VARCHAR(50),VARCHAR(50),VARCHAR(50),VARCHAR(50),VARCHAR(50),VARCHAR(50),VARCHAR(50),VARCHAR(50),INTEGER,INTEGER,VARCHAR(255),VARCHAR(255),VARCHAR(255),VARCHAR(255),VARCHAR(50),VARCHAR(50),VARCHAR(129),VARCHAR(129),INTEGER,INTEGER,INTEGER,VARCHAR(50),VARCHAR(50),VARCHAR(50),VARCHAR(50),VARCHAR(100),VARCHAR(100),VARCHAR(128),VARCHAR(128)
1,2,28377,36951964,1593547211,2,36951964,152200202,"+49123456780","",0,16,4,152200202,23618,2,20,0,0,0,0,0,0,"0","0",36951967,2,36951967,152200202,"+492222222225","+492222222225","",0,0,4,152200202,23320,2,20,0,0,0,0,0,0,"0","0",1593547211,1593547987,"+49222222220","dbfc3109-c95c-4a46-96fe-d94d89e20cde","PSTN_OUT","","PSTN_OUT","",776,"CUBE","CUBE",12,17,0,18,0,146,0,"StandAloneCluster",18,"","",0,"",2,2,0,0,64,64,"","","","10.10.10.9","10.10.10.9",0,0,0,0,0,0,0,0,0,0,0,0,1,"216DF07ABA4311EAB627B490D23A7537",1,"4AF15980000100000009151F0212640A",0,0,0,"","",5,"","","","","","","","",0,0,"","","","","","","","",0,0,0,"+!","+!","","","","","fe17729285175238945d2e0c49e9929c","b78e7565083b5488b59e1cdc3ba7f58c"
1,2,28381,36951974,1593547949,2,36951974,152200202,"+49811111186","",0,0,4,152200202,25650,2,20,0,0,0,0,0,0,"0","0",36951975,2,36951975,152200202,"+49333333333336","+493333333333336","",0,16,4,152200202,25118,2,20,0,0,0,0,0,0,"0","0",1593547949,1593548010,"+4933333336","c6de9862-b162-4b0b-9e62-81dcdc6d3d3b","PSTN_OUT","","PSTN_OUT","PSTN_OUT",61,"CUBE","CUBE",0,12,0,0,0,0,0,"StandAloneCluster",0,"","",0,"",2,2,0,0,64,64,"","","","10.10.10.9","10.10.10.9",0,0,0,0,0,0,0,0,0,0,0,0,1,"DB6E689EBA4411EAB68BB490D23A7537",1,"049CFA0000010000000915230212640A",0,0,0,"","",5,"","","","","","","","",0,0,"","","","","","","","",0,0,0,"+!","+!","+!","","","","111bc3de1f13551090ee3b07a8b368b9","887ee37864ae553ea014984224b04b77"

from pastash.

each new file contains x datasets, where each row contains one dataset (call detail record). except the first two rows of each files are containing headers. hope to calrify it. ;)

from pastash.

overall, the first step is pretty easy - pastash already has a CSV parser - we just skip the first 2 lines and teach it what headers. From there we have a JSON object we can use to form a HEP packet of various types.

input {
  file {
    path => '/opt/cisco.cdr'
    start_index => 2
  }
}


filter {
  csv {
    headers => ["cdrRecordType","globalCallID_callManagerId","globalCallID_callId","origLegCallIdentifier","dateTimeOrigination","origNodeId","origSpan","origIpAddr","callingPartyNumber","callingPartyUnicodeLoginUserID","origCause_location","origCause_value","origPrecedenceLevel","origMediaTransportAddress_IP","origMediaTransportAddress_Port","origMediaCap_payloadCapability","origMediaCap_maxFramesPerPacket","origMediaCap_g723BitRate","origVideoCap_Codec","origVideoCap_Bandwidth","origVideoCap_Resolution","origVideoTransportAddress_IP","origVideoTransportAddress_Port","origRSVPAudioStat","origRSVPVideoStat","destLegIdentifier","destNodeId","destSpan","destIpAddr","originalCalledPartyNumber","finalCalledPartyNumber","finalCalledPartyUnicodeLoginUserID","destCause_location","destCause_value","destPrecedenceLevel","destMediaTransportAddress_IP","destMediaTransportAddress_Port","destMediaCap_payloadCapability","destMediaCap_maxFramesPerPacket","destMediaCap_g723BitRate","destVideoCap_Codec","destVideoCap_Bandwidth","destVideoCap_Resolution","destVideoTransportAddress_IP","destVideoTransportAddress_Port","destRSVPAudioStat","destRSVPVideoStat","dateTimeConnect","dateTimeDisconnect","lastRedirectDn","pkid","originalCalledPartyNumberPartition","callingPartyNumberPartition","finalCalledPartyNumberPartition","lastRedirectDnPartition","duration","origDeviceName","destDeviceName","origCallTerminationOnBehalfOf","destCallTerminationOnBehalfOf","origCalledPartyRedirectOnBehalfOf","lastRedirectRedirectOnBehalfOf","origCalledPartyRedirectReason","lastRedirectRedirectReason","destConversationId","globalCallId_ClusterID","joinOnBehalfOf","comment","authCodeDescription","authorizationLevel","clientMatterCode","origDTMFMethod","destDTMFMethod","callSecuredStatus","origConversationId","origMediaCap_Bandwidth","destMediaCap_Bandwidth","authorizationCodeValue","outpulsedCallingPartyNumber","outpulsedCalledPartyNumber","origIpv4v6Addr","destIpv4v6Addr","origVideoCap_Codec_Channel2","origVideoCap_Bandwidth_Channel2","origVideoCap_Resolution_Channel2","origVideoTransportAddress_IP_Channel2","origVideoTransportAddress_Port_Channel2","origVideoChannel_Role_Channel2","destVideoCap_Codec_Channel2","destVideoCap_Bandwidth_Channel2","destVideoCap_Resolution_Channel2","destVideoTransportAddress_IP_Channel2","destVideoTransportAddress_Port_Channel2","destVideoChannel_Role_Channel2","IncomingProtocolID","IncomingProtocolCallRef","OutgoingProtocolID","OutgoingProtocolCallRef","currentRoutingReason","origRoutingReason","lastRedirectingRoutingReason","huntPilotPartition","huntPilotDN","calledPartyPatternUsage","IncomingICID","IncomingOrigIOI","IncomingTermIOI","OutgoingICID","OutgoingOrigIOI","OutgoingTermIOI","outpulsedOriginalCalledPartyNumber","outpulsedLastRedirectingNumber","wasCallQueued","totalWaitTimeInQueue","callingPartyNumber_uri","originalCalledPartyNumber_uri","finalCalledPartyNumber_uri","lastRedirectDn_uri","mobileCallingPartyNumber","finalMobileCalledPartyNumber","origMobileDeviceName","destMobileDeviceName","origMobileCallDuration","destMobileCallDuration","mobileCallType","originalCalledPartyPattern","finalCalledPartyPattern","lastRedirectingPartyPattern","huntPilotPattern","origDeviceType","destDeviceType","origDeviceSessionID","destDeviceSessionID"]
  }
  compute_field {
    field => correlation_id
    value => "#{globalCallID_callId}"
  }
}

output {
  stdout {}
}

From here its on you to tell us what could/should be done with all those columns. If you have enough elements, we can transform them into a meaningful payload.

If you'd like to send the CDR as log type for playing, make sure the compute_field actually grabs the right Call-ID column, and then use the following output:

output {
  hep {
    host => 127.0.0.1
    port => 9060
    hep_id => 2022
    hep_type => 100
  }
}

from pastash.

sounds really good, correlation id is fine.
if i run pastash twice on the same file, will I get duplicated entries?

from pastash.

besides my last question i have some errors, a field seems to be too large in my files:

[Fri, 20 Nov 2020 09:09:48 GMT] INFO Start reading /opt/ciscoCdr/cdr_StandAloneCluster_02_202011191314_46926 at 2 fd 578
[Fri, 20 Nov 2020 09:09:49 GMT] ERROR PREHEP ERROR: RangeError [ERR_OUT_OF_RANGE]: The value of "value" is out of range. It must be >= 0 and <= 65535. Received 1072476

from pastash.

and last but not least: paStash is running, but i don't see any data in table hep_proto_100_default or hep_proto_100_default__.

how to debug it?

running pastash in debug mode looks quite fine:

[Fri, 20 Nov 2020 11:25:05 GMT] DEBUG CSV Data! { '0': '1', '1': '2', '2': '979553', '3': '41410853', '4': '1605870844', '5': '2', '6': '41410853', '7': '1729258506', '8': '+491794181082', '9': '', '10': '0', '11': '393216', '12': '4', '13': '1729258506', '14': '18130', '15': '4', '16': '20', '17': '0', '18': '0', '19': '0', '20': '0', '21': '0', '22': '0', '23': '0', '24': '0', '25': '41410854', '26': '2', '27': '0', '28': '-354288118', '29': '1306', '30': '1306', '31': 'cuwl', '32': '0', '33': '393216', '34': '4', '35': '-354288118', '36': '18054', '37': '4', '38': '20', '39': '0', '40': '0', '41': '0', '42': '0', '43': '0', '44': '0', '45': '0', '46': '0', '47': '1605870844', '48': '1605870845', '49': '1306', '50': 'c79b1957-08b4-4905-bb43-84fd8998acd1', '51': 'Eingeloggt', '52': '', '53': 'Eingeloggt', '54': 'Eingeloggt', '55': '1', '56': 'big-vacdgw1', '57': 'CSF1306', '58': '18', '59': '18', '60': '0', '61': '0', '62': '0', '63': '0', '64': '0', '65': 'StandAloneCluster', '66': '0', '67': '', '68': '', '69': '0', '70': '', '71': '2', '72': '3', '73': '0', '74': '0', '75': '64', '76': '64', '77': '', '78': '', '79': '', '80': '10.100.18.103', '81': '10.254.225.234', '82': '0', '83': '0', '84': '0', '85': '0', '86': '0', '87': '0', '88': '0', '89': '0', '90': '0', '91': '0', '92': '0', '93': '0', '94': '1', '95': '7E676B8000010000001B46C30212640A', '96': '3', '97': '00000000000EF2610277E12600000000', '98': '0', '99': '0', '100': '0', '101': '', '102': '', '103': '2', '104': '', '105': '', '106': '', '107': '', '108': '', '109': '', '110': '', '111': '', '112': '0', '113': '0', '114': '', '115': '[email protected]', '116': '[email protected]', '117': '[email protected]', '118': '', '119': '', '120': '', '121': '', '122': '0', '123': '0', '124': '0', '125': '1306', '126': '1306', '127': '1306', '128': '', '129': '', '130': '', '131': 'd3e5b1d52522636a5ba969ab83463097', '132': '79df6b5800105000a00098e7434cdd1d' }

config file:
output {
# stdout {}
hep {
host => 127.0.0.1
port => 9060
hep_id => 18000
hep_type => 100
}
}

netstat on local server:
root@sipmon01:/opt/ciscoCdr# netstat -tulpen|grep 9060
tcp6 0 0 :::9060 :::* LISTEN 0 31381 1588/docker-proxy
udp 0 0 0.0.0.0:51105 0.0.0.0:* 0 39060831 1022260/node
udp6 0 0 :::9060 :::* 0 31438 1616/docker-proxy

from pastash.

I doubt this would or should work, but I can't tell without seeing the config you're running.

The point of pastash is not sending the data as-is (unless its already well formatted) for rather transforming it into what it should be. In this case, nothing will work without a correlation_id to begin with and sending the raw CSV seems to me the wrong choice. The CSV parser (below) will turn the rows into a JSON object. You might want to try send that AFTER electing a correlation ID (call-id of the sip session, from the CDR)

filter {
  csv {
    headers => ["cdrRecordType","globalCallID_callManagerId","globalCallID_callId","origLegCallIdentifier","dateTimeOrigination","origNodeId","origSpan","origIpAddr","callingPartyNumber","callingPartyUnicodeLoginUserID","origCause_location","origCause_value","origPrecedenceLevel","origMediaTransportAddress_IP","origMediaTransportAddress_Port","origMediaCap_payloadCapability","origMediaCap_maxFramesPerPacket","origMediaCap_g723BitRate","origVideoCap_Codec","origVideoCap_Bandwidth","origVideoCap_Resolution","origVideoTransportAddress_IP","origVideoTransportAddress_Port","origRSVPAudioStat","origRSVPVideoStat","destLegIdentifier","destNodeId","destSpan","destIpAddr","originalCalledPartyNumber","finalCalledPartyNumber","finalCalledPartyUnicodeLoginUserID","destCause_location","destCause_value","destPrecedenceLevel","destMediaTransportAddress_IP","destMediaTransportAddress_Port","destMediaCap_payloadCapability","destMediaCap_maxFramesPerPacket","destMediaCap_g723BitRate","destVideoCap_Codec","destVideoCap_Bandwidth","destVideoCap_Resolution","destVideoTransportAddress_IP","destVideoTransportAddress_Port","destRSVPAudioStat","destRSVPVideoStat","dateTimeConnect","dateTimeDisconnect","lastRedirectDn","pkid","originalCalledPartyNumberPartition","callingPartyNumberPartition","finalCalledPartyNumberPartition","lastRedirectDnPartition","duration","origDeviceName","destDeviceName","origCallTerminationOnBehalfOf","destCallTerminationOnBehalfOf","origCalledPartyRedirectOnBehalfOf","lastRedirectRedirectOnBehalfOf","origCalledPartyRedirectReason","lastRedirectRedirectReason","destConversationId","globalCallId_ClusterID","joinOnBehalfOf","comment","authCodeDescription","authorizationLevel","clientMatterCode","origDTMFMethod","destDTMFMethod","callSecuredStatus","origConversationId","origMediaCap_Bandwidth","destMediaCap_Bandwidth","authorizationCodeValue","outpulsedCallingPartyNumber","outpulsedCalledPartyNumber","origIpv4v6Addr","destIpv4v6Addr","origVideoCap_Codec_Channel2","origVideoCap_Bandwidth_Channel2","origVideoCap_Resolution_Channel2","origVideoTransportAddress_IP_Channel2","origVideoTransportAddress_Port_Channel2","origVideoChannel_Role_Channel2","destVideoCap_Codec_Channel2","destVideoCap_Bandwidth_Channel2","destVideoCap_Resolution_Channel2","destVideoTransportAddress_IP_Channel2","destVideoTransportAddress_Port_Channel2","destVideoChannel_Role_Channel2","IncomingProtocolID","IncomingProtocolCallRef","OutgoingProtocolID","OutgoingProtocolCallRef","currentRoutingReason","origRoutingReason","lastRedirectingRoutingReason","huntPilotPartition","huntPilotDN","calledPartyPatternUsage","IncomingICID","IncomingOrigIOI","IncomingTermIOI","OutgoingICID","OutgoingOrigIOI","OutgoingTermIOI","outpulsedOriginalCalledPartyNumber","outpulsedLastRedirectingNumber","wasCallQueued","totalWaitTimeInQueue","callingPartyNumber_uri","originalCalledPartyNumber_uri","finalCalledPartyNumber_uri","lastRedirectDn_uri","mobileCallingPartyNumber","finalMobileCalledPartyNumber","origMobileDeviceName","destMobileDeviceName","origMobileCallDuration","destMobileCallDuration","mobileCallType","originalCalledPartyPattern","finalCalledPartyPattern","lastRedirectingPartyPattern","huntPilotPattern","origDeviceType","destDeviceType","origDeviceSessionID","destDeviceSessionID"]
  }
  compute_field {
    field => correlation_id
    value => "#{globalCallID_callId}"
  }
}

from pastash.

ah sorry, just piped the output of my config, the filter was set as suggested from you before.

here completely:
`root@sipmon01:/opt/ciscoCdr# cat /opt/paStash/bin/config.yaml
input {
file {
path => '/opt/ciscoCdr/cdr*'
start_index => 2
}
}

filter {
csv {
headers => ["cdrRecordType","globalCallID_callManagerId","globalCallID_callId","origLegCallIdentifier","dateTimeOrigination","origNodeId","origSpan","origIpAddr","callingPartyNumber","callingPartyUnicodeLoginUserID","origCause_location","origCause_value","origPrecedenceLevel","origMediaTransportAddress_IP","origMediaTransportAddress_Port","origMediaCap_payloadCapability","origMediaCap_maxFramesPerPacket","origMediaCap_g723BitRate","origVideoCap_Codec","origVideoCap_Bandwidth","origVideoCap_Resolution","origVideoTransportAddress_IP","origVideoTransportAddress_Port","origRSVPAudioStat","origRSVPVideoStat","destLegIdentifier","destNodeId","destSpan","destIpAddr","originalCalledPartyNumber","finalCalledPartyNumber","finalCalledPartyUnicodeLoginUserID","destCause_location","destCause_value","destPrecedenceLevel","destMediaTransportAddress_IP","destMediaTransportAddress_Port","destMediaCap_payloadCapability","destMediaCap_maxFramesPerPacket","destMediaCap_g723BitRate","destVideoCap_Codec","destVideoCap_Bandwidth","destVideoCap_Resolution","destVideoTransportAddress_IP","destVideoTransportAddress_Port","destRSVPAudioStat","destRSVPVideoStat","dateTimeConnect","dateTimeDisconnect","lastRedirectDn","pkid","originalCalledPartyNumberPartition","callingPartyNumberPartition","finalCalledPartyNumberPartition","lastRedirectDnPartition","duration","origDeviceName","destDeviceName","origCallTerminationOnBehalfOf","destCallTerminationOnBehalfOf","origCalledPartyRedirectOnBehalfOf","lastRedirectRedirectOnBehalfOf","origCalledPartyRedirectReason","lastRedirectRedirectReason","destConversationId","globalCallId_ClusterID","joinOnBehalfOf","comment","authCodeDescription","authorizationLevel","clientMatterCode","origDTMFMethod","destDTMFMethod","callSecuredStatus","origConversationId","origMediaCap_Bandwidth","destMediaCap_Bandwidth","authorizationCodeValue","outpulsedCallingPartyNumber","outpulsedCalledPartyNumber","origIpv4v6Addr","destIpv4v6Addr","origVideoCap_Codec_Channel2","origVideoCap_Bandwidth_Channel2","origVideoCap_Resolution_Channel2","origVideoTransportAddress_IP_Channel2","origVideoTransportAddress_Port_Channel2","origVideoChannel_Role_Channel2","destVideoCap_Codec_Channel2","destVideoCap_Bandwidth_Channel2","destVideoCap_Resolution_Channel2","destVideoTransportAddress_IP_Channel2","destVideoTransportAddress_Port_Channel2","destVideoChannel_Role_Channel2","IncomingProtocolID","IncomingProtocolCallRef","OutgoingProtocolID","OutgoingProtocolCallRef","currentRoutingReason","origRoutingReason","lastRedirectingRoutingReason","huntPilotPartition","huntPilotDN","calledPartyPatternUsage","IncomingICID","IncomingOrigIOI","IncomingTermIOI","OutgoingICID","OutgoingOrigIOI","OutgoingTermIOI","outpulsedOriginalCalledPartyNumber","outpulsedLastRedirectingNumber","wasCallQueued","totalWaitTimeInQueue","callingPartyNumber_uri","originalCalledPartyNumber_uri","finalCalledPartyNumber_uri","lastRedirectDn_uri","mobileCallingPartyNumber","finalMobileCalledPartyNumber","origMobileDeviceName","destMobileDeviceName","origMobileCallDuration","destMobileCallDuration","mobileCallType","originalCalledPartyPattern","finalCalledPartyPattern","lastRedirectingPartyPattern","huntPilotPattern","origDeviceType","destDeviceType","origDeviceSessionID","destDeviceSessionID"]
}
compute_field {
field => correlation_id
value => "#{globalCallID_callId}"
}
}

output {
# stdout {}
hep {
host => 127.0.0.1
port => 9060
hep_id => 18000
hep_type => 100
}
}
`

from pastash.

Mind the host parameter should be HOMER's IP. Is HOMER running on the same host reading the CDRs? Sniff around port 9060 see if they actually show up.

from pastash.

yes, homer is running locally. will check communication on port 9060

from pastash.

lot of traffic from heplify-clients, but no traffic from my local server, where paStash for parsing CDRs and heplify-server (docker) are running. changed the config to host sipmon01.fqdn but no change...

from pastash.

ok, got a step further. there is traffic from paStash to heplify-server (had to dump the docker interface), but all packets are failing:

14:11:50.762308 IP sipmon01.37059 > 172.18.0.3.9060: UDP, bad length 8151 > 1472
14:11:50.762525 IP sipmon01.37059 > 172.18.0.3.9060: UDP, bad length 8151 > 1472
14:11:50.762659 IP sipmon01.37059 > 172.18.0.3.9060: UDP, bad length 8151 > 1472
14:11:50.763065 IP sipmon01.37059 > 172.18.0.3.9060: UDP, bad length 8151 > 1472
[...]

from pastash.

Try adding the following filter to generate a custom payload with a few elements of your CDR to test with:

 mustache {
    template => 'CDR: {{direction}} {{caller_id_number}} -> {{destination_number}}: {{lcr_carrier}} {{lcr_rate}}'
    target_field => 'payload'
  }

from pastash.