Comments (2)
While running our monitoring script which uses jc
to parse netstat
output on linux, it started throwing "index out of range" errors. I added some debug printing to jc/parsers/netsat_linux.py
to see what is going on.
First one was to see the data comming in
def parse_network(headers, entry):
LIST_OF_STATES = [
"ESTABLISHED", "SYN_SENT", "SYN_RECV", "FIN_WAIT1", "FIN_WAIT2",
"TIME_WAIT", "CLOSED", "CLOSE_WAIT", "LAST_ACK", "LISTEN", "CLOSING",
"UNKNOWN", "7"
]
# split entry based on presence of value in "State" column
contains_state = any(state in entry for state in LIST_OF_STATES)
split_modifier = 1 if contains_state else 2
### DEBUG PRINT ###
print(entry)
entry = entry.split(maxsplit=len(headers) - split_modifier)
output:
tcp 0 0 0.0.0.0:9898 0.0.0.0:* LISTEN 1178/pgpool
tcp 0 0 192.168.68.116:9102 0.0.0.0:* LISTEN 584/bareos-fd
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 600/sshd: /usr/sbin
tcp 0 0 0.0.0.0:5432 0.0.0.0:* LISTEN 1178/pgpool
tcp 0 0 0.0.0.0:5433 0.0.0.0:* LISTEN 1676/postgres
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 928/exim4
tcp 0 0 0.0.0.0:10050 0.0.0.0:* LISTEN 1817931/zabbix_agen
tcp 0 0 0.0.0.0:9000 0.0.0.0:* LISTEN 1187/pgpool: watchd
tcp6 0 0 :::9898 :::* LISTEN 1178/pgpool
tcp6 0 0 :::22 :::* LISTEN 600/sshd: /usr/sbin
tcp6 0 0 :::5432 :::* LISTEN 1178/pgpool
tcp6 0 0 :::5433 :::* LISTEN 1676/postgres
tcp6 0 0 :::10050 :::* LISTEN 1817931/zabbix_agen
udp 0 0 0.0.0.0:68 0.0.0.0:* 535/dhclient
udp 0 0 0.0.0.0:37300 0.0.0.0:* 1204/pgpool: heartb
udp 0 0 0.0.0.0:9694 0.0.0.0:* 1205/pgpool: heartb
udp 0 0 0.0.0.0:9694 0.0.0.0:* 1203/pgpool: heartb
udp 0 0 0.0.0.0:44649 0.0.0.0:* 1206/pgpool: heartb
udp 0 0 0.0.0.0:52868 0.0.0.0:* 494/rsyslogd
The interesting part comes here, It is clear that none of the udp
processes has value in the STATE
column, howerver after running the script with this debug print added, we can see the error:
def parse_network(headers, entry):
LIST_OF_STATES = [
"ESTABLISHED", "SYN_SENT", "SYN_RECV", "FIN_WAIT1", "FIN_WAIT2",
"TIME_WAIT", "CLOSED", "CLOSE_WAIT", "LAST_ACK", "LISTEN", "CLOSING",
"UNKNOWN", "7"
]
# split entry based on presence of value in "State" column
contains_state = any(state in entry for state in LIST_OF_STATES)
split_modifier = 1 if contains_state else 2
entry = entry.split(maxsplit=len(headers) - split_modifier)
### DEBUG PRINT ###
print(entry, len(entry))
['tcp', '0', '0', '0.0.0.0:9898', '0.0.0.0:*', 'LISTEN', '1178/pgpool '] 7
['tcp', '0', '0', '192.168.68.116:9102', '0.0.0.0:*', 'LISTEN', '584/bareos-fd '] 7
['tcp', '0', '0', '0.0.0.0:22', '0.0.0.0:*', 'LISTEN', '600/sshd: /usr/sbin '] 7
['tcp', '0', '0', '0.0.0.0:5432', '0.0.0.0:*', 'LISTEN', '1178/pgpool '] 7
['tcp', '0', '0', '0.0.0.0:5433', '0.0.0.0:*', 'LISTEN', '1676/postgres '] 7
['tcp', '0', '0', '127.0.0.1:25', '0.0.0.0:*', 'LISTEN', '928/exim4 '] 7
['tcp', '0', '0', '0.0.0.0:10050', '0.0.0.0:*', 'LISTEN', '1817931/zabbix_agen '] 7
['tcp', '0', '0', '0.0.0.0:9000', '0.0.0.0:*', 'LISTEN', '1187/pgpool: watchd '] 7
['tcp6', '0', '0', ':::9898', ':::*', 'LISTEN', '1178/pgpool '] 7
['tcp6', '0', '0', ':::22', ':::*', 'LISTEN', '600/sshd: /usr/sbin '] 7
['tcp6', '0', '0', ':::5432', ':::*', 'LISTEN', '1178/pgpool '] 7
['tcp6', '0', '0', ':::5433', ':::*', 'LISTEN', '1676/postgres '] 7
['tcp6', '0', '0', ':::10050', ':::*', 'LISTEN', '1817931/zabbix_agen '] 7
['udp', '0', '0', '0.0.0.0:68', '0.0.0.0:*', '535/dhclient '] 6
['udp', '0', '0', '0.0.0.0:37300', '0.0.0.0:*', '1204/pgpool:', 'heartb '] 7
['udp', '0', '0', '0.0.0.0:9694', '0.0.0.0:*', '1205/pgpool: heartb '] 6
['udp', '0', '0', '0.0.0.0:9694', '0.0.0.0:*', '1203/pgpool: heartb '] 6
['udp', '0', '0', '0.0.0.0:44649', '0.0.0.0:*', '1206/pgpool: heartb '] 6
['udp', '0', '0', '0.0.0.0:52868', '0.0.0.0:*', '494/rsyslogd '] 6
The row with the heartb
process gets parsed incorrectly, because it is running on port :37300
. The number 7, which is one of the recognized states after the update, gets picked up as delimetter and breaks the formatting of the row with 1204/pgpool:
ending up in the STATE column and heartb
in PID/Program name
The "index out of range error" I'm getting is because I take the output of jc and split the column PID/Program name on '/'
and access both parts of it.
In this corner case however, it breaks the script, because of the parsing error I described higher. (since there is no /
in the column)
I was localy able to fix the problem when I removed the number 7 from the LIST_OF_STATES. I remember tho, that without the 7 present in it, some tests of yours were failing.
What I would like to do, is to remove the number 7 from LIST_OF_STATES because according to the documentation it is not a valid state that can appear as output of the netstat command, and to try and debug why the tests are failing.
I will update you on the progress. Is this approach okay with you?
from jc.
Oh, dear - that's a nasty little bug. Thanks for looking into this - looking forward to working with you on this!
from jc.
Related Issues (20)
- New Parser - NUT-upsc command HOT 1
- [parsers /pci_ids.py] Only last [subvendor][subdevice] got loaded into result when multiple subdevices having the same subvendor id HOT 3
- Bug: Traceroute ignored IPs when multiple IPs on hop HOT 3
- "ss" Parser not working correctly: Entries Missing HOT 5
- .netrc support HOT 1
- Problem with --pip-show parser for a module with nothing on the License: line but which has following lines HOT 4
- lspci strips trailing digit on bus ID HOT 3
- `--zpool-status` fails to handle the `spares` section of `zpool status` HOT 3
- New parser request: pacman -Si HOT 6
- 1.25.3: pytest fails in nits which are comparing time HOT 8
- Python 3.12 test regression: `tests.test_ip_address.MyTests.test_ip_address_ipv6_6to4` HOT 11
- Parsing mounts with spaces in filenames fails - AttributeError: 'NoneType' object has no attribute 'groupdict' HOT 2
- New Streaming-parser proposal: traceroute-s HOT 1
- ethtool parsing doesn't work HOT 1
- jc ls --time-style=+"%F %T" -al -> wrong output HOT 1
- New Parser: cloudformation template.yaml HOT 4
- ifconfig parser of IPv4 do not support utun interfaces on macOS HOT 1
- Python 3.13 test failure: `test_ip_address_ipv6_ipv4_mapped` HOT 4
- uptime parser with 0 user(s) on ubuntu 24.04 docker not matching HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jc.