Git Product home page Git Product logo

Comments (2)

HervisDaubeny avatar HervisDaubeny commented on August 12, 2024

While running our monitoring script which uses jc to parse netstat output on linux, it started throwing "index out of range" errors. I added some debug printing to jc/parsers/netsat_linux.py to see what is going on.

First one was to see the data comming in

def parse_network(headers, entry):
    LIST_OF_STATES = [
        "ESTABLISHED", "SYN_SENT", "SYN_RECV", "FIN_WAIT1", "FIN_WAIT2",
        "TIME_WAIT", "CLOSED", "CLOSE_WAIT", "LAST_ACK", "LISTEN", "CLOSING",
        "UNKNOWN", "7"
    ]

    # split entry based on presence of value in "State" column
    contains_state = any(state in entry for state in LIST_OF_STATES)
    split_modifier = 1 if contains_state else 2

    ### DEBUG PRINT ###
    print(entry)
    entry = entry.split(maxsplit=len(headers) - split_modifier)

output:

tcp        0      0 0.0.0.0:9898            0.0.0.0:*               LISTEN      1178/pgpool
tcp        0      0 192.168.68.116:9102     0.0.0.0:*               LISTEN      584/bareos-fd
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      600/sshd: /usr/sbin
tcp        0      0 0.0.0.0:5432            0.0.0.0:*               LISTEN      1178/pgpool
tcp        0      0 0.0.0.0:5433            0.0.0.0:*               LISTEN      1676/postgres
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      928/exim4
tcp        0      0 0.0.0.0:10050           0.0.0.0:*               LISTEN      1817931/zabbix_agen
tcp        0      0 0.0.0.0:9000            0.0.0.0:*               LISTEN      1187/pgpool: watchd
tcp6       0      0 :::9898                 :::*                    LISTEN      1178/pgpool
tcp6       0      0 :::22                   :::*                    LISTEN      600/sshd: /usr/sbin
tcp6       0      0 :::5432                 :::*                    LISTEN      1178/pgpool
tcp6       0      0 :::5433                 :::*                    LISTEN      1676/postgres
tcp6       0      0 :::10050                :::*                    LISTEN      1817931/zabbix_agen
udp        0      0 0.0.0.0:68              0.0.0.0:*                           535/dhclient
udp        0      0 0.0.0.0:37300           0.0.0.0:*                           1204/pgpool: heartb
udp        0      0 0.0.0.0:9694            0.0.0.0:*                           1205/pgpool: heartb
udp        0      0 0.0.0.0:9694            0.0.0.0:*                           1203/pgpool: heartb
udp        0      0 0.0.0.0:44649           0.0.0.0:*                           1206/pgpool: heartb
udp        0      0 0.0.0.0:52868           0.0.0.0:*                           494/rsyslogd

The interesting part comes here, It is clear that none of the udp processes has value in the STATE column, howerver after running the script with this debug print added, we can see the error:

def parse_network(headers, entry):
    LIST_OF_STATES = [
        "ESTABLISHED", "SYN_SENT", "SYN_RECV", "FIN_WAIT1", "FIN_WAIT2",
        "TIME_WAIT", "CLOSED", "CLOSE_WAIT", "LAST_ACK", "LISTEN", "CLOSING",
        "UNKNOWN", "7"
    ]

    # split entry based on presence of value in "State" column
    contains_state = any(state in entry for state in LIST_OF_STATES)
    split_modifier = 1 if contains_state else 2

    entry = entry.split(maxsplit=len(headers) - split_modifier)
    ### DEBUG PRINT ###
    print(entry, len(entry))
['tcp', '0', '0', '0.0.0.0:9898', '0.0.0.0:*', 'LISTEN', '1178/pgpool         '] 7
['tcp', '0', '0', '192.168.68.116:9102', '0.0.0.0:*', 'LISTEN', '584/bareos-fd       '] 7
['tcp', '0', '0', '0.0.0.0:22', '0.0.0.0:*', 'LISTEN', '600/sshd: /usr/sbin '] 7
['tcp', '0', '0', '0.0.0.0:5432', '0.0.0.0:*', 'LISTEN', '1178/pgpool         '] 7
['tcp', '0', '0', '0.0.0.0:5433', '0.0.0.0:*', 'LISTEN', '1676/postgres       '] 7
['tcp', '0', '0', '127.0.0.1:25', '0.0.0.0:*', 'LISTEN', '928/exim4           '] 7
['tcp', '0', '0', '0.0.0.0:10050', '0.0.0.0:*', 'LISTEN', '1817931/zabbix_agen '] 7
['tcp', '0', '0', '0.0.0.0:9000', '0.0.0.0:*', 'LISTEN', '1187/pgpool: watchd '] 7
['tcp6', '0', '0', ':::9898', ':::*', 'LISTEN', '1178/pgpool         '] 7
['tcp6', '0', '0', ':::22', ':::*', 'LISTEN', '600/sshd: /usr/sbin '] 7
['tcp6', '0', '0', ':::5432', ':::*', 'LISTEN', '1178/pgpool         '] 7
['tcp6', '0', '0', ':::5433', ':::*', 'LISTEN', '1676/postgres       '] 7
['tcp6', '0', '0', ':::10050', ':::*', 'LISTEN', '1817931/zabbix_agen '] 7
['udp', '0', '0', '0.0.0.0:68', '0.0.0.0:*', '535/dhclient        '] 6
['udp', '0', '0', '0.0.0.0:37300', '0.0.0.0:*', '1204/pgpool:', 'heartb '] 7
['udp', '0', '0', '0.0.0.0:9694', '0.0.0.0:*', '1205/pgpool: heartb '] 6
['udp', '0', '0', '0.0.0.0:9694', '0.0.0.0:*', '1203/pgpool: heartb '] 6
['udp', '0', '0', '0.0.0.0:44649', '0.0.0.0:*', '1206/pgpool: heartb '] 6
['udp', '0', '0', '0.0.0.0:52868', '0.0.0.0:*', '494/rsyslogd        '] 6

The row with the heartb process gets parsed incorrectly, because it is running on port :37300. The number 7, which is one of the recognized states after the update, gets picked up as delimetter and breaks the formatting of the row with 1204/pgpool: ending up in the STATE column and heartb in PID/Program name

The "index out of range error" I'm getting is because I take the output of jc and split the column PID/Program name on '/' and access both parts of it.
In this corner case however, it breaks the script, because of the parsing error I described higher. (since there is no / in the column)

I was localy able to fix the problem when I removed the number 7 from the LIST_OF_STATES. I remember tho, that without the 7 present in it, some tests of yours were failing.

What I would like to do, is to remove the number 7 from LIST_OF_STATES because according to the documentation it is not a valid state that can appear as output of the netstat command, and to try and debug why the tests are failing.
I will update you on the progress. Is this approach okay with you?

from jc.

kellyjonbrazil avatar kellyjonbrazil commented on August 12, 2024

Oh, dear - that's a nasty little bug. Thanks for looking into this - looking forward to working with you on this!

from jc.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.