iffy / ppo Goto Github PK

Please parse output

License: Apache License 2.0

Python 100.00%

ppo's Introduction

ppo parses output from commands that don't have nicely parsed output. Here's a picture:

whatever output the program gives -> ppo -> YAML/JSON

Installation

pip install git+https://github.com/iffy/ppo.git

Example

For instance, here's parsing nmap xml output:

nmap -sP 192.168.13.205 -oX - | ppo -f yaml

Which produces:

hosts:
- addresses:
  - addr: 192.168.13.205
    addrtype: ipv4
  hostnames: []
  status:
    reason: reset
    reason_ttl: 255
    state: up
  times:
    rttvar: 5000
    srtt: 123
    to: 100000
nmaprun:
  args: nmap -sP -oX - 192.168.13.205
  debugging:
    level: 0
  runstats:
    finished:
      elapsed: 0.02
      exit: success
      summary: Nmap done at Thu Aug 27 17:19:29 2015; 1 IP address (1 host up) scanned
        in 0.02 seconds
      time: 1440710369
      timestr: Thu Aug 27 17:19:29 2015
    hosts:
      down: 0
      total: 1
      up: 1
  scanner: nmap
  start: 1440710369
  startstr: Thu Aug 27 17:19:29 2015
  verbose:
    level: 0
  version: '6.47'
  xmloutputversion: '1.04'

And here's iptables -L:

iptables -nvL | ppo -f yaml

INPUT:
  ACCEPT:
    packets: "3930"
    bytes: "449K"
  items:
    - pkts: "0"
      bytes: "0"
      target: ACCEPT
      prot: all
      opt: --
      in: "*"
      out: "*"
      source: 192.168.1.205
      destination: 0.0.0.0/0
    - pkts: "3113"
      bytes: "128K"
      target: ACCEPT
      prot: all
      opt: --
      in: "*"
      out: "*"
      source: 192.168.13.206
      destination: 0.0.0.0/0
FORWARD:
  ACCEPT:
    packets: "0"
    bytes: "0"
  items: []
OUTPUT:
  ACCEPT:
    packets: "4352"
    bytes: "523K"
  items:
    - pkts: "4074"
      bytes: "183K"
      target: ACCEPT
      prot: all
      opt: --
      in: "*"
      out: "*"
      source: 0.0.0.0/0
      destination: 192.168.13.206

Use it with jq

By default, ppo renders JSON, making it nice to use with jq:

# iptables -nvL | ppo | jq '.INPUT.items[] | select(.source == "192.168.1.205")'
{
  "opt": "--",
  "destination": "0.0.0.0/0",
  "target": "ACCEPT",
  "prot": "all",
  "bytes": "0",
  "source": "192.168.1.205",
  "in": "*",
  "pkts": "0",
  "out": "*"
}

Use it with grep

You can produce greppable/cuttable output with -f grep:

$ cat functests/cases/in-nmap-1 | ppo -f grep | grep 'port: 443'
hosts: endtime: 1440623310 hostnames: [] ipv4: 192.168.13.203 starttime: 1440623308 ports: port: 443 protocol: tcp
hosts: endtime: 1440623310 hostnames: [] ipv4: 192.168.13.203 starttime: 1440623308 ports: port: 443 protocol: tcp service: conf: 3 method: table name: https
hosts: endtime: 1440623310 hostnames: [] ipv4: 192.168.13.203 starttime: 1440623308 ports: port: 443 protocol: tcp state: reason: no-response reason_ttl: 0 state: filtered

Or this:

$ cat functests/cases/in-nmap-1 | ppo -f grep | grep 'state: open'
hosts: endtime: 1440623310 hostnames: [] ipv4: 192.168.13.203 starttime: 1440623308 ports: port: 80 protocol: tcp state: reason: syn-ack reason_ttl: 128 state: open

Supported programs

See a list of parseable formats with ppo --ls or look in ppo/parse_plugins.

ppo's People

Contributors

Watchers

ppo's Issues

Support `rpcclient` output

Support `theharvester` output

Support `nbtscan` output

Get tests passing on python3.4

support nmap resumed scans (where it appends xml)

Support `dnsrecon` output

Support `snmpwalk` output

Unable to install on some kinds of python 2.7.5

Collecting git+https://github.com/iffy/ppo.git
  Cloning https://github.com/iffy/ppo.git to /var/folders/8q/9ct1l_r12vsdpg_f19j6j4600000gn/T/pip-RZkROT-build
    Complete output from command python setup.py egg_info:
    error in ppo setup command: Invalid environment marker: python_version<="2.7"
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /var/folders/8q/9ct1l_r12vsdpg_f19j6j4600000gn/T/pip-RZkROT-build

iptables wrong destination

Parse this correctly:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
DROP       tcp  --  anywhere            !192.168.12.215       tcp dpt:13327
DROP       tcp  --  anywhere            !192.168.12.215       tcp dpt:4444

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

broken pipe

On some big output (maybe? seems like the problem), I get broken pipes. Perhaps I shouldn't read and write the whole thing at once :)

I saw it when doing cat /tmp/nmap.xml | ppo -f grep | grep process

Support python3.4 and python2.7

Handle this iptables output

Chain INPUT (policy ACCEPT 171 packets, 12693 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       tcp  --  *      *       0.0.0.0/0           !127.0.0.1            tcp dpt:13327
    0     0 DROP       tcp  --  *      *       0.0.0.0/0           !127.0.0.1            tcp dpt:4444

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 92 packets, 9291 bytes)
 pkts bytes target     prot opt in     out     source               destination

Nikto: Support SSL info

- Nikto v2.1.6
---------------------------------------------------------------------------
+ Target IP:          192.168.13.223
+ Target Hostname:    192.168.13.223
+ Target Port:        443
---------------------------------------------------------------------------
+ SSL Info:        Subject: /CN=localhost
                   Ciphers: DHE-RSA-AES256-SHA
                   Issuer:  /CN=localhost
+ Start Time:         2015-10-01 22:51:07 (GMT-4)
---------------------------------------------------------------------------
+ Server: Apache/2.2.14 (Win32) DAV/2 mod_ssl/2.2.14 OpenSSL/0.9.8l mod_autoindex_color PHP/5.3.1 mod_apreq2-20090110/2.7.1 mod_perl/2.0.4 Perl/v5.10.1
+ The anti-clickjacking X-Frame-Options header is not present.
+ The site uses SSL and the Strict-Transport-Security HTTP header is not defined.
+ OSVDB-3268: /: Directory indexing found.
...