jonmuehlst / pcap-feature-extractor Goto Github PK

Saving the trained classifiers to files in order to use them in the simulator.

We will probably use "pickle" (not rick) or serialisation.

Making sure that:

1. Opening the file for reading (in the classification) does not make the listening-sending script re-send it (due to re-closing it).
2. That there is no file sending prior to the end of the file writing.

Classifiers: Traffic type, Application name and Action name

Keep the original string that represents the session time and add a new column that represent the session time in milliseconds

Depending on the tshark fields requested, the amount of packets retrieved from a pcap is variable!

• Assertion for pcap size (packet count) when reading a pcap file before proceeding

By evaluating the metadata in the destination mac address the system should:

1. Discard non-relevant pcaps (with metadata that is used by "Flash").
2. Use the relevant model for the appropriate time window.

python (2.7):
pip install --user scikit_learn==0.18.1 numpy==1.13.1 scapy==2.3.2 matplotlib==2.1.0 pandas==0.20.3 nose==1.3.7 pyhcl==0.1.15 path.py==8.1.2 scipy==1.0.0 ipython==5.5.0 jupyter==1.0.0 pathos==0.2.1 pyspark scapy-ssl_tls==1.2.3.2

ubuntu(16.04):
sudo apt install wireshark-common=2.0.2+ga16e22e-1
sudo apt install tshark=2.0.2+ga16e22e-1
sudo apt install inotify-tools=3.14-1ubuntu1

There are 5 data folders, each one has different type of data.
For every pcap file we should change the destination mac address such that it will be consistent with the new format (will contain the real label + metadata for later analysis).

Will be done using "over_load_mac.py" and "edit_pcaps_mac_dst.py".