Why identify a Flow using these features in flow_generator.py? about flowprint HOT 2 CLOSED

Hi wesly2000,

TL;DR

You are correct, we define a flow as the (ip source, ip destination, sport, dport, protocol)-tuple.
However, when using (ip source, ip destination, sport, dport, protocol) a flow communicating in one direction (e.g., client to server) would be different from a flow communicating in the other direction (e.g., server to client) (ip destination, ip source, dport, sport, protocol).

In the reader, tshark deals with this for us by giving a unique identifier for the TCP/UDP connection (packet[2]), however identifiers are reused for TCP and UDP so we need protocol information (packet[1]). Finally, these identifiers are reused per .pcap file so we also need to specify the filename to be unique. So the (packet[0], packet[1], packet[2]) should be equivalent to the (ip source, ip destination, sport, dport, protocol)-tuple, but without having to deal with the mess of flow directions.

Detailed explanation

You correctly understood from the paper that the same (ip source, ip destination, sport, dport, protocol)-tuple constitutes a flow. However, there are a couple of gory details that make the implementation a bit more complex. The most important one here is that often, a network connection is a two-way communication (e.g., a client sending data to a server, and a server responding with data to the client). In our work, we believe that this two-way communication should still be considered as a single flow. Therefore, if you use the (ip source, ip destination, sport, dport, protocol)-tuple as a key for grouping, you would have to take into account that (ip destination, ip source, dport, sport, protocol) has to map to the same grouping.

The tool that we use for reading tshark offers the following two options:

• -e tcp.stream, assigns a unique number to each different TCP stream in the pcap
• -e udp.stream, assigns a unique number to each different UDP stream in the pcap

That means that the first (ip source, ip destination, sport, dport, TCP)/(ip destination, ip source, dport, sport, TCP) it recognizes will get ID 0, the second will get ID 1, etc. The same goes for the UDP variant. We capture this in packet[2].
To differentiate between TCP and UDP connections (and make it a true (ip source, ip destination, sport, dport, protocol)-tuple), we have to include information about the protocol packet[1]. Finally, these options start counting from 0 for each file that is being parsed, we differentiate these identifiers by the filepath of the analyzed pcap.

In short, (packet[0], packet[1], packet[2]) should be equivalent to (ip source, ip destination, sport, dport, protocol) where we let tshark deal with the gory details of figuring out the direction of communication, but we need to include some extra information to make tshark identifiers truely unique as a flow-tuple.

from flowprint.

Thanks for your detailed reply!
Following your answer, I've looked up the Wireshark manual and found it quite helpful. For those who are also wondering the problem, this link will also be useful:
https://cse.sc.edu/~pokeefe/tutorials/wireshark/ChAdvFollowTCPSection.html

Thanks for the nice explanation:)

from flowprint.