randombangaloreguy / libflowmanager Goto Github PK

Libflowmanager - A library for matching packets from trace files to TCP and
UDP flows

Libflowmanager is Copyright (c) 2009-2012, 2016 The University of Waikato,
Hamilton, New Zealand. All rights reserved.

See the files COPYING and COPYING.LESSER for full licensing details for this
software.

Report any bugs, questions or comments to [email protected]

Introduction
============
Libflowmanager was designed to perform all of the common tasks that are
required when performing measurement tasks at the flow-level. In particular, it 
maintains a table of active flows and exports an API for matching packets to
existing flows (or creating new flows, if required). It also deals sensibly
with flow expiration, ensuring that flows are expired after an appropriate 
amount of idle time.

Libflowmanager also comes bundled with an API for reordering TCP packets based
on sequence number. Trace files typically record packets chronologically, 
which means that packets often will be read from the trace file out-of-order.
However, many TCP measurement applications become difficult if the packets are
not in strict sequence number order. This is where the TCP reordering API comes
in. 


Required Libraries
==================
libtrace 3.0.6 or better
	* available from http://research.wand.net.nz/software/libtrace.php

Installation
============
After having installed the required libraries, running the following series of
commands should install libflowmanager:

	./configure
	make
	make install

By default, libflowmanager installs to /usr/local/ - this can be changed by
appending the --prefix=<new location> option to ./configure.


Libflowmanager API Notes
========================

  * Start by creating a new instance of the FlowManager class, e.g.
      fm = new FlowManager()

  * Before reading any packets, use fm->setConfigOption() to set any
    configuration options.

  * For each packet perform the following tasks in the specified order:
  	+ Call fm->matchPacketToFlow() on the packet to get a pointer to 
	  the flow object that the packet matches. 
	+ Call fm->updateFlowExpiry() to update the TCP state and expiry time
          for the current flow
	+ Repeatedly call fm->expireNextFlow() until all expired flows have
	  been returned
	
  * The flow object contains a void pointer called "extension". This is
    designed for the user to store their own per-flow data. Memory should be
    allocated and initialised for "extension" data whenever the "is_new_flow"
    flag is set to true by matchPacketToFlow().

    For example, if I want to store packet and byte counts for each flow, I
    would allocate a structure containing two counters for each new flow and 
    store a pointer to that structure inside "extension". Everytime that flow 
    is subsequently returned by matchPacketToFlow(), I can cast 
    the void pointer "extension" to my structure and update the counters.
    
  * You are responsible for freeing any "extension" data belonging to a flow
    when that flow expires.

  * Similarly, fm->releaseFlow() MUST be called for any flow object returned by 
    expireNextFlow(). It has been removed from the active flows table
    but the memory is not freed because libflowmanager assumes that you might
    want to do something with the flow itself upon expiry.
  
  * Some API functions require a direction parameter to indicate whether the 
    packet was incoming or outgoing. We leave this up to the user to define
    direction because a) trace_get_direction() only works for some trace
    formats and b) the user may want to define direction differently anyway.

  * When accessing the IP addresses and ports from the flow Id, you now have
    two options. You can use the get_server_* and get_client_* API functions
    which have been present in previous versions of libflowmanager or you can
    use the get_local_* and get_external_* API functions which were newly 
    added in 2.0.3.

    In this context, we use the following definitions:
    	server - the host that was connected to when the flow first began, i.e.
		 the recipient of the first packet.
	client - the host that initiated the flow, i.e. the sender of the first
		 packet.
	local - the host that is the transmitter of "outgoing" packets, where
		outgoing is defined as having a direction value of 0.
	external - the host that is the transmitter of "incoming" packets,
		   where incoming is defined as having a direction value of 1.

TCP Reordering API Notes
========================

  * Each flow requires two TCP reorderers, one for each direction. If you
    are using libflowmanager to manage the flows themselves, you will want to
    store your TCP reorderers inside the "extension" variable described above.

  * Each reorderer needs to be created with tcp_create_reorderer() and
    should be destroyed using trace_destroy_reorderer when the flow is expired.

  * Each reorderer requires two callback functions - a read callback and a
    destroy callback. The read callback will be called by the reorderer every
    time a packet is inserted into the reordering list. The destroy callback
    to tidy up after any packets left in the reordering list when a reorderer
    is destroyed, i.e. if the flow expires while there are still gaps in the
    packet sequence.
    
  * The read callback should take a libtrace packet and extract only the 
    information your analysis requires from the packet. It should return a
    pointer to an allocated structure containing your required data. This
    pointer can then be accessed when the packet has been reordered so you
    can access your data.

  * The destroy callback must free any memory that was allocated during the
    read callback, including the structure that was used to store all the 
    data.

  * Every TCP packet should be passed to tcp_reorder_packet().

  * If the return value of tcp_reorder_packet() did not recommend ignoring the
    packet, repeatedly call tcp_pop_packet() until there are no packets 
    available to pop from the reordering list.
  
  * You will need to manually call your destroy callback on the "data"
    pointer in the reordered packets returned by tcp_pop_packet(). 

  * free() MUST also be called on the reordered packets themselves, i.e. the
    tcp_packet_t, returned by tcp_pop_packet() once you have finished with
    them.