topologymapping / mca Goto Github PK

This is part of the effort of better understanding the code. The type hints will make it easier to understand the code internals.

Create the IPv6ExtHdrRFC3692_style_experiment subclass, that will be used during the IPv6 extended classification step. The extension header should have 8 octets, containing:

• Next header (1 byte), value 59 (which means no next headers, it gets overwritten by the upper layer header);
• Header length (1 byte), value 0 (which means just the first 8 octets);
• Value (2 bytes), value should vary according to the high-entropy values during the extended classification step;
• Padding (4 bytes).

The class should overwrite the next header field of the lower layer header to 253 (RFC3692-style experiment).

After #3, the new extended classification flow is using the high-entropy flow ids (mca/flow_ids module). The task at hand is to replace all usages of sequential flow ids during the MCA flow with the generated high-entropy flow ids. The "flow ids" generated by the Identifier class (e.g. Identifiers's create_new_discovery_flow_id method) are merely indices to the high-entropy flow ids, not the flow ids themselves.

Rafael had a topology simulator that we used to debug the algorithm and quantify the probe savings gained with the optimizations. We should include the simulator in this repository and then add some tests.

Since the code was not tested due to concerns on how the probes would behave in the wild, this is one of the bugs that sneaked in unnoticed. Prepend self to all instances that use the identifiers attribute inside MCA's load_balancer_node_extclass_correctness method.

The ext-hdr-test-packets.py script does not send the test packets when -s (--send-packets) is not provided. Please consider printing the TCP packets for IPv4/IPv6 (considering the provided destination) when -s is not provided, so users could make sure that the packets are correctly being concocted.

In order to increase the safety of our experiments, we'll be replacing RFC3692-style experiment headers with (1) Record Route for IPv4 and (2) Destination Options for IPv6. Consider checking #26 for details on IPv4 implementation and #25 on details for the IPv6 implementation.

For TCP, replace the default (80) with 50000.
For UDP, replace the default (53) with 50001.

As suggested by Ethan Benjamin Katz-Bassett, we should consider using one of the already implemented Extension Headers/Options instead of RFC3692-style experiment headers for safer experiments. Here is the record of the discussion between Cunha and Ethan:

[Ethan] perhaps it is safer to use something more common like Record Route? Can you vary the length of it by varying the length field (not sure that's possible) or by adding variable amounts of padding / no-ops?
[Cunha] That works. We would set random values to the "pointer" field and the first byte of the first IP address. The RFC states that if the pointer is greater than the length, then the RR is full. Seems unlikely that routers would break due to a random pointer value.

After a discussion with @cunha, we came to a close that using the Destination Options extension header should meet our requirements. Below is the RFC schema:

The Hdr Ext Len should be zero, meaning that 8 octets are enough for our purpose. The Options field must have 6 octets, and is a TLV-encoded option.

• Option Type: 1 (PadN);
• Opt Data Len: Should be incrementally modified during the experiment. It is automatically modified by Scapy when Option Data is incrementally set;
• Option Data: For each desired incremental Opt Data Len value, one byte (character) should be added to this field.

It seems the current code sends small int values on packets as flow IDs. We should create a list with high-entropy, random constants in the code and take the values sent on packets from this list (we could, e.g., index the list with the flow ID).

A single list with 256 entries and 16-bit values should work, as Scapy (seems to) do a binary and to take the least-significant bits if fields have less than 16 bits.

Add __pycache__ folder to the ignore list.

As suggested by Ethan Benjamin Katz-Bassett, we should consider using one of the already implemented Extension Headers/Options instead of RFC3692-style experiment headers for safer experiments. Here is the record of the discussion between Cunha and Ethan:

[Ethan] perhaps it is safer to use something more common like Record Route? Can you vary the length of it by varying the length field (not sure that's possible) or by adding variable amounts of padding / no-ops?
[Cunha] That works. We would set random values to the "pointer" field and the first byte of the first IP address. The RFC states that if the pointer is greater than the length, then the RR is full. Seems unlikely that routers would break due to a random pointer value.

After a discussion with @cunha, we came to a close that using the Record Route IP option should meet our requirements. Below is the RFC schema:

• Option Type: 7 (Record Route)
• length: The length in bytes, considering the type, length. pointer and route data fields. Since we need to change just the 4th byte, 7 is enough.
• pointer: The pointer is relative to the option, and the smallest legal value is 4. Since we don't want routers along the path to add their IPs to the record route, we should initialize the pointer to 8.
• route data: a single 4-bytes IP address. The first byte will be incrementally (check with Italo) changed, while the remaining bytes of the IP address will be set to zero.

This single script should accept one parameter:

• destination_ip: The destination of the packet, an IP address (string);

The script should be able to identify whether IPv4 or IPv6 is being used, and send the packets accordingly.

• For IPv4, the script should ping the destination including the option IPOption_RFC3692_style_experiment using a random value on the value field;
• For IPv6, the script should ping the destination including a forged fragmentation header that uses a random value on the offset field.

A payload should be included on all packets pointing to http://www.dcc.ufmg.br/~cunha/hosted/ipv6-exthdr-balancing. The script will be used to assess the behavior of the packet against versions of Windows, Linux, MacOS, iOS, and Android devices in controlled environments.

The mca package currently runs MDA/MCA using the run method of the MCA class with the arguments provided by the user. Among the outputs of the run, there is the discovered topology generated when traversing each TTL, which is an instance of the Topology class.

The task at hand is to implement the run_extended_classification method inside the MCA class, which is an additional classification step to check whether load balancers follow the IPv6 next-header chain or correctly parse IPv4 options. The goal is to assess whether the new set of probes will reach the same answer_ip as the original ones, considering the desired level of confidence.

Nodes in the topology that were previously marked as per-packet during the MCA run should be ignored by this new feature.

After running MDA/MCA, the topology object has the discovered nodes for each TTL. In addition, the flow identifiers used during the discovery process can be accessed by the get_discovery_flow_ids method of the Identifier class, providing the TTL and the answer_ip. Both objects are members of the MCA class.

In order to modify the Forge object of each Probe to take extension headers/options into account without breaking the MCA probing scheme, there should be a new argument to the __init__ of the Probe class called chaining_test (bool). Care must be taken so that the parsing of the replies is also updated to consider IPv6 extension headers and IPv4 options (the flow identifier may change in the encapsulated headers);

For each node in the topology that was not classified as per-packet, resend the modified versions of each probe a given number of times, according to the desired level of confidence, and compare their answer_ips with the original probe used in the MCA run. The classification must be stopped the first time the algorithm finds a answer_ip for one of the new probes that does not match the answer_ip of the original probe.

After a discussion with @cunha regarding the extended classification implementation, it was decided that we are going to use the RFC3692-style experiment IPv4 option during the extended classification step. Below is the default scapy header for an IP option:

where (check RFC791 [page 15 and below] for more details):

• copy_flag: 0 (Don't copy into all fragments of fragmentation);
• optclass: 2 (Debugging and measurement);
• option: 30 (RFC3692-style experiment);
• length: 4 (4 bytes);
• value: 2 bytes that will contain the generated high-entropy values. The value, composed of the 3rd and 4th bytes of this header, can be incorrectly parsed by the load balancer as if it where the upper layer packet destination port, which would (wrongly) affect the load balancing.

Please create a IPOption_RFC3692_style_experiment class to be used during the extended classification step, that being a subclass of the scapy.all.IPOption class. The option will be used to check whether the load balancer is correctly parsing the option. If that is the case, the load balancing shouldn't be affected by the option.