topologymapping / mca Goto Github PK
View Code? Open in Web Editor NEWImplementation of the Multipath Classification Algorithm
License: GNU General Public License v3.0
Implementation of the Multipath Classification Algorithm
License: GNU General Public License v3.0
This is part of the effort of better understanding the code. The type hints will make it easier to understand the code internals.
Create the IPv6ExtHdrRFC3692_style_experiment
subclass, that will be used during the IPv6 extended classification step. The extension header should have 8 octets, containing:
The class should overwrite the next header
field of the lower layer header to 253 (RFC3692-style experiment).
After #3, the new extended classification flow is using the high-entropy flow ids (mca/flow_ids
module). The task at hand is to replace all usages of sequential flow ids during the MCA flow with the generated high-entropy flow ids. The "flow ids" generated by the Identifier
class (e.g. Identifiers
's create_new_discovery_flow_id
method) are merely indices to the high-entropy flow ids, not the flow ids themselves.
Rafael had a topology simulator that we used to debug the algorithm and quantify the probe savings gained with the optimizations. We should include the simulator in this repository and then add some tests.
Since the code was not tested due to concerns on how the probes would behave in the wild, this is one of the bugs that sneaked in unnoticed. Prepend self
to all instances that use the identifiers
attribute inside MCA
's load_balancer_node_extclass_correctness
method.
The ext-hdr-test-packets.py
script does not send the test packets when -s
(--send-packets
) is not provided. Please consider printing the TCP packets for IPv4/IPv6 (considering the provided destination) when -s
is not provided, so users could make sure that the packets are correctly being concocted.
For TCP, replace the default (80) with 50000.
For UDP, replace the default (53) with 50001.
As suggested by Ethan Benjamin Katz-Bassett, we should consider using one of the already implemented Extension Headers/Options instead of RFC3692-style experiment headers for safer experiments. Here is the record of the discussion between Cunha and Ethan:
[Ethan] perhaps it is safer to use something more common like Record Route? Can you vary the length of it by varying the length field (not sure that's possible) or by adding variable amounts of padding / no-ops?
[Cunha] That works. We would set random values to the "pointer" field and the first byte of the first IP address. The RFC states that if the pointer is greater than the length, then the RR is full. Seems unlikely that routers would break due to a random pointer value.
After a discussion with @cunha, we came to a close that using the Destination Options extension header should meet our requirements. Below is the RFC schema:
The Hdr Ext Len
should be zero, meaning that 8 octets are enough for our purpose. The Options
field must have 6 octets, and is a TLV-encoded
option.
Option Type
: 1 (PadN
);Opt Data Len
: Should be incrementally modified during the experiment. It is automatically modified by Scapy when Option Data
is incrementally set;Option Data
: For each desired incremental Opt Data Len
value, one byte (character) should be added to this field.It seems the current code sends small int
values on packets as flow IDs. We should create a list with high-entropy, random constants in the code and take the values sent on packets from this list (we could, e.g., index the list with the flow ID).
A single list with 256 entries and 16-bit values should work, as Scapy (seems to) do a binary and
to take the least-significant bits if fields have less than 16 bits.
Add __pycache__
folder to the ignore list.
As suggested by Ethan Benjamin Katz-Bassett, we should consider using one of the already implemented Extension Headers/Options instead of RFC3692-style experiment headers for safer experiments. Here is the record of the discussion between Cunha and Ethan:
[Ethan] perhaps it is safer to use something more common like Record Route? Can you vary the length of it by varying the length field (not sure that's possible) or by adding variable amounts of padding / no-ops?
[Cunha] That works. We would set random values to the "pointer" field and the first byte of the first IP address. The RFC states that if the pointer is greater than the length, then the RR is full. Seems unlikely that routers would break due to a random pointer value.
After a discussion with @cunha, we came to a close that using the Record Route IP option should meet our requirements. Below is the RFC schema:
Option Type
: 7 (Record Route)length
: The length in bytes, considering the type
, length
. pointer
and route data
fields. Since we need to change just the 4th byte, 7 is enough.pointer
: The pointer is relative to the option, and the smallest legal value is 4. Since we don't want routers along the path to add their IPs to the record route, we should initialize the pointer
to 8.route data
: a single 4-bytes IP address. The first byte will be incrementally (check with Italo) changed, while the remaining bytes of the IP address will be set to zero.This single script should accept one parameter:
destination_ip
: The destination of the packet, an IP address (string);The script should be able to identify whether IPv4 or IPv6 is being used, and send the packets accordingly.
IPOption_RFC3692_style_experiment
using a random value on the value
field;offset
field.A payload should be included on all packets pointing to http://www.dcc.ufmg.br/~cunha/hosted/ipv6-exthdr-balancing. The script will be used to assess the behavior of the packet against versions of Windows, Linux, MacOS, iOS, and Android devices in controlled environments.
The mca
package currently runs MDA/MCA using the run
method of the MCA
class with the arguments provided by the user. Among the outputs of the run, there is the discovered topology generated when traversing each TTL, which is an instance of the Topology
class.
The task at hand is to implement the run_extended_classification
method inside the MCA
class, which is an additional classification step to check whether load balancers follow the IPv6 next-header chain or correctly parse IPv4 options. The goal is to assess whether the new set of probes will reach the same answer_ip
as the original ones, considering the desired level of confidence.
Nodes in the topology that were previously marked as per-packet
during the MCA run should be ignored by this new feature.
After running MDA/MCA
, the topology object has the discovered nodes for each TTL
. In addition, the flow identifiers used during the discovery process can be accessed by the get_discovery_flow_ids
method of the Identifier
class, providing the TTL
and the answer_ip
. Both objects are members of the MCA
class.
In order to modify the Forge
object of each Probe
to take extension headers/options into account without breaking the MCA probing scheme, there should be a new argument to the __init__
of the Probe
class called chaining_test
(bool
). Care must be taken so that the parsing of the replies is also updated to consider IPv6 extension headers and IPv4 options (the flow identifier may change in the encapsulated headers);
For each node in the topology that was not classified as per-packet
, resend the modified versions of each probe a given number of times, according to the desired level of confidence, and compare their answer_ip
s with the original probe used in the MCA run. The classification must be stopped the first time the algorithm finds a answer_ip
for one of the new probes that does not match the answer_ip
of the original probe.
After a discussion with @cunha regarding the extended classification implementation, it was decided that we are going to use the RFC3692-style experiment IPv4 option during the extended classification step. Below is the default scapy header for an IP option:
where (check RFC791 [page 15 and below] for more details):
Please create a IPOption_RFC3692_style_experiment
class to be used during the extended classification step, that being a subclass of the scapy.all.IPOption
class. The option will be used to check whether the load balancer is correctly parsing the option. If that is the case, the load balancing shouldn't be affected by the option.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.