n0-computer / chuck Goto Github PK

• Try 50 node transfers and various mesh setups
• Simulate n0-computer/iroh#1514

minimal repro could be this:
P1 creates a doc + ticket
P2, P3, P4 join
P4 goes offline
P4 adds entries while offline
P4 goes online again (rejoins P1)
check after some timeout that P3 and P2 have the new entries from P4. I would guess that only one of them will have them. If it works, try the same with more peers, then it will fail.

• Diva outlined some good use cases

We rely too much on brittle parsing of output for a large part of metrics. Should move to a more deterministic setup by querying the local nodes for stats directly.

Currently we pull in the latest release, but that means we never test FFI or the latest iroh code with these playbooks.

There's a lot of cruft and stale things which should be removed and cleaned up.

We missed some issues because the behavior of derpers was not identical in netsim and in prod.
Example fix we had to do n0-computer/iroh#2090

We will eventually hit a ceiling with how many nodes we can and want to simulate. Should implement the possibility to peer and coordinate multiple boxes that run in conjunction. This depends on both this being implemented code wise but also infrastructure that supports it.

Netsim and mininet allow for the usage of a remote "controller" node for managing the network stack and flow, the missing pieces are and external infrastructure orchestrator and synchronization engine.

I had some failures in the netsim tests because files in the IROH_DATA_DIR /root/.local/share/iroh/ are being deleted while the test runs.

The flat file database assumes that when it copies files into the IROH_DATA_DIR, they will remain there. So each provide process needs it's own independent IROH_DATA_DIR that is left alone while the provide process is running.

• Time to 1st Byte
• Time to 1st Hole Punch
• Average latencies between peers

Current playbooks are direct configs to netsim which then play out but are not fully configurable. Also, all of the nodes shell out and we don't have direct control over them.

Rough plan is to utilize the new python ffi bindings to write scripts to play out various simulations.

Currently the integration runs only exit with an error if the node fails completely or times out, otherwise it passes even though it reports failed transfer etc.
Integration runs should support validating results and report on those too.