Stress tests and integration tests about openbazaar-go HOT 8 CLOSED

but little effort was done to find and understand the bugs that are subtle, hard-to-reproduce, take time to trigger, affect only some platforms, etc.

Of course, that isn't true. I spent hours upon hours staring at the code trying to track down memory and cpu issues. You need to consider as well that it was basically just me writing/debugging that code on strict time deadlines, while also working on this code base at the same time.

Thankfully, I'm not the only person working on it anymore. If you include the ipfs team, we have four or five full time developers working on the codebase. So it should be much improved.

Please feel free to contribute resources as well if you want to build on top of it.

I do agree with #2 though and have voiced my opinion on that.

from openbazaar-go.

I spent hours upon hours staring at the code trying to track down memory and cpu issues.

@cpacia Ok, understood. My suggestion for OB2.0 is to invest time in a stress testing framework. It will make it much easier to find bugs than staring at the code. I'm happy to contribute to that. It can be something easy like "instantiate n nodes and make them send a random message to x% of the nodes every m seconds". My guess is that bugs will start manifesting themselves in minutes, bypassing hours of staring.

from openbazaar-go.

This is something that came up in one of our team meetings the other day. Do you have any scripts already that we can use as a baseline or process recommendations here? I was thinking about a few tools I've used in the past but hadn't started to run with this yet.

from openbazaar-go.

@hoffmabc Ideally we want to test 100+ nodes communicating with each other at high speed, submitting listings, etc. More likely than not, we're going to need a cluster of machines for that. I believe the most professional way of doing that is with Docker Swarm and Docker Machine (see here. My preference would be to load up DigitalOcean droplets.

Here's a possible roadmap:

1. Figure out how to instantiate a single IPFS node (I'm still at this stage!)
2. Get two IPFS nodes on a single machine to talk to each other. Write a few stress tests between these two nodes.
3. Understand the max number of test IPFS nodes that can fit in a droplet (say n) and build a docker container for that.
4. Get two droplets (each with n nodes from the container) to talk to each other and write more stress tests
5. Automate the provision of droplets with Docker Machine and Docker Swarm to get to 100+ nodes in 3 different continents
6. Use the above infrastructure as a basis for a public testnet?

from openbazaar-go.

@cpacia have we gotten #2 to work yet?

from openbazaar-go.

Should just be a matter of using two different repo directories.

from openbazaar-go.

btw here's for tracking ipfs test coverage ipfs/kubo#3053

from openbazaar-go.

I created the qa package to build out integration tests.

from openbazaar-go.