A distrubuted system implementation aimed to be quick to deploy and simple to use. All it takes to start using it is to write your problem as 3 functions on the master side: split(), combine() and processor().

This works without the worker (slave) nodes having any knowledge that is usually required for an RPC call.

On all the masters and slaves, clone the antnest repository:

git clone https://github.com/mtahmed/antnest.git
cd antnest

On the slaves, add a JSON config file, telling the slaves about how to find masters:

cat > config/$(hostname)-slave-config.json
{
  "masters": [
    {
      "ip": "192.168.0.1",
      "hostname": "master"
    }
  ]
}
^D

NOTE: the $(hostname)-slave-config.json used as the filename for the config file. That's the convention used and that's what the node will look for when it's started up.

Start the slave with the help commands/start-slave.py script:

python commands/start-slave.py

Do the same for the master:

python commands/start-master.py

Then write a job (see below on how to do that) and tell the master to run the job:

python commands/create_job.py -j jobs/reverse_strings.py

The jobs are, by convention, defined in jobs/ directory as python files. reverse_strings.py is one of the sample jobs provided.

• Muhammad Tauqir Ahmad
• [email protected]
• csclub.uwaterloo.ca/~mtahmed

• Add license info.
• Testing