snlab-freedom / sdntest Goto Github PK

Currently, we specify "workflow" in "config.json".

For general purpose, we can allow users to set extra arguments in the config file. But consider controller ip as the first argument by default.

Current implementation strongly depends on docker containers. But refer to moby/moby#7857, network performance is so poor in the docker containers. Even in the host mode, docker network is still with higher delay.

On potential solution is to support to execute test jobs on the physical machines. Another one may be cluster mode (swarm) for a parallel test.

bootstrap_mininet() in sdntest/suite.py need to read self.controller.attrs to get the ip address of the controller container. But it seems that this variable is not updated so that mininet cannot get the latest information.

Since each TestSuite() instance is independent, framework should support to execute testcases parallelly.

One potential solution is to specify the parallel size in config file like this:

{
    "...": "...",
    "workflow": "main.py",
    "repeat": 12,
    "parallel": 4
}

It will create 4 threads to run testcases parallelly. Each thread can repeat the testcases by 3 times.

Docker python APIs can get and maintain the docker images and containers easily. In the current implementation, the python version script (bin/sdntest) has been ahead of the bash version (bin/sdntest.sh).

Maybe we can deprecate the bash version script. The only consideration is that some minimal working environment (like CoreOS) may not have python runtime.

Any thoughts?