CI testing for VM/CNV about benchmark-wrapper HOT 6 OPEN

@dry923 Just FYI, if you add CNV to the CI environment, we can run fio inside CNV VMs, that's already supported (except for issue 527 which I haven't had time to work on, but it has an easy fix described there).

from benchmark-wrapper.

Hey @bengland2 The hope would be we increase the coverage of SNAFU to cover, BM, VM, Container (Podman/docker), OCP...

from benchmark-wrapper.

+1 for VM/CNV testing, but we should probably look at moving away from python setup.py develop for CI builds and normal usage. That was the main difference between the two in #261.

from benchmark-wrapper.

Issue #261 was caused because of missing init file in the log_wrapper directory. It would still fail on outside of openshift (ie VMs, etc) without that file. benchmark-wrapper is installed as a python package and directories without init file are not included in the package.

Benchmark-wrapper CI uses benchmark-operator, the test for log_wrapper were not merged at that moment, else our CI would have found this issue.

@sarahbx Another way to install benchmark-wrapper is pip install git+https://github.com/cloud-bulldozer/benchmark-wrapper.git but I guess that is similar to git clone and python setup.py develop

from benchmark-wrapper.

Hi @amitsagtani97 :) For VMs in benchmark-operator, we are using pip install git+https://... which behaves as python setup.py install. When running the pod versions of the same tests, they were successful. Looks like pip install -e is being used for the wrapper installs, which behaves the same as python setup.py develop.

from benchmark-wrapper.

@learnitall I wonder how your staged CI would mitigate the need for this?

from benchmark-wrapper.