Inefficient indexing about benchmark-wrapper HOT 7 CLOSED

Yup - we need a shared library for all the workloads to inherit. This is on the backlog - if you have cycles, it would be great to see this added!

from benchmark-wrapper.

I do have cycles for this, I just need to update snafu to produce the calculated averages then I can switch to updating it to use py_es_bulk.

from benchmark-wrapper.

*This work would be for fio-wrapper.py, but the py_es_bulk module could be used for all wrappers later on.

from benchmark-wrapper.

First attempt at solving this issue.

https://github.com/acalhounRH/snafu/tree/added_logging_and_py_es_bulk

from benchmark-wrapper.

implemented run_snafu.py and update fio-wrapper to be compatible, last item to resolve to make this fully functional is resolving import errors.

https://github.com/acalhounRH/snafu/tree/added_run_snafu

With the use of run_snafu ripsaw is able to have a single point of accessing snafu and ensures all wrappers are indexing data the same way into elasticsearch. Instead of each x_wrapper script implementing indexing separately, run_snafu handles all indexing, with the expectation that x_wrapper would execute its test, parse its own data, and yield back normalized document/index pairs.

from benchmark-wrapper.

The core of the issue is resolved with PR 38, the remaining amount of work is moving existing Benchmarks over to the new structure.

from benchmark-wrapper.

All workloads use the same indexing bits.

from benchmark-wrapper.