uperf Docker Image creation fails with dependency issues on ppc64le about benchmark-wrapper HOT 5 CLOSED

Update - I did manage to eventually build the uperf image on ppc64le.
What I did additionally to make it happen -

• Install a couple of rpm packages before scipy and numpy from the Dockerfile - atlas-devel libffi-devel gcc gcc-c++ gcc-gfortran libffi-devel make openssl-devel (added them in line 3 of the Dockerfiles) - this seems to be a known requirement for ppc64le in general for both scipy & numpy
• I had to add this to the Dockerfile as the same somehow doesn't work by adding as dependency in setup.cfg - RUN python3 -m pip install --upgrade cython numpy
• I had to add a couple of python packages to setup.cfg - importlib_metadata & urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (version clash and eventual failure while running workload if version specifics not added)

Since this requirement is specific to ppc64le systems only, can we probably add this info as a small Note in the README doc somewhere for future reference?

from benchmark-wrapper.

Update - I did manage to eventually build the uperf image on ppc64le.
What I did additionally to make it happen -

• Install a couple of rpm packages before scipy and numpy from the Dockerfile - atlas-devel libffi-devel gcc gcc-c++ gcc-gfortran libffi-devel make openssl-devel (added them in line 3 of the Dockerfiles) - this seems to be a known requirement for ppc64le in general for both scipy & numpy
• I had to add this to the Dockerfile as the same somehow doesn't work by adding as dependency in setup.cfg - RUN python3 -m pip install --upgrade cython numpy
• I had to add a couple of python packages to setup.cfg - importlib_metadata & urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (version clash and eventual failure while running workload if version specifics not added)

Since this requirement is specific to ppc64le systems only, can we probably add this info as a small Note in the README doc somewhere for future reference?

Interesting on the ppc64 specific needs. I wonder if we can have a uperf Dockerfile that is ppc64 specific?

Would you like to do a PR against benchmark-wrapper to include this ppc64. And if you have a container image, maybe you can share that, so I can push to Quay?

from benchmark-wrapper.

@jtaleric , yes I will raise the PR for ppc64le Dockerfiles. In my opinion all dockerfiles except iperf will need these additions to be ppc64le compliant.
So far I've created 4 images for ppc64le -

• quay.io/piyushgupta1551/benchmark-operator:master (this needs recreation after yesterday's uperf sa issue. The template needs to be updated)
• quay.io/piyushgupta1551/uperf:master
• quay.io/piyushgupta1551/iperf:master
• quay.io/piyushgupta1551/fio:master

How do you want me to add in the python packages that I added in setup.cfg? Should I add them in the Dockerfiles directly? Adding them in setup.cfg might create a conflict as they are not really required for x86.

from benchmark-wrapper.

Hi @jtaleric @vasukulkarni
Below are the images that could be created for benchmark-operator and benchmark-wrapper workloads for ppc64le arch -

quay.io/piyushgupta1551/benchmark-operator:ppc64le
quay.io/piyushgupta1551/iperf3-m:ppc64le
quay.io/piyushgupta1551/fs-drift-m:ppc64le
quay.io/piyushgupta1551/fio-m:ppc64le
quay.io/piyushgupta1551/ycsb-m:ppc64le
quay.io/piyushgupta1551/smallfile-m:ppc64le
quay.io/piyushgupta1551/upgrade-m:ppc64le
quay.io/piyushgupta1551/scale-openshift-m:ppc64le
quay.io/piyushgupta1551/uperf-m:ppc64le

The remaining (oslat, hammerdb, sysbench, pgbench, cyclictest, vegeta) could not be created as dependent rpms, installers etc were not available for the required distros for ppc64le arch.

from benchmark-wrapper.

Open Issues for ppc64le workload images -

• oslat - rt-tests-1.8-11.el8.x86_64.rpm not available for ppc64le for the required distro (Only OpenSUSE available)

• hammerdb - https://github.com/TPC-Council/HammerDB/releases/download/v3.3/HammerDB-3.3-Linux-x86-64-Install' not available for ppc64le

• sysbench - rpm not available for ppc64le for the required distro

• stress-ng - not available for the required distro for ppc64le

• pgbench - https://download.postgresql.org/pub/repos/yum/reporpms/EL-8-x86_64/pgdg-redhat-repo-latest.noarch.rpm - ppc64le available only for EL7

• cyclictest - rt-tests-1.8-11.el8.x86_64.rpm available only for OpenSUSE distro for ppc64le

• vegeta - github release not available for ppc64le

FYI @vasukulkarni

from benchmark-wrapper.