Git Product home page Git Product logo

Comments (3)

IanHoang avatar IanHoang commented on September 13, 2024

Ran into an issue during release process. We were able to release to PyPi and are able to pull in the latest version and confirm it works. However, when we were testing Dockerhub staging, we came across the following error.

benchmark@c5aca43430ba:~$ opensearch-benchmark --help
Traceback (most recent call last):
  File "/usr/local/bin/opensearch-benchmark", line 5, in <module>
    from osbenchmark.benchmark import main
  File "/usr/local/lib/python3.11/site-packages/osbenchmark/benchmark.py", line 37, in <module>
    from osbenchmark import version, actor, config, paths, \
  File "/usr/local/lib/python3.11/site-packages/osbenchmark/test_execution_orchestrator.py", line 33, in <module>
    from osbenchmark import actor, config, doc_link, \
  File "/usr/local/lib/python3.11/site-packages/osbenchmark/worker_coordinator/__init__.py", line 26, in <module>
    from .worker_coordinator import (
  File "/usr/local/lib/python3.11/site-packages/osbenchmark/worker_coordinator/worker_coordinator.py", line 44, in <module>
    from osbenchmark import actor, config, exceptions, metrics, workload, client, paths, PROGRAM_NAME, telemetry
  File "/usr/local/lib/python3.11/site-packages/osbenchmark/workload/__init__.py", line 25, in <module>
    from .loader import (
  File "/usr/local/lib/python3.11/site-packages/osbenchmark/workload/loader.py", line 41, in <module>
    from osbenchmark.workload import params, workload
  File "/usr/local/lib/python3.11/site-packages/osbenchmark/workload/params.py", line 42, in <module>
    from osbenchmark.utils.dataset import DataSet, get_data_set, Context
  File "/usr/local/lib/python3.11/site-packages/osbenchmark/utils/dataset.py", line 13, in <module>
    import h5py
  File "/usr/local/lib/python3.11/site-packages/h5py/__init__.py", line 25, in <module>
    from . import _errors
  File "h5py/_errors.pyx", line 1, in init h5py._errors
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

We were able to discover the root cause was due to the recent numpy 2.0.0 release and how it conflicts with h5py 3.10.0. Our local desktops did not encounter this issue b, the Docker container only had Python 3.11 installed and needed to pull in numpy, which is why it pulled in the latest version.

We can either solve this either by restricting the setup.py to use numpy versions less than 2.0.0 or we can remove the restriction for h5py in this line of the dockerfile

RUN python3 -m pip install h5py==3.10.0; if [ -z "$VERSION" ] ; then python3 -m pip install opensearch-benchmark ; else python3 -m pip install opensearch-benchmark==$VERSION ; fi

from opensearch-benchmark.

IanHoang avatar IanHoang commented on September 13, 2024

Since this impacted the Docker image, we will go ahead and do a copy-over. We'll re-release with a patch version.

from opensearch-benchmark.

rishabh6788 avatar rishabh6788 commented on September 13, 2024

I think rather than bumping to a major version let us pin the numpy to last released 1.x version, which is 1.26.4.

from opensearch-benchmark.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.