Comments (3)
Ran into an issue during release process. We were able to release to PyPi and are able to pull in the latest version and confirm it works. However, when we were testing Dockerhub staging, we came across the following error.
benchmark@c5aca43430ba:~$ opensearch-benchmark --help
Traceback (most recent call last):
File "/usr/local/bin/opensearch-benchmark", line 5, in <module>
from osbenchmark.benchmark import main
File "/usr/local/lib/python3.11/site-packages/osbenchmark/benchmark.py", line 37, in <module>
from osbenchmark import version, actor, config, paths, \
File "/usr/local/lib/python3.11/site-packages/osbenchmark/test_execution_orchestrator.py", line 33, in <module>
from osbenchmark import actor, config, doc_link, \
File "/usr/local/lib/python3.11/site-packages/osbenchmark/worker_coordinator/__init__.py", line 26, in <module>
from .worker_coordinator import (
File "/usr/local/lib/python3.11/site-packages/osbenchmark/worker_coordinator/worker_coordinator.py", line 44, in <module>
from osbenchmark import actor, config, exceptions, metrics, workload, client, paths, PROGRAM_NAME, telemetry
File "/usr/local/lib/python3.11/site-packages/osbenchmark/workload/__init__.py", line 25, in <module>
from .loader import (
File "/usr/local/lib/python3.11/site-packages/osbenchmark/workload/loader.py", line 41, in <module>
from osbenchmark.workload import params, workload
File "/usr/local/lib/python3.11/site-packages/osbenchmark/workload/params.py", line 42, in <module>
from osbenchmark.utils.dataset import DataSet, get_data_set, Context
File "/usr/local/lib/python3.11/site-packages/osbenchmark/utils/dataset.py", line 13, in <module>
import h5py
File "/usr/local/lib/python3.11/site-packages/h5py/__init__.py", line 25, in <module>
from . import _errors
File "h5py/_errors.pyx", line 1, in init h5py._errors
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
We were able to discover the root cause was due to the recent numpy 2.0.0 release and how it conflicts with h5py 3.10.0. Our local desktops did not encounter this issue b, the Docker container only had Python 3.11 installed and needed to pull in numpy, which is why it pulled in the latest version.
We can either solve this either by restricting the setup.py to use numpy versions less than 2.0.0 or we can remove the restriction for h5py in this line of the dockerfile
RUN python3 -m pip install h5py==3.10.0; if [ -z "$VERSION" ] ; then python3 -m pip install opensearch-benchmark ; else python3 -m pip install opensearch-benchmark==$VERSION ; fi
from opensearch-benchmark.
Since this impacted the Docker image, we will go ahead and do a copy-over. We'll re-release with a patch version.
from opensearch-benchmark.
I think rather than bumping to a major version let us pin the numpy to last released 1.x version, which is 1.26.4
.
from opensearch-benchmark.
Related Issues (20)
- [META] Enhance OpenSearch Benchmark Documentation HOT 2
- Support Multiple Search Client Settings in a Single OSB Run
- OSB build scripts need to be unified HOT 1
- [META] Create-Workload Enhancements
- [META] Synthetic data corpus generator
- Red-line performance testing
- Support for serverless implementations
- Manual approval required for workflow run 10511695101: Release opensearch-benchmark HOT 2
- OSB Release Calendar HOT 1
- [RFC]: Introducing Aggregation, Enhanced Comparison, and JSON Export for OSB HOT 1
- [META] Introduce Aggregate Command
- [Introduce Aggregate Command] Add Ability to aggregate results across multiple OSB test executions
- [Introduce Aggregate Subcommand] Enhance opensearch-benchmark `compare` command
- [Introduce Aggregate Command] Incorporate Automatic Aggregation
- [BUG] OSB hangs when runs carried out with large number of clients and target-throughput enabled
- [Bug]: OSB Failing Intermittently on Elasticsearch 7.10 - OSB 1.8.0
- [Bug]: OSB 1.9.0 Docker Image fails when mounting file HOT 1
- Add short-hand alternatives for commonly-used command line options
- Update to release from feature branches rather than main
- Blog post on OSB
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opensearch-benchmark.