Comments (4)
I've been super caught up in #299, as well some other changes for migrating benchmark-wrapper over to the more object-orientated structure. @Maxusmusti do you want to get started on the design of the protocol and we can touch base on implementation?
from benchmark-wrapper.
Create functionality that allows for benchmarks to understand how many consumers have acknowledged a published signal.
I think this is a mis-specification: I think the requirement is for the SignalExporter
class to be able to synchronize the signal export with the subscribers. In order to do this, the class will, among other things, need to know how many subscribers there are (so that it can determine when they've all acknowledged the signal), but this is an implementation detail, not an architectural requirement (synchronization could, for instance, be implemented in other ways).
---o--O--o---
The first item needed is an architecture and protocol description laying out how benchmark runners like Benchmark-wrapper can publish event notifications to subscribers like Pbench. (There should be no expectation that Benchmark-wrapper will be the only runner to use this architecture, and there should be no expectation that Pbench will be the only subscriber.)
Items which need to be outlined include:
- The list of events which can be signaled, including but not limited to:
- benchmark initialization
- benchmark start
- iteration start
- iteration stop
- benchmark stop
- Payload details for each event and example responses to each event
- Capabilities for synchronization between the publisher and the subscribers
- Capabilities for publishing non-signal information, such as benchmark and/or iteration metadata
- Capabilities for non-transient information (e.g., a log of events and other state changes, to allow late-coming clients to "catch up")
And then a more functional set of descriptions is needed:
- How the publisher and subscribers locate the Redis server. (Presumably they acquire the IP address and port via configuration arguments on the command line, etc.)
- What channels on the Redis server are used, and how they located/named. (With due care given to support for sharing a single Redis server between multiple benchmark instances, which will presumably require a prefix obtained from configuration arguments.)
- How non-signal information is conveyed and accessed; ditto for non-transient information. (How will these be initialized and finalized, given asynchronous operation of the server, publisher, and subscribers?)
- How synchronization works between the publisher and subscribers. (How the subscribers acknowledge a signal, and how the publisher waits until all ACKs are received; how the publisher handles NAKs or missing ACKs.)
from benchmark-wrapper.
I think this is a mis-specification: I think the requirement is for the SignalExporter class to be able to synchronize the signal export with the subscribers.
s/Create functionality that allows for benchmarks to understand how many consumers have acknowledged a published signal./Create functionality within the SignalExporter class which benchmarks can use to understand if consumers are ready for the benchmark to begin./
There should be no expectation that Benchmark-wrapper will be the only runner to use this architecture
Why is this? I understand this:
there should be no expectation that Pbench will be the only subscriber.
But the only use case that the cloud-bulldozer org has for this protocol is for exporting state from benchmark-wrapper (correct me if I'm wrong here @jtaleric @rsevilla87). If there are going to be other tools which use this protocol then shouldn't it live outside of benchmark-wrapper? Otherwise hosting it within benchmark-wrapper seems arbitrary. I don't think it really matters anyways either way, but just curious as to why we have this expectation when we've only talked about the protocol within the scope of benchmark-wrapper.
I think your included architecture requirements and functional description requirements are spot on, I'll get to work on filling those out.
from benchmark-wrapper.
I've been super caught up in #299, as well some other changes for migrating benchmark-wrapper over to the more object-orientated structure. @Maxusmusti do you want to get started on the design of the protocol and we can touch base on implementation?
Working on design doc here
from benchmark-wrapper.
Related Issues (20)
- Add CLI Option to Purge Empty Fields in ES
- Determine and Implement Versioning Schema
- ElasticSearch Unified Index HOT 3
- Addition of Collectors and Sample Abstraction Request HOT 2
- Extend Common Benchmark Fields HOT 4
- [Proposal] Index Environment Metadata for every benchmark HOT 4
- Quay trigger sometimes fails HOT 2
- snafu base image HOT 16
- 2 things I don't like about current cache dropping HOT 1
- backlog of ES results results in failure to get yielded results to ES?
- Inconsistent Benchmark and Quay Image Names HOT 10
- Can the grafana dashboard metric percentiles of bandwidth/latency in the fio ES index can use compare different run? HOT 5
- run_snafu does not protect against bad JSON data in certain failures HOT 2
- FIO and SF failed to run HOT 8
- Uperf test fails but does not exit with error HOT 2
- Elasticsearch: credential authentication HOT 5
- centos image resources .repo points to wrong URL
- scale_openshift_wrapper should expose parameters to target machineset from different labels HOT 1
- uperf Docker Image creation fails on ppc64le HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from benchmark-wrapper.