arcalot / arcaflow-plugins-incubator Goto Github PK
View Code? Open in Web Editor NEWUnofficial test and under-development plugins for Arcaflow
License: Apache License 2.0
Unofficial test and under-development plugins for Arcaflow
License: Apache License 2.0
Uperf's README could use some more explanation on using it in CLI mode.
This plugin (or plugins) should be able to run the benchmark-operator and should be able to accept an input schema that allows for parameters to be passed as a CR to the operator for workload execution.
This feature will enable the Arcaflow engine to serve better in drop-in replacement scenarios for existing workflows that leverage the benchmark-operator.
Instead of including a .gitignore in every plugin, we should define one per language.
Avoid duplication.
The .gitignore file should be self-created and not copied off the Internet to avoid possible copyright issues.
We need a plugin that is able to look up which cloud provider instance a specific OpenShift node corresponds to using the machine objects.
We discussed that it would be a good idea to require using docker-compose to run an end-to-end test.
It would setup all components, and the exit code of each or a single container can be used to determine pass or fail.
It will also be used to show an example of how to use the plugin.
In the future, there can be an automation that goes through all the folders, tries to build all the containers in there, then runs the docker-compose to make sure it runs properly.
The use case is testing and example code.
uperf plugin builds correctly with podman-compose
but fails to build with docker-compose
.
With podman-compose
:
...
[2/2] COMMIT arcaflow-uperf
--> cc90295e6c9
Successfully tagged localhost/arcaflow-uperf:latest
cc90295e6c9408cd43612b8a4d86e1fb4c99abc5147c1476d7b064f5034a7bc9
exit code: 0
$ podman images
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost/arcaflow-uperf latest cc90295e6c94 10 seconds ago 631 MB
With docker-compose
:
Building server
[+] Building 8.4s (9/18)
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 37B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for quay.io/centos/centos:stream8 0.9s
=> [stage-1 1/11] FROM quay.io/centos/centos:stream8@sha256:de7327910e7 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 147B 0.0s
=> CACHED [stage-1 2/11] RUN dnf module -y install python39 && dnf inst 0.0s
=> CACHED [builder 2/4] RUN dnf install -y git make gcc lksctp-tools-dev 0.0s
=> CACHED [builder 3/4] RUN git clone -b 1.0.7 https://github.com/uperf/ 0.0s
=> ERROR [builder 4/4] RUN cd /uperf && ./configure && make && make inst 7.3s
------
> [builder 4/4] RUN cd /uperf && ./configure && make && make install:
...
#8 7.231 CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /uperf/missing aclocal-1.16
#8 7.234 /uperf/missing: line 81: aclocal-1.16: command not found
#8 7.238 WARNING: 'aclocal-1.16' is missing on your system.
#8 7.238 You should only need it if you modified 'acinclude.m4' or
#8 7.238 'configure.ac' or m4 files included by 'configure.ac'.
#8 7.238 The 'aclocal' program is part of the GNU Automake package:
#8 7.238 <https://www.gnu.org/software/automake>
#8 7.238 It also requires GNU Autoconf, GNU m4 and Perl in order to run:
#8 7.238 <https://www.gnu.org/software/autoconf>
#8 7.238 <https://www.gnu.org/software/m4/>
#8 7.238 <https://www.perl.org/>
#8 7.239 make: *** [Makefile:329: aclocal.m4] Error 127
------
executor failed running [/bin/sh -c cd /uperf && ./configure && make && make install]: exit code: 2
ERROR: Service 'server' failed to build : Build failed
$ uname -a
Darwin jlema-mac 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10 14:25:27 PDT 2022; root:xnu-8020.141.5~2/RELEASE_X86_64 x86_64
$ docker version
Client:
Cloud integration: v1.0.25
Version: 20.10.16
API version: 1.41
Go version: go1.17.10
Git commit: aa7e414
Built: Thu May 12 09:20:34 2022
OS/Arch: darwin/amd64
Context: default
Experimental: true
Server: Docker Desktop 4.9.0 (80466)
Engine:
Version: 20.10.16
API version: 1.41 (minimum version 1.12)
Go version: go1.17.10
Git commit: f756502
Built: Thu May 12 09:15:42 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.4
GitCommit: 212e8b6fa2f44b9c21b2798135fc6fb7c53efc16
runc:
Version: 1.1.1
GitCommit: v1.1.1-0-g52de29d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
We would like a new plugin for HP's netperf benchmark utility.
Netperf is a valuable network benchmarking tool that may prove useful in cases similar to uperf.
It seems that netperf is not well maintained. There are release tags, but the most recent one is from 2015. There are main branch updates as recently at 2021. It may be best to align the plugin to the commit hash of what is currently the latest commit.
It also might be useful to use or integrate the super-netperf script features from @jtaleric
Smallfile currently uses a tag to clone a release, but tags can move. This could be problematic, so change the method to pull the release tarball instead.
We need a plugin that, on demand, is able to perform actions on Azure instances.
Actions:
The plugin should be able to select based on:
We need a plugin that, on demand, deletes a namespace and checks for recovery. This plugin should provide two steps, one for deletion and one for timed recovery check. The deletion should be done based on the namespace name (regexp) and/or a label selector.
$ python test_fio_plugin.py
E.
======================================================================
ERROR: test_functional_success (__main__.FioPluginTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "~/arcaflow-plugins/python/fio/test_fio_plugin.py", line 47, in test_functional_success
with open("fio-plus.json", "r") as fio_output_file:
FileNotFoundError: [Errno 2] No such file or directory: 'fio-plus.json'
----------------------------------------------------------------------
Ran 2 tests in 0.010s
FAILED (errors=1)
Try running test for fio plugin.
$ pip list
Package Version
------------------- -------
arcaflow-plugin-sdk 0.7.0
attrs 22.1.0
Automat 20.2.0
build 0.8.0
constantly 15.1.0
hyperlink 21.0.0
idna 3.3
incremental 21.3.0
numpy 1.23.2
packaging 21.3
pep517 0.13.0
pip 22.2.2
pyparsing 3.0.9
PyYAML 6.0
scipy 1.9.1
setuptools 50.3.2
six 1.16.0
tomli 2.0.1
Twisted 22.4.0
typing_extensions 4.3.0
zope.interface 5.4.0
We need a plugin that, on demand, is able to perform actions on OpenStack instances.
Actions:
The plugin should be able to select based on:
We need a plugin that, on demand, is able to perform actions on GCP instances.
Actions:
The plugin should be able to select based on:
This is inspired by and should be index-compatible with the cloud-bulldozer metadata-collector and stockpile tools.
SUT (system under test) metadata is collected with stockpile using ansible facts. Collected metadata is formatted to be indexed alongside benchmark data from other plugins.
This plugin work will need to coordinate with #12 and #13 to handle the ES indexing and compatibility.
Plugin should be able to take input from the proposed general-use ES plugin #12 and post-process it into a document format that is compatible with existing ES indexes used by the team.
In order for our workflow to serve as a drop-in replacement for existing systems, it is important that the output we generate is compatible with our existing data retention efforts.
We are seeing a good amount of divergence across the plugins as we take slightly different approaches to organizing files. We should decide on some more standardization, incorporate those standards into the template example plugins, and update documentation to make it clear that the standards are expected.
Anyone coming to our official repos should be able to browse through a large number of supported plugins and easily understand the organization and function of files. Adhering to standardization will significantly lower the barrier to entry for new users and contributors.
We need a plugin that, on demand, is able to perform actions on bare metal instances using IPMI.
Actions:
The plugin should be able to select based on:
I think the plugin code and the benchmark tool should run in different containers. IMHO it should not be arcaflow-plugins
responsibility to create images for the benchmark tools nor to compile benchmarks tools (unless a very specific tuning is needed).
arcaflow-plugins
should allow to use existing benchmark tool images.
Smallfile currently uses a string type for a date field:
https://github.com/arcalot/arcaflow-plugins/blob/8e0be1ee6ebcb385b0c3a038a0e4e80e71b88f35/python/smallfile/smallfile_schema.py#L370-L377
Once the SDK enhancement to add the timestamp type is complete, the type needs to be implemented here.
Note: The details here need fleshing out by a pbench expert
Add a plug that enables the pbench agent to be run in parallel to other workflow plugins in order to collect system metrics
Metrics collection will be a core critical feature to many intended benchmark and other workload use cases of the Arcaflow engine.
Based on the initial Kraken implementation, we should create a plugin that lets a user delete random pods in a Kubernetes cluster in a specified namespace or namespaces. It should also provide the functionality to wait for a specified number of pods to be present afterwards.
The selection criteria should be based on:
These should also be published on PyPI so Kraken can consume them.
Specifically what's needed now is a plugin to do random node selection and labeling with affinity and anti-affinity. I think it would be best to implement this as a step of a k8s utilities plugin that may have more use cases in the future. We may also want to allow for node selection by name or label via user input.
I have plugins that I want to schedule in a workflow with certain relationships, such as necessarily being on the same nodes or on different nodes.
We need a build system that can build the plugins in the respective folders, and then check them for compliance with the plugin requirements. If run on the main branch, the images should also be pushed to Quay.
Placeholder description; please expand the details.
Add a plugin that is capable of accepting output generally from other plugins, formatting the data into an opinionated document, and optionally indexing that data into an Elasticsearch instance or passing it on to another plugin.
This is the base functionality to replicate existing use cases for data retention.
In order to make Python plugins consumable by tools like Krkn, the Python plugins should be published to PyPI. We already have a namespace set up, it's only a matter of porting the SDK publishing process and apply it to the build process
Right now plugin must contain a Dockerfile that is based on CentOS Stream 8 (quay.io/centos/centos:stream8).
I would propose to change it and rebase images on Rocky Linux 9-minimal (docker pull rockylinux/rockylinux:9-minimal
; current at https://hub.docker.com/layers/rockylinux/rockylinux/9-minimal/images/sha256-57595b7ba5d7189cc0e749a7dd0324216cbedbca2272187f50f4f3783ad06d7d?context=explore ) which mimics RHEL UBI 9 minimal (https://hub.docker.com/r/redhat/ubi9-minimal) or equivalent from almalinux (https://hub.docker.com/r/almalinux/9-minimal)
Both rocky and alma are available on quay.io as well.
Currently, the Engine tests use an image from my personal GH namespace. An official hello world plugin should be released.
We need a plugin that, on demand, is able to perform actions on GCP instances.
Actions:
The plugin should be able to select based on:
We need a plugin or multiple plugins that can adequately replace the scenarios in Krkn currently powered by the external Litmus project.
Litmus uses operators currently, which goes against the Krkn goal of operating all load tests from orchestration outside of the cluster. Krkn currently leverages the CPU, memory, and I/O hog functionalities of Litmus.
It's possible that we can simply replace the Litmus loads with those from stressng, sysbench, and/or fio, which we already support, but work needs to be done to ensure that we can provide comparable or better functionality.
We need a plugin that can remotely execute a command using SSH.
We need a plugin that, on demand, is able to perform actions on AWS EC2 instances.
Actions:
The plugin should be able to select based on:
Currently, every plugin needs to include a LICENSE file. This leads to unnecessary duplication.
We should avoid duplicating licenses as it is hard to police. Instead, we should remove the license from the individual plugins entirely.
We need to decide if the LICENSE is required inside the container. (Likely, yes.)
We need a plugin that is able to fetch a list of EC2 instances in a specific zone for removing them as part of a chaos scenario. The plugin should be able to filter by AWS tags.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.