arcalot / arcaflow-plugins-incubator Goto Github PK

Uperf's README could use some more explanation on using it in CLI mode.

Please describe what you would like to see in this project

This plugin (or plugins) should be able to run the benchmark-operator and should be able to accept an input schema that allows for parameters to be passed as a CR to the operator for workload execution.

Please describe your use case

This feature will enable the Arcaflow engine to serve better in drop-in replacement scenarios for existing workflows that leverage the benchmark-operator.

Additional context

Please describe what you would like to see in this project

Instead of including a .gitignore in every plugin, we should define one per language.

Please describe your use case

Avoid duplication.

Additional context

The .gitignore file should be self-created and not copied off the Internet to avoid possible copyright issues.

We need a plugin that is able to look up which cloud provider instance a specific OpenShift node corresponds to using the machine objects.

Please describe what you would like to see in this project

We discussed that it would be a good idea to require using docker-compose to run an end-to-end test.
It would setup all components, and the exit code of each or a single container can be used to determine pass or fail.
It will also be used to show an example of how to use the plugin.

In the future, there can be an automation that goes through all the folders, tries to build all the containers in there, then runs the docker-compose to make sure it runs properly.

Please describe your use case

The use case is testing and example code.

Describe the bug

uperf plugin builds correctly with podman-compose but fails to build with docker-compose.

To reproduce

With podman-compose:

...
[2/2] COMMIT arcaflow-uperf
--> cc90295e6c9
Successfully tagged localhost/arcaflow-uperf:latest
cc90295e6c9408cd43612b8a4d86e1fb4c99abc5147c1476d7b064f5034a7bc9
exit code: 0

$ podman images
REPOSITORY                TAG         IMAGE ID      CREATED             SIZE
localhost/arcaflow-uperf  latest      cc90295e6c94  10 seconds ago      631 MB

With docker-compose:

Building server
[+] Building 8.4s (9/18)                                                        
 => [internal] load build definition from Dockerfile                       0.1s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for quay.io/centos/centos:stream8             0.9s
 => [stage-1  1/11] FROM quay.io/centos/centos:stream8@sha256:de7327910e7  0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 147B                                          0.0s
 => CACHED [stage-1  2/11] RUN dnf module -y install python39 && dnf inst  0.0s
 => CACHED [builder 2/4] RUN dnf install -y git make gcc lksctp-tools-dev  0.0s
 => CACHED [builder 3/4] RUN git clone -b 1.0.7 https://github.com/uperf/  0.0s
 => ERROR [builder 4/4] RUN cd /uperf && ./configure && make && make inst  7.3s
------                                                                          
 > [builder 4/4] RUN cd /uperf && ./configure && make && make install: 
 ...
#8 7.231 CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /uperf/missing aclocal-1.16 
#8 7.234 /uperf/missing: line 81: aclocal-1.16: command not found
#8 7.238 WARNING: 'aclocal-1.16' is missing on your system.
#8 7.238          You should only need it if you modified 'acinclude.m4' or
#8 7.238          'configure.ac' or m4 files included by 'configure.ac'.
#8 7.238          The 'aclocal' program is part of the GNU Automake package:
#8 7.238          <https://www.gnu.org/software/automake>
#8 7.238          It also requires GNU Autoconf, GNU m4 and Perl in order to run:
#8 7.238          <https://www.gnu.org/software/autoconf>
#8 7.238          <https://www.gnu.org/software/m4/>
#8 7.238          <https://www.perl.org/>
#8 7.239 make: *** [Makefile:329: aclocal.m4] Error 127
------
executor failed running [/bin/sh -c cd /uperf && ./configure && make && make install]: exit code: 2
ERROR: Service 'server' failed to build : Build failed

Additional context

$ uname -a
Darwin jlema-mac 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10 14:25:27 PDT 2022; root:xnu-8020.141.5~2/RELEASE_X86_64 x86_64

$ docker version
Client:
 Cloud integration: v1.0.25
 Version:           20.10.16
 API version:       1.41
 Go version:        go1.17.10
 Git commit:        aa7e414
 Built:             Thu May 12 09:20:34 2022
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.9.0 (80466)
 Engine:
  Version:          20.10.16
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.10
  Git commit:       f756502
  Built:            Thu May 12 09:15:42 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.4
  GitCommit:        212e8b6fa2f44b9c21b2798135fc6fb7c53efc16
 runc:
  Version:          1.1.1
  GitCommit:        v1.1.1-0-g52de29d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Please describe what you would like to see in this project

We would like a new plugin for HP's netperf benchmark utility.

Please describe your use case

Netperf is a valuable network benchmarking tool that may prove useful in cases similar to uperf.

Additional context

It seems that netperf is not well maintained. There are release tags, but the most recent one is from 2015. There are main branch updates as recently at 2021. It may be best to align the plugin to the commit hash of what is currently the latest commit.

It also might be useful to use or integrate the super-netperf script features from @jtaleric

Please describe what you would like to see in this project

Smallfile currently uses a tag to clone a release, but tags can move. This could be problematic, so change the method to pull the release tarball instead.

We need a plugin that, on demand, is able to perform actions on Azure instances.

Actions:

• stop
• start
• terminate
• reboot
• wait for status (running, stopped, terminated)

The plugin should be able to select based on:

• Node name (regexp)
• Node tag(s) (regexp)

We need a plugin that, on demand, deletes a namespace and checks for recovery. This plugin should provide two steps, one for deletion and one for timed recovery check. The deletion should be done based on the namespace name (regexp) and/or a label selector.

Describe the bug

$ python test_fio_plugin.py
E.
======================================================================
ERROR: test_functional_success (__main__.FioPluginTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "~/arcaflow-plugins/python/fio/test_fio_plugin.py", line 47, in test_functional_success
    with open("fio-plus.json", "r") as fio_output_file:
FileNotFoundError: [Errno 2] No such file or directory: 'fio-plus.json'

----------------------------------------------------------------------
Ran 2 tests in 0.010s

FAILED (errors=1)

To reproduce

Try running test for fio plugin.

Additional context

$ pip list
Package             Version
------------------- -------
arcaflow-plugin-sdk 0.7.0
attrs               22.1.0
Automat             20.2.0
build               0.8.0
constantly          15.1.0
hyperlink           21.0.0
idna                3.3
incremental         21.3.0
numpy               1.23.2
packaging           21.3
pep517              0.13.0
pip                 22.2.2
pyparsing           3.0.9
PyYAML              6.0
scipy               1.9.1
setuptools          50.3.2
six                 1.16.0
tomli               2.0.1
Twisted             22.4.0
typing_extensions   4.3.0
zope.interface      5.4.0

We need a plugin that, on demand, is able to perform actions on OpenStack instances.

Actions:

• stop
• start
• terminate
• reboot
• wait for status (running, stopped, terminated)

The plugin should be able to select based on:

• Node name (regexp)
• Node tag(s) (regexp)

We need a plugin that, on demand, is able to perform actions on GCP instances.

Actions:

• stop
• start
• terminate
• reboot
• wait for status (running, stopped, terminated)

The plugin should be able to select based on:

• Node name (regexp)
• Node tag(s) (regexp)

Please describe what you would like to see in this project

This is inspired by and should be index-compatible with the cloud-bulldozer metadata-collector and stockpile tools.

Please describe your use case

SUT (system under test) metadata is collected with stockpile using ansible facts. Collected metadata is formatted to be indexed alongside benchmark data from other plugins.

Additional context

This plugin work will need to coordinate with #12 and #13 to handle the ES indexing and compatibility.

Please describe what you would like to see in this project

Plugin should be able to take input from the proposed general-use ES plugin #12 and post-process it into a document format that is compatible with existing ES indexes used by the team.

Please describe your use case

In order for our workflow to serve as a drop-in replacement for existing systems, it is important that the output we generate is compatible with our existing data retention efforts.

Additional context

Please describe what you would like to see in this project

We are seeing a good amount of divergence across the plugins as we take slightly different approaches to organizing files. We should decide on some more standardization, incorporate those standards into the template example plugins, and update documentation to make it clear that the standards are expected.

Please describe your use case

Anyone coming to our official repos should be able to browse through a large number of supported plugins and easily understand the organization and function of files. Adhering to standardization will significantly lower the barrier to entry for new users and contributors.

Additional context

We need a plugin that, on demand, is able to perform actions on bare metal instances using IPMI.

Actions:

• stop
• start
• terminate
• reboot
• wait for status (running, stopped, terminated)

The plugin should be able to select based on:

• Node name (regexp)
• Node tag(s) (regexp)

Please describe what you would like to see in this project

I think the plugin code and the benchmark tool should run in different containers. IMHO it should not be arcaflow-plugins responsibility to create images for the benchmark tools nor to compile benchmarks tools (unless a very specific tuning is needed).

arcaflow-plugins should allow to use existing benchmark tool images.

Please describe what you would like to see in this project

Smallfile currently uses a string type for a date field:
https://github.com/arcalot/arcaflow-plugins/blob/8e0be1ee6ebcb385b0c3a038a0e4e80e71b88f35/python/smallfile/smallfile_schema.py#L370-L377

Once the SDK enhancement to add the timestamp type is complete, the type needs to be implemented here.

Please describe what you would like to see in this project

Note: The details here need fleshing out by a pbench expert

Add a plug that enables the pbench agent to be run in parallel to other workflow plugins in order to collect system metrics

Please describe your use case

Metrics collection will be a core critical feature to many intended benchmark and other workload use cases of the Arcaflow engine.

Additional context

Based on the initial Kraken implementation, we should create a plugin that lets a user delete random pods in a Kubernetes cluster in a specified namespace or namespaces. It should also provide the functionality to wait for a specified number of pods to be present afterwards.

The selection criteria should be based on:

• Pod name (regexp)
• Pod labels
• Pod namespace (regexp)

These should also be published on PyPI so Kraken can consume them.

Please describe what you would like to see in this project

Specifically what's needed now is a plugin to do random node selection and labeling with affinity and anti-affinity. I think it would be best to implement this as a step of a k8s utilities plugin that may have more use cases in the future. We may also want to allow for node selection by name or label via user input.

Please describe your use case

I have plugins that I want to schedule in a workflow with certain relationships, such as necessarily being on the same nodes or on different nodes.

We need a build system that can build the plugins in the respective folders, and then check them for compliance with the plugin requirements. If run on the main branch, the images should also be pushed to Quay.

1. • build an image given a directory of plugin directories (sub-directories, each with a dockerfile)
2. • build every image given a directory of plugin directories
3. • build an image given a plugin directory
4. • add readme
5. • check plugin source directory for compliance with requirements
6. • push a built image to at least one registry
7. • containerize
8. • add github action to carpenter's repo that executes carpenter on caller
9. • add github action trigger to plugin template repo to use carpenter's github action
10. • push an image to ghcr.io or quay.io
11. • push an image with quay robot credentials
12. • make python plugin template pass build requirements
13. • make separate repo for python plugin fio and make it passes build requirements
14. • allow for semantic versioning of image tags
15. • manage the version of a plugin
16. • check plugin source code for python code style format
17. • check plugin source code for python code quality

Please describe what you would like to see in this project

Placeholder description; please expand the details.

Add a plugin that is capable of accepting output generally from other plugins, formatting the data into an opinionated document, and optionally indexing that data into an Elasticsearch instance or passing it on to another plugin.

Please describe your use case

This is the base functionality to replicate existing use cases for data retention.

Additional context

In order to make Python plugins consumable by tools like Krkn, the Python plugins should be published to PyPI. We already have a namespace set up, it's only a matter of porting the SDK publishing process and apply it to the build process

Please describe what you would like to see in this project

Right now plugin must contain a Dockerfile that is based on CentOS Stream 8 (quay.io/centos/centos:stream8).

I would propose to change it and rebase images on Rocky Linux 9-minimal (docker pull rockylinux/rockylinux:9-minimal ; current at https://hub.docker.com/layers/rockylinux/rockylinux/9-minimal/images/sha256-57595b7ba5d7189cc0e749a7dd0324216cbedbca2272187f50f4f3783ad06d7d?context=explore ) which mimics RHEL UBI 9 minimal (https://hub.docker.com/r/redhat/ubi9-minimal) or equivalent from almalinux (https://hub.docker.com/r/almalinux/9-minimal)

Both rocky and alma are available on quay.io as well.

Please describe what you would like to see in this project

Currently, the Engine tests use an image from my personal GH namespace. An official hello world plugin should be released.

We need a plugin that, on demand, is able to perform actions on GCP instances.

Actions:

• stop
• start
• terminate
• reboot
• wait for status (running, stopped, terminated)

The plugin should be able to select based on:

• Node name (regexp)
• Node tag(s) (regexp)

Please describe what you would like to see in this project

We need a plugin or multiple plugins that can adequately replace the scenarios in Krkn currently powered by the external Litmus project.

Please describe your use case

Litmus uses operators currently, which goes against the Krkn goal of operating all load tests from orchestration outside of the cluster. Krkn currently leverages the CPU, memory, and I/O hog functionalities of Litmus.

Additional context

It's possible that we can simply replace the Litmus loads with those from stressng, sysbench, and/or fio, which we already support, but work needs to be done to ensure that we can provide comparable or better functionality.

We need a plugin that can remotely execute a command using SSH.

We need a plugin that, on demand, is able to perform actions on AWS EC2 instances.

Actions:

• stop
• start
• terminate
• reboot
• wait for status (running, stopped, terminated)

The plugin should be able to select based on:

• Node name (regexp)
• Node tag(s) (regexp)

Please describe what you would like to see in this project

Currently, every plugin needs to include a LICENSE file. This leads to unnecessary duplication.

Please describe your use case

We should avoid duplicating licenses as it is hard to police. Instead, we should remove the license from the individual plugins entirely.

Additional context

We need to decide if the LICENSE is required inside the container. (Likely, yes.)

We need a plugin that is able to fetch a list of EC2 instances in a specific zone for removing them as part of a chaos scenario. The plugin should be able to filter by AWS tags.