Summary

I'd like to propose implementing an automated testing process for Gatling-operator when new version of Kubernetes.
Currently, when a new Kubernetes version is released, we need to test Gatling-operator manually to ensure app works.

Tasks

• Create Check CI for new Kubernets version
• Create note pages to show the version already tested k8s versions

Description

Currently, as described in architecture diagram (see Gatling Operator Architecture and Design), gatling-waiter and gatling-runner container run as init containers and gatling-result-transferer as a main container in the case of generating an aggregated Gatling result report while gatling-runner runs as a main container in the case of not generating the report.

My proposal in the issue is that gatling-waiter runs as an init container and both gatling-runner and gatling-result-transferer should run as main containers in a Gatling Runner Pod.
It's generally expected behavior that gatling-runner runs as a main container in Gatling Runner Pod as it's main workload.

Summary

I always do the regression test by executing the below commands since the tests in CI are not enough.

$ make kind-create
$ kubectl config use-context kind-gatling-cluster 
$ make build 
$ make install-crd 
$ make kind-deploy 
$ make kind-sample-deploy

I hope these commands are executed in CI.

Summary

Add kind: CronGatling that creates Gatling resources periodically, just as Cronjob creates Job.

Use-case

I'd like to run gatling automatically everyday.

Summary

• I would like to set up a development environment using Docker to avoid cluttering my local development environment.
• The goal is to be able to execute operations such as running Makefile within a specific container and deploy using kind in a separate container.

Given that the last command executed after a Gatling simulation has completed is to touch a file, the container will always succeed (even when gatling failed or its assertions failed). Normally this wouldn't matter, but I'm trying to use the Gatling operator as a part of our end-to-end tests. I need to know if any of the requests fail; so when the Gatling assertion fails, it will mark the container as failed. I can then pick that up in our CI tool (GHA in this case) and act accordingly.

https://github.com/st-tech/gatling-operator/blob/main/pkg/commands/commands.go#L75

Maybe do something like this instead?

gatling.sh -sf ${SIMULATIONS_DIR_PATH} -s %s -rsf ${RESOURCES_DIR_PATH} -rf ${RESULTS_DIR_PATH} %s

GATLING_EXIT_STATUS=$?
if [ $GATLING_EXIT_STATUS -ne 0 ]; then
  RUN_STATUS_FILE="${RESULTS_DIR_PATH}/FAILED"
  echo "gatling.sh has failed!" 1>&2
fi
touch ${RUN_STATUS_FILE}
exit $GATLING_EXIT_STATUS

EDIT: you also have to set the JobSpec.BackoffLimit to 0. Otherwise failed pods from jobs will get restarted, and I typically wouldn't want to restart Gatling scenarios.

Thanks for the great job.

As my tests are written in java, I use a custom 'gatlingImage' build using Gatling 3.9.2.

But the execution of this image gives an error:

gatling-waiter pod/gatling-sample01-runner-6jfz9 labeled
gatling-waiter 1/1 pods are ready
gatling-runner Wait until 2023-03-30 08:38:57
gatling-runner GATLING_HOME is set to /opt/gatling
gatling-runner Do you want to run the simulation locally, on Gatling Enterprise, or just package it?
gatling-runner Type the number corresponding to your choice and press enter
gatling-runner [0] <Quit>
gatling-runner [1] Run the Simulation locally
gatling-runner [2] Package and upload the Simulation to Gatling Enterprise Cloud, and run it there
gatling-runner [3] Package the Simulation for Gatling Enterprise
gatling-runner [4] Show help and exit
gatling-runner Exception in thread "main" java.util.NoSuchElementException: next on empty iterator
gatling-runner     at scala.collection.Iterator$$anon$19.next(Iterator.scala:973)
gatling-runner     at scala.collection.Iterator$$anon$19.next(Iterator.scala:971)
gatling-runner     at scala.io.BufferedSource$BufferedLineIterator.next(BufferedSource.scala:82)
gatling-runner     at scala.io.BufferedSource$BufferedLineIterator.next(BufferedSource.scala:67)
gatling-runner     at io.gatling.bundle.BundleIO$$anon$2.readInt(BundleIO.scala:33)
gatling-runner     at io.gatling.plugin.io.input.InputChoice.inputInt(InputChoice.java:59)
gatling-runner     at io.gatling.plugin.io.input.InputChoice.inputFromList(InputChoice.java:113)
gatling-runner     at io.gatling.plugin.io.input.InputChoice.inputFromStringList(InputChoice.java:144)
gatling-runner     at io.gatling.bundle.commands.RunCommand.run(RunCommand.scala:59)
gatling-runner     at io.gatling.bundle.GatlingCLI$.main(GatlingCLI.scala:97)
gatling-runner     at io.gatling.bundle.GatlingCLI.main(GatlingCLI.scala)
gatling-runner gatling.sh has failed!
Stream closed EOF for default/gatling-sample01-runner-6jfz9 (gatling-runner)
Stream closed EOF for default/gatling-sample01-runner-6jfz9 (gatling-waiter)

Since Gatling 3.8.0 (to be confirmed), gatling.sh requires a run mode parameter

I tested by running my docker image and using the same args used by the gatling-runner container.
The test is executed if the '-rm' parameter is added:
gatling.sh -sf ${SIMULATIONS_DIR_PATH} -s com.example.gatling.TodoSimulation -rsf ${RESOURCES_DIR_PATH} -rf ${RESULTS_DIR_PATH} -nr -rm local

zozo-gatling-operator:953ad5f044938180381aa09443c0a08e1d85674d (debian 11.9)

debian [os-pkgs]

No vulnerabilities found

manager

gobinary [lang-pkgs]

No vulnerabilities found

Feature request

This is a feature request to add configurable MaxConcurrentReconciles option to Gatling Controller.

Gatling Controller can be configured to run with multiple concurrent reconciles with MaxConcurrentReconcilesint option that controller runtime supports (see source). Here is a snippet:

// SetupWithManager sets up the controller with the Manager.
func (r *GatlingReconciler) SetupWithManager(mgr ctrl.Manager) error {
	return ctrl.NewControllerManagedBy(mgr).
		For(&gatlingv1alpha1.Gatling{}).
		WithEventFilter(predicate.Funcs{
			DeleteFunc: func(e event.DeleteEvent) bool {
				return false
			},
		}).
		WithOptions(controller.Options{MaxConcurrentReconciles: 3}).
		Complete(r)
}

It's really convenient if users can set an option of MaxConcurrentReconcilesint in creating the controller via their config files or environment variables

For more information on MaxConcurrentReconcilesint option, please see also OpenKruise's Learning Concurrent Reconciling article

Issue

Deleting a Gatling CR causes repeated gatling CR not found errors. Let's say I delete a Gatling CR named gatling-sample01, there will be the following error occurs repeatedly

        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99
2021-11-10T10:52:41.750Z        ERROR   controller-runtime.manager.controller.gatling.gatling.Reconcile Unable to fetch Gatling for some reason, and requeue  {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "name": "gatling-sample01", "namespace": "default", "error": "Gatling.gatling-operator.tech.zozo.com \"gatling-sample01\" not found"}
github.com/go-logr/zapr.(*zapLogger).Error
        /go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132
github.com/st-tech/gatling-operator/controllers.(*GatlingReconciler).Reconcile
        /workspace/controllers/gatling_controller.go:70
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99

How to reproduce the issue

Simply deleting the gatling CR will cause the issue. Suppose you have a Gatling CR named gatling-sample01, delete it like this

kubectl delete gatling gatling-sample01 -n <your namespace>

Then, check the log of gatling-operator manager like this. You'll see gatling-sample01 not found error repeatedly

export GATLING_MANAGER_POD=$(kubectl get pods -n gatling-system -o 'jsonpath={.items[0].metadata.name}')
kubectl logs $GATLING_MANAGER_POD -n gatling-system -c manager -f

Expected behavior

Gatling CR not found errors occurs repeatedly after deleting a Gatling CR

#1 (comment)

Feature request

It's good to have an option to clean up the external resources such as the s3 bucket created for the Gatling object when the corresponding Gatling object is deleted from Kubernetes. Currently the external resources are not cleaned up.

For example, adding a new field named cleanupExternalResources in Gatling API type, if the field is set true, the controller deletes external resources for the Gatling object when the corresponding Gatling is deleted

apiVersion: gatling-operator.tech.zozo.com/v1alpha1
kind: Gatling
metadata:
  name: gatling-sample01
spec:
  generateReport: false
  notifyReport: false
  cleanupAfterJobDone: false    
  cleanupExternalResources: true <<<<  new field
...omit...

Description

This is a feature request from @yokochin0105

To support debugging and tracing with logback.xml to dig into the detail of benchmarking test kicked from Gatling Operator

Summary

go get and go-get-tool in Makefile doesn't work in Go1.18+.

So make build failed like below.
(My Go Version is 1.20.2)

make build
go: creating new go.mod: module tmp
Downloading sigs.k8s.io/controller-tools/cmd/[email protected]
go: added github.com/fatih/color v1.7.0
go: added github.com/gobuffalo/flect v0.2.0
go: added github.com/gogo/protobuf v1.3.1
go: added github.com/google/gofuzz v1.1.0
go: added github.com/inconshreveable/mousetrap v1.0.0
go: added github.com/json-iterator/go v1.1.8
go: added github.com/mattn/go-colorable v0.1.2
go: added github.com/mattn/go-isatty v0.0.8
go: added github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: added github.com/modern-go/reflect2 v1.0.1
go: added github.com/spf13/cobra v1.0.0
go: added github.com/spf13/pflag v1.0.5
go: added golang.org/x/mod v0.2.0
go: added golang.org/x/net v0.0.0-20200226121028-0de0cce0169b
go: added golang.org/x/sys v0.0.0-20191022100944-742c48ecaeb7
go: added golang.org/x/text v0.3.2
go: added golang.org/x/tools v0.0.0-20200616195046-dc31b401abb5
go: added golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
go: added gopkg.in/inf.v0 v0.9.1
go: added gopkg.in/yaml.v2 v2.2.8
go: added gopkg.in/yaml.v3 v3.0.0-20190905181640-827449938966
go: added k8s.io/api v0.18.2
go: added k8s.io/apiextensions-apiserver v0.18.2
go: added k8s.io/apimachinery v0.18.2
go: added k8s.io/klog v1.0.0
go: added k8s.io/utils v0.0.0-20200324210504-a9aa75ae1b89
go: added sigs.k8s.io/controller-tools v0.4.1
go: added sigs.k8s.io/structured-merge-diff/v3 v3.0.0
go: added sigs.k8s.io/yaml v1.2.0
/Users/koki.hatano/github/gold-kou/gatling-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
bash: /Users/koki.hatano/github/gold-kou/gatling-operator/bin/controller-gen: No such file or directory
make: *** [generate] Error 127

These may help.
kubernetes-sigs/kubebuilder#2486
https://github.com/kubernetes-sigs/kubebuilder/releases/tag/v3.4.0

Manually deleting a Gatling resource will output an error.

2021-11-29T09:55:01.990Z        ERROR   controller-runtime.manager.controller.gatling.gatling.Reconcile Unable to fetch Gatling, thus no longer requeue {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "name": "gatling-sample01", "namespace": "default", "error": "Gatling.gatling-operator.tech.zozo.com \"gatling-sample01\" not found"}
github.com/go-logr/zapr.(*zapLogger).Error
        /go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132
github.com/st-tech/gatling-operator/controllers.(*GatlingReconciler).Reconcile
        /workspace/controllers/gatling_controller.go:71
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99

This error occurs when a gatling resource is deleted and Reconcile detects the event and executes it, but the gatling resource does not already exist, resulting in an error at L70.

The following error occurs when startTime of a Gatling resource is set to a time more than 3 hours in the future.

1. The waiting time for job execution reaches maxJobRunWaitTimeInSeconds and the following error occurs.
   

   gatling-operator/controllers/gatling_controller.go

   Line 244 in 8f1e109

   msg := fmt.Sprintf("Runs out of time (%d sec) in running the runner job", maxJobCreationWaitTimeInSeconds)

2.The waiting time for job creation reaches maxJobCreationWaitTimeInSeconds, and the following error occurs.

To avoid this error, I would like to be able to set maxJobRunWaitTimeInSeconds and maxJobCreationWaitTimeInSeconds externally as environment variables.

Description

To support Azure Blog Storage for Cloud Storage Provider. Currently the following 2 providers are supported

• Amazon S3
• Google Cloud Storage

See this for Cloud Storage Provider configuration

At least, the following docs are needed

• Architecture design doc
• features list
• Quick start guide
• Configuration docs
• API reference

Thanks for putting this together. Without it I am not sure how I will test my service from GCP using Gatling.

However I cannot figure out where to put the YAML value for spec.cloudstoragespec.provider. The documentation does not specify how to get the docker build -t command to pick up on this YAML config that is needed for GCP.

If you can let me know ASAP that would be greatly appreciated. In any case, thanks again for the good work here!

will close shortly

Issue

It's been observed that sometimes duplicated slack notification messages per a Gatling test run are sent. It's not always but sometimes.

How to reproduce the issue

Again, it's not always but sometimes duplicated slack notification messages are sent while it's expected that a single slack message per a Gatling test run when you set notifyReport: true like this:

apiVersion: gatling-operator.tech.zozo.com/v1alpha1
kind: Gatling
metadata:
  name: gatling-sample01
spec:
  generateReport: true                              
  notifyReport: true                               
  cleanupAfterJobDone: true                   

Expected behavior

Single slack message ( NOT duplicate messages ) per a Gatling test run should be sent when you set notifyReport: true

Description

Add Golang based e2e tests for gatling operator using Kind

Feature request

It would be useful to have a feature like CronJob that executes gatling at a specified time.
For example, it would be possible to automatically execute gatlings at specified times during the night.

As a way to achieve this, I am thinking of creating a new CronGatling(CR) that creates a gatling(CR) at a specified time.

Please let me know what you think.

Hi,

I am trying to test gatling in locally running minikube on M1 Pro machine, and after installing operator and starting sample as:

kustomize build config/samples | kubectl apply -f -

and checking the logs:

+ gatling-sample01-runner-6bjs2 › gatling-runner
gatling-sample01-runner-6bjs2 gatling-runner Wait until 2023-03-01 16:26:32
gatling-sample01-runner-6bjs2 gatling-runner GATLING_HOME is set to /opt/gatling

gatling-runner is completely stuck when exeucting gatling.sh - as I suppose.

Also, by checking the k8s resources, I see that gatling operator has RUNNED 0/1

kns default
k get gatling,job,pod
Context "3-node" modified.
Active namespace is "default".
NAME                                                      RUNNED   REPORTED   NOTIFIED   REPORTURL   AGE
gatling.gatling-operator.tech.zozo.com/gatling-sample01   0/1                                        3m17s

NAME                                COMPLETIONS   DURATION   AGE
job.batch/gatling-sample01-runner   0/1           3m17s      3m17s

NAME                                READY   STATUS    RESTARTS   AGE
pod/gatling-sample01-runner-6bjs2   1/1     Running   0          3m17s

I reduced parallelism: 1 as I thought that this could be an issue, but not differences at all.

Any idea why is the issue?
Thanks

Summary

I wanted to bring to your attention that the current Quick Start Guide provided for the project appears to be outdated.
Some of the steps mentioned in the guide are no longer applicable, and it might cause confusion among new users trying to get started with the project.
I saw, some makefile contains helpful information for quick start.

• Update quickstart-guide.md
• Add information about make commands

Feature request

From k8s 1.11, kubectl get can ask the server what columns to display. It would be great if some useful Gatling status will be displayed with kubectl get gatling <name>

relevant pages:

• https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#additional-printer-columns
• https://book.kubebuilder.io/reference/generating-crd.html#additional-printer-columns

• test CI
• release CI

Fix vulnerabilities detected by dependabot

https://github.com/st-tech/gatling-operator/security/dependabot/2

Thank you for developing useful tool!

Is there any way to use container images placed in a private registry?

Context

I specified my container image in gatling-operator_v1alpha1_gatling01.yaml as follows:

apiVersion: gatling-operator.tech.zozo.com/v1alpha1
kind: Gatling
metadata:
  name: gatling-sample01
spec:
  generateReport: false
  generateLocalReport: false
  notifyReport: false
  cleanupAfterJobDone: true
  podSpec:
    serviceAccountName: "gatling-operator-worker"
    gatlingImage: myPriveteRepo.io/foo:latest

Applying the yaml above of course cause ImagePullBackOff error.

What I tried

I tried adding imagePullSecret: mysecret to the yaml, but found the current API does not support it.

Thanks,

Feature request

It would be great if we could add custom labels and annotations to Pods create from Gatling Runner Job.

How to achieve it?

I can come up with the following 2 ways:

1. Extends all .metadata.labels and .metadata.annotations defined in Gatling CR to Gatling runner pods

For example, if you define the labels and annotations in Gatling CR like this, the same labels and annotations are added to relevant Gatling runner pods

apiVersion: gatling-operator.tech.zozo.com/v1alpha1
kind: Gatling
metadata:
  name: gatling-sample01
  ############# this part ###############
  labels:
    app: my-sample-app
  annotations:
    gatling-operator.tech.zozo.com/hello: "true"
  ############# end this part ###############

2. Add a new ObjectMeta field to Gatling CR and allow to add to define labels and annotations in it like this:

apiVersion: gatling-operator.tech.zozo.com/v1alpha1
kind: Gatling
metadata:
  name: gatling-sample01
spec:
  ############# this part ###############
  objectMeta: 
    labels:
       app: my-sample-app
    annotations:
      gatling-operator.tech.zozo.com/hello: "true"
  ############# end this part ###############
  podSpec:
    serviceAccountName: "gatling-operator-worker"
    gatlingImage: ghcr.io/st-tech/gatling:latest          # Optional. Default: ghcr.io/st-tech/gatling:latest. The image that will be used for Gatling container.
    rcloneImage: rclone/rclone   

These are just ideas that I can think of now. There may be a better way to achieve it.

It would be nice to have a way to view merged reports. For example, a deployment that pulls down the results in the s3 bucket (or rclone), runs gatling.sh -ro, and hosts the static html with nginx.

The default value of maxJobRunWaitTimeInSeconds is 600.