[BENCHMARK] Onboarding OSB to Continuous Builds for Dockerhub Staging and TestPyPi + Automatic Copy-Over about opensearch-build HOT 2 OPEN

As discussed in the meeting there are multiple approaches, I will list a few here:

1. Move to GitHub Actions:

• Move both pypi build and docker image build step to GitHub Actions
• Use AWS Secret Manager to hold the credentials, and retrieve only when the workflow runs
• Each commit will trigger a github action to build pypi, build docker image, publish to staging registry
• When a github repo tag is cut for an rc, pypi will be published and docker image will be built with the published pypi binary, then push to staging, with automatic copy image from staging to production to follow

2. Use Jenkins for all:

• As of now, both pypi build/publish and docker image build step are using existing Jenkins pipelines
• Instead of using github actions, just change the webhook to trigger jenkins on every commit push/merge
• Make tweaks to the workflow so that pypi will not publish, but only built, then use to build docker staging image and push to staging registry
• When a github repo tag is cut for an rc, pypi will be published and docker image will be built with the published pypi binary, then push to staging, with automatic copy image from staging to production to follow

Thanks.

from opensearch-build.

Regarding,

OSB Jenkins file should also be updated to include a one-click copy-over so that OpenSearch Build on-call does not need to be engaged to copy over Dockerhub staging images to Dockerhub production.

We can leverage something like https://github.com/opensearch-project/opensearch-k8s-operator/blob/main/jenkins/release.jenkinsfile#L85-L127, today the operator docker images are released in the same way, please check.

from opensearch-build.