[Enhancement] Move knn lib building process to its standalone stage or pipeline about opensearch-build HOT 3 OPEN

In short term:

• Trigger knnlib standalone pipeline once every release (ideally), upload to s3 with same version number as the release plugin version.
• Trigger distribution build with cron and build with the uploaded knnlib.
• If knnlib requires any updates just repeat first step

In long term:

• knnlib has its own version sit into its own repo. Every time when a release comes, maintainer cut a tag to trigger a build, upload to maven.
• Trigger distribution build with cron and build with knnlib pulled from maven.
• If knnlib requires any updates just repeat first step

from opensearch-build.

With the success of the AL2 compilation, we will go ahead with the S3 approach for now, and see if we can bring it up for 2.15.0. If not, we will fall back to use AL2 for all the builds. Else, we will use Almalinux8 for the build and AL2 for knnlib.

#4379 (comment)

Start checking the jenkins lib and write a POC for the knnlib pipeline.

from opensearch-build.

After discussion with team on the current progress, here are some of the points suggested by comments:

1. Since #4379 confirms that AL2 can compile the lib, we can fully deprecate CentOS7 by 2.15.0.
2. k-NN team should consider move the jni libs to its own repo, so they can independently version the lib outside of OpenSearch version, and release from the github actions.
3. We can use AL2 to compile core + plugins + knnlib for now, and switch to Almalinux8 in the next year. The existing pipeline is pretty stable for now.

Welcome comments,

Thanks.

from opensearch-build.