opensearch-project / skills Goto Github PK

The integration test failed at distribution level for component skills
Version: 2.14.0
Distribution: tar
Architecture: arm64
Platform: linux

Please check the logs: https://build.ci.opensearch.org/job/integ-test/8107/display/redirect

* Test-report manifest:*
- https://ci.opensearch.org/ci/dbc/integ-test/2.14.0/9713/linux/arm64/tar/test-results/8107/integ-test/test-report.yml

Note: Steps to reproduce, additional logs and other files can be found within the above test-report manifest.
Instructions of this test-report manifest can be found here.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

Renovate tried to run on this repository, but found these problems.

• WARN: Error executing gradle wrapper update command. It can be not a critical one though.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• fix(deps): update dependency com.google.code.gson:gson to v2.11.0
• fix(deps): update dependency commons-validator:commons-validator to v1.9.0
• fix(deps): update mockito monorepo to v5.12.0 (org.mockito:mockito-junit-jupiter, org.mockito:mockito-core)
• 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update dependency lombok to v1.18.32
• fix(deps): update dependency net.bytebuddy:byte-buddy to v1.14.17
• fix(deps): update dependency net.bytebuddy:byte-buddy-agent to v1.14.17
• fix(deps): update dependency org.apache.logging.log4j:log4j-slf4j-impl to v2.23.1
• chore(deps): update dependency gradle to v8.8
• fix(deps): update dependency com.google.guava:guava to v33.2.1-jre
• fix(deps): update dependency org.apache.commons:commons-text to v1.12.0
• fix(deps): update dependency org.eclipse.platform:org.eclipse.core.runtime to v3.31.0
• chore(deps): update codecov/codecov-action action to v4
• chore(deps): update peter-evans/create-pull-request action to v6
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update actions/checkout action to v4
• chore(deps): update actions/setup-java action to v4

Detected dependencies

Received Error: Error building skills, retry with: ./build.sh manifests/2.15.0/opensearch-2.15.0.yml --component skills.
skills failed during the distribution build for version: 2.15.0.
Please see build log at https://build.ci.opensearch.org/job/distribution-build-opensearch/9793/display/redirect.
The failed build stage will be marked as unstable(!). Please see ./build.sh step for more details

CVE-2023-4218 - Medium Severity Vulnerability

Vulnerable Library - org.eclipse.core.runtime-3.26.100.jar

Core Runtime

Library home page: https://projects.eclipse.org/projects/eclipse.platform

Path to dependency file: /build.gradle

Path to vulnerable library: /home/wss-scanner/.gradle/caches/modules-2/files-2.1/org.eclipse.platform/org.eclipse.core.runtime/3.26.100/83c77ee0cfc948ea33f5054dda3f5c39250a7ed5/org.eclipse.core.runtime-3.26.100.jar

Dependency Hierarchy:

• ❌ org.eclipse.core.runtime-3.26.100.jar (Vulnerable Library)

Found in HEAD commit: 1675b7c41d2fe9707209de46efd7e8432909bf32

Found in base branch: main

Vulnerability Details

In Eclipse IDE versions < 2023-09 (4.29) some files with xml content are parsed vulnerable against all sorts of XXE attacks. The user just needs to open any evil project or update an open project with a vulnerable file (for example for review a foreign repository or patch).

Publish Date: 2023-11-09

URL: CVE-2023-4218

CVSS 3 Score Details (5.0)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Local
  • Attack Complexity: Low
  • Privileges Required: Low
  • User Interaction: Required
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: None
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://www.cve.org/CVERecord?id=CVE-2023-4218

Release Date: 2023-11-09

Fix Resolution: 3.29.0

• Check this box to open an automated fix PR

Is your feature request related to a problem?

We created skills repo for decoupling ml-commons and other plugins. So ml-commons doesn't need to depends on other plugins, this can void circular dependency. But now we have SearchIndexTool and VisualizationTool in skills repo which doesn't need other plugins. This will add risk for the release process: skills repo needs to wait for other plugin dependencies ready first, then we can build skills to release version. That means skills repo will be the last one to merge to release version and we have little time to fix bugs/failed ITs etc. For example, in future we have 100 tools in skills repo, but only 10 tools depends on other plugins. But we still need to wait to the last minute to test these 90 tools because other plugins not ready. Propose move these tools which doesn't depend on other plugins to ml-commons. So we can build these tools to release version as early as possible, test and fix bugs rather than always wait until the last minute.

What solution would you like?
Move tools which doesn't depend on other plugins to ml-commons, and follow this rule to reduce release risk.

What alternatives have you considered?
No

Do you have any additional context?
No

Received Error: Error building skills, retry with: ./build.sh manifests/2.12.0/opensearch-2.12.0.yml --component skills.
skills failed during the distribution build for version: 2.12.0.
Please see build log at https://build.ci.opensearch.org/job/distribution-build-opensearch/9417/display/redirect.
The failed build stage will be marked as unstable(!). Please see ./build.sh step for more details

This is a component issue for 2.13.0.
Coming from opensearch-project/opensearch-build#4433. Please follow the following checklist.
Please refer to the DATES in that post.

Preparation

• Assign this issue to a release owner.
• Finalize scope and feature set and update the Public Roadmap.
• All the tasks in this issue have been reviewed by the release owner.
• Create, update, triage and label all features and issues targeted for this release with v2.13.0.
• Finalize the code and create the the release branch 2.13 from the 2.x branch.

CI/CD

• All code changes for 2.13.0 are complete.
• Ensure working and passing CI.
• Check that this repo is included in the distribution manifest.

Pre-Release

• Increment the version on the parent branch to the next development iteration.
• Gather, review and publish release notes following the rules and back port it to the release branch.git-release-notes may be used to generate release notes from your commit history.
• Confirm that all changes for 2.13.0 have been merged.
• Add this repo to the manifest for the next developer iteration.

Release Testing

• Find/fix bugs using latest tarball and docker image provided in parent release issue and update the release notes if necessary.
• Code Complete: Test within the distribution, ensuring integration, backwards compatibility, and performance tests pass.
• Sanity Testing: Sanity testing and fixing of critical issues found.
• File issues for all intermittent test failures.

Release

• Complete documentation.
• Verify all issued labeled for this release are closed or labelled for the next release.
• Verify the release date mentioned in release notes is correct and matches actual release date.

Post Release

• Prepare for an eventual security fix development iteration by incrementing the version on the release branch to the next eventual patch version.
• Add this repo to the manifest of the next eventual security patch version.
• Suggest improvements to this template.
• Conduct a retrospective, and publish its results.

What is the bug?
When OpenSearch 2.13 releases, skills repo 2.13 branch build failed, more details can be found here: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/8053/pipeline.

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Checkout 2.13 branch at commit: a7c8b44
2. Run ./gradlew build -Dos.arch=x86_64 under skills repo
3. See error

What is the expected behavior?
The build should be succeed

What is your host/environment?

• OS: MacOS
• 14.3.1

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

This is a component issue for 2.15.0.
Coming from opensearch-project/opensearch-build#4681. Please follow the following checklist.
Please refer to the DATES in that post.

Preparation

• Assign this issue to a release owner.
• Finalize scope and feature set and update the Public Roadmap.
• All the tasks in this issue have been reviewed by the release owner.
• Create, update, triage and label all features and issues targeted for this release with v2.15.0.
• Finalize the code and create the the release branch 2.15 from the 2.x branch.

CI/CD

• All code changes for 2.15.0 are complete.
• Ensure working and passing CI.
• Check that this repo is included in the distribution manifest.

Pre-Release

• Increment the version on the parent branch to the next development iteration.
• Gather, review and publish release notes following the rules and back port it to the release branch.git-release-notes may be used to generate release notes from your commit history.
• Confirm that all changes for 2.15.0 have been merged.
• Add this repo to the manifest for the next developer iteration.

Release Testing

• Find/fix bugs using latest tarball and docker image provided in parent release issue and update the release notes if necessary.
• Code Complete: Test within the distribution, ensuring integration, backwards compatibility, and performance tests pass.
• Sanity Testing: Sanity testing and fixing of critical issues found.
• File issues for all intermittent test failures.

Release

• Complete documentation.
• Verify all issued labeled for this release are closed or labelled for the next release.
• Verify the release date mentioned in release notes is correct and matches actual release date.

Post Release

• Prepare for an eventual security fix development iteration by incrementing the version on the release branch to the next eventual patch version.
• Add this repo to the manifest of the next eventual security patch version.
• Suggest improvements to this template.
• Conduct a retrospective, and publish its results.

Received Error: Error building skills, retry with: ./build.sh manifests/2.14.1/opensearch-2.14.1.yml --component skills.
skills failed during the distribution build for version: 2.14.1.
Please see build log at https://build.ci.opensearch.org/job/distribution-build-opensearch/9845/display/redirect.
The failed build stage will be marked as unstable(!). Please see ./build.sh step for more details

What is the bug?

when execute SearchAlertsTool with agent, below error happened and will java process crash

2024-01-09 10:51:25.521	
[2024-01-09T02:51:25,518][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [opensearch-cluster-master-1] fatal error in thread [Thread-1693], exiting

2024-01-09 10:51:25.521	
java.lang.NoClassDefFoundError: kotlin/jvm/internal/Intrinsics
2024-01-09 10:51:25.521	
	at org.opensearch.commons.alerting.model.Table.<init>(Table.kt) ~[?:?]
2024-01-09 10:51:25.521	
	at org.opensearch.agent.tools.SearchAlertsTool.run(SearchAlertsTool.java:81) ~[?:?]
2024-01-09 10:51:25.521	
	at org.opensearch.ml.engine.algorithms.agent.MLChatAgentRunner.lambda$runReAct$6(MLChatAgentRunner.java:481) ~[?:?]

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

What is the expected behavior?
A clear and concise description of what you expected to happen.

What is your host/environment?

• OS: [e.g. iOS]
• Version [e.g. 22]
• Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?

The ToolFactory and Tool itself are loaded by Classloader java.net.FactoryURLClassLoader @4afbb6c2 which is ClassLoader for skills plugin. When execute the tool, it will initialize Table class which comes from common-utils. According to the Gradle dependency of common-utils, the Table class loaded by its parent ClassLoader which is ml-commons plugin ClassLoader java.net.FactoryURLClassLoader @4e08acf9. therefore, we need add kotlin stdlib depenency into ml-commons plugin instead of skills plugin.

This is a component issue for 2.14.0.
Coming from opensearch-project/opensearch-build#4562. Please follow the following checklist.
Please refer to the DATES in that post.

Preparation

• Assign this issue to a release owner.
• Finalize scope and feature set and update the Public Roadmap.
• All the tasks in this issue have been reviewed by the release owner.
• Create, update, triage and label all features and issues targeted for this release with v2.14.0.
• Finalize the code and create the the release branch 2.14 from the 2.x branch.

CI/CD

• All code changes for 2.14.0 are complete.
• Ensure working and passing CI.
• Check that this repo is included in the distribution manifest.

Pre-Release

• Increment the version on the parent branch to the next development iteration.
• Gather, review and publish release notes following the rules and back port it to the release branch.git-release-notes may be used to generate release notes from your commit history.
• Confirm that all changes for 2.14.0 have been merged.
• Add this repo to the manifest for the next developer iteration.

Release Testing

• Find/fix bugs using latest tarball and docker image provided in parent release issue and update the release notes if necessary.
• Code Complete: Test within the distribution, ensuring integration, backwards compatibility, and performance tests pass.
• Sanity Testing: Sanity testing and fixing of critical issues found.
• File issues for all intermittent test failures.

Release

• Complete documentation.
• Verify all issued labeled for this release are closed or labelled for the next release.
• Verify the release date mentioned in release notes is correct and matches actual release date.

Post Release

• Prepare for an eventual security fix development iteration by incrementing the version on the release branch to the next eventual patch version.
• Add this repo to the manifest of the next eventual security patch version.
• Suggest improvements to this template.
• Conduct a retrospective, and publish its results.

CVE-2023-5072 - High Severity Vulnerability

Vulnerable Library - json-20230227.jar

JSON is a light-weight, language independent, data interchange format. See http://www.JSON.org/

    The files in this package implement JSON encoders/decoders in Java.
    It also includes the capability to convert between JSON and XML, HTTP
    headers, Cookies, and CDL.

    This is a reference implementation. There is a large number of JSON packages
    in Java. Perhaps someday the Java community will standardize on one. Until
    then, choose carefully.</p>

Library home page: https://github.com/douglascrockford/JSON-java

Path to dependency file: /build.gradle

Path to vulnerable library: /home/wss-scanner/.gradle/caches/modules-2/files-2.1/org.json/json/20230227/7a0d4aca76513d8ce81f9b044ce8126b84809ad8/json-20230227.jar

Dependency Hierarchy:

• ❌ json-20230227.jar (Vulnerable Library)

Found in HEAD commit: 1675b7c41d2fe9707209de46efd7e8432909bf32

Found in base branch: main

Vulnerability Details

Denial of Service in JSON-Java versions up to and including 20230618.  A bug in the parser means that an input string of modest size can lead to indefinite amounts of memory being used. 

Publish Date: 2023-10-12

URL: CVE-2023-5072

CVSS 3 Score Details (7.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: None
  • Integrity Impact: None
  • Availability Impact: High

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-rm7j-f5g5-27vv

Release Date: 2023-10-12

Fix Resolution: 20231013

• Check this box to open an automated fix PR

Vulnerable Library - org.eclipse.core.runtime-3.26.100.jar

Core Runtime

Library home page: https://projects.eclipse.org/projects/eclipse.platform

Path to dependency file: /build.gradle

Path to vulnerable library: /home/wss-scanner/.gradle/caches/modules-2/files-2.1/org.eclipse.platform/org.eclipse.core.runtime/3.26.100/83c77ee0cfc948ea33f5054dda3f5c39250a7ed5/org.eclipse.core.runtime-3.26.100.jar

Found in HEAD commit: 6261bb58c27f31fb187389465c943d4f2df507cf

Vulnerabilities

CVE	Severity	CVSS	Dependency	Type	Fixed in (org.eclipse.core.runtime version)	Remediation Possible**
CVE-2023-4218	Medium	5.0	org.eclipse.core.runtime-3.26.100.jar	Direct	3.29.0	✅

**In some cases, Remediation PR cannot be created automatically for a vulnerability despite the availability of remediation

Details

CVE-2023-4218

Vulnerable Library - org.eclipse.core.runtime-3.26.100.jar

Core Runtime

Library home page: https://projects.eclipse.org/projects/eclipse.platform

Path to dependency file: /build.gradle

Path to vulnerable library: /home/wss-scanner/.gradle/caches/modules-2/files-2.1/org.eclipse.platform/org.eclipse.core.runtime/3.26.100/83c77ee0cfc948ea33f5054dda3f5c39250a7ed5/org.eclipse.core.runtime-3.26.100.jar

Dependency Hierarchy:

• ❌ org.eclipse.core.runtime-3.26.100.jar (Vulnerable Library)

Found in HEAD commit: 6261bb58c27f31fb187389465c943d4f2df507cf

Found in base branch: main

Vulnerability Details

In Eclipse IDE versions < 2023-09 (4.29) some files with xml content are parsed vulnerable against all sorts of XXE attacks. The user just needs to open any evil project or update an open project with a vulnerable file (for example for review a foreign repository or patch).

Publish Date: 2023-11-09

URL: CVE-2023-4218

CVSS 3 Score Details (5.0)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Local
  • Attack Complexity: Low
  • Privileges Required: Low
  • User Interaction: Required
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: None
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://www.cve.org/CVERecord?id=CVE-2023-4218

Release Date: 2023-11-09

Fix Resolution: 3.29.0

⛑️ Automatic Remediation will be attempted for this issue.

⛑️Automatic Remediation will be attempted for this issue.

Is your feature request related to a problem?
Now in flow agent and chat agent, tool would be hard to fetch previous tools' output because

1. in flow agent, the key in parameters map would be "<tool_name>.output" and tool_name is defined by customer. So we cannot config this in java code. We need one-more customer defined field like #131
2. In chat agent, the previous output would be dropped. Only the action_input of this round will be used.

What solution would you like?

1. Extract the field and method inside #131 into an abstract class implements Tool. Then each tool should extend this abstract Tool
2. A map should be kept inside the runReAct function to store all "<tool_name>: <tool_output>" pairs

CVE-2023-2976 - High Severity Vulnerability

Vulnerable Library - guava-31.1-jre.jar

Guava is a suite of core and expanded libraries that include utility classes, Google's collections, I/O classes, and much more.

Path to dependency file: /build.gradle

Path to vulnerable library: /home/wss-scanner/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/31.1-jre/60458f877d055d0c9114d9e1a2efb737b4bc282c/guava-31.1-jre.jar

Dependency Hierarchy:

• google-java-format-1.17.0.jar (Root Library)
  • ❌ guava-31.1-jre.jar (Vulnerable Library)

Found in HEAD commit: 1675b7c41d2fe9707209de46efd7e8432909bf32

Found in base branch: main

Vulnerability Details

Use of Java's default temporary directory for file creation in FileBackedOutputStream in Google Guava versions 1.0 to 31.1 on Unix systems and Android Ice Cream Sandwich allows other users and apps on the machine with access to the default Java temporary directory to be able to access the files created by the class.

Even though the security vulnerability is fixed in version 32.0.0, we recommend using version 32.0.1 as version 32.0.0 breaks some functionality under Windows.

Mend Note: Even though the security vulnerability is fixed in version 32.0.0, maintainers recommend using version 32.0.1 as version 32.0.0 breaks some functionality under Windows.

Publish Date: 2023-06-14

URL: CVE-2023-2976

CVSS 3 Score Details (7.1)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Local
  • Attack Complexity: Low
  • Privileges Required: Low
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: High
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-7g45-4rm6-3mm3

Release Date: 2023-06-14

Fix Resolution: com.google.guava:guava:32.0.1-android,32.0.1-jre

The integration test failed at distribution level for component skills
Version: 2.13.0
Distribution: tar
Architecture: arm64
Platform: linux

Please check the logs: https://build.ci.opensearch.org/job/integ-test/7984/display/redirect

* Test-report manifest:*
- https://ci.opensearch.org/ci/dbc/integ-test/2.13.0/9616/linux/arm64/tar/test-results/7984/integ-test/test-report.yml

Note: Steps to reproduce, additional logs and other files can be found within the above test-report manifest.
Instructions of this test-report manifest can be found here.

What is the bug?
When asking agent on the questions: Can you tell me what's the population increase of Seattle from 2021 to 2023 by using RAGTool?, the agent will gives an error of

2024-01-16 15:28:58 | java.lang.IllegalArgumentException: Some parameter placeholder not filled in payload: prompt
-- | --
2024-01-16 15:28:58 | at org.opensearch.ml.common.connector.Connector.validatePayload(Connector.java:88) ~[opensearch-ml-common-3.0.0.0-SNAPSHOT.jar:?]
2024-01-16 15:28:58 | at org.opensearch.ml.engine.algorithms.remote.RemoteConnectorExecutor.preparePayloadAndInvokeRemoteModel(RemoteConnectorExecutor.java:118) ~[opensearch-ml-algorithms-3.0.0.0-SNAPSHOT.jar:?]
2024-01-16 15:28:58 | at org.opensearch.ml.engine.algorithms.remote.RemoteConnectorExecutor.executePredict(RemoteConnectorExecutor.java:74) ~[opensearch-ml-algorithms-3.0.0.0-SNAPSHOT.jar:?]
2024-01-16 15:28:58 | at org.opensearch.ml.engine.algorithms.remote.RemoteModel.predict(RemoteModel.java:61) ~[opensearch-ml-algorithms-3.0.0.0-SNAPSHOT.jar:?]

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Go to Chatbot
2. Click on Chatbot button
3. Start a new conversation and ask Can you tell me what's the population increase of Seattle from 2021 to 2023 by using RAGTool?
4. See error

What is the expected behavior?
It should response without error

What is your host/environment?

• OS: Linux
• Version 3.0.0
• Plugins

Do you have any screenshots?

Do you have any additional context?
Add any other context about the problem.

Received Error: Error building skills, retry with: ./build.sh manifests/2.13.0/opensearch-2.13.0.yml --component skills.
skills failed during the distribution build for version: 2.13.0.
Please see build log at https://build.ci.opensearch.org/job/distribution-build-opensearch/9370/display/redirect.
The failed build stage will be marked as unstable(!). Please see ./build.sh step for more details

Received Error: Error building skills, retry with: ./build.sh manifests/2.14.0/opensearch-2.14.0.yml --component skills.
skills failed during the distribution build for version: 2.14.0.
Please see build log at https://build.ci.opensearch.org/job/distribution-build-opensearch/9630/display/redirect.
The failed build stage will be marked as unstable(!). Please see ./build.sh step for more details

Received Error: Error building skills, retry with: ./build.sh manifests/2.12.0/opensearch-2.12.0.yml --component skills.
The distribution build for skills has failed for version: 2.12.0.
Please see build log at https://build.ci.opensearch.org/job/distribution-build-opensearch/9319/display/redirect.
The failed build stage will be marked as unstable(!). Please see ./build.sh step for more details

What is the bug?
The pom file generate missing inceptionYear, without inceptionYear the maven publish can be rejected when published to central maven. For more information please check the issue opensearch-project/opensearch-build#3638 which happened in past. Also refer opensearch-project/opensearch-build#2928.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.opensearch.plugin</groupId>
  <artifactId>opensearch-skills</artifactId>
  <version>2.12.0.0-SNAPSHOT</version>
  <packaging>zip</packaging>
  <name>opensearch-skills</name>
  <description>OpenSearch Skills</description>
  <licenses>
    <license>
      <name>The Apache License, Version 2.0</name>
      <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
    </license>
  </licenses>
  <developers>
    <developer>
      <name>OpenSearch</name>
      <url>https://github.com/opensearch-project/skills</url>
    </developer>
  </developers>
  <url>https://github.com/opensearch-project/skills</url>
  <scm>
    <url>https://github.com/opensearch-project/skills</url>
  </scm>
</project>

Refer the sample pom file: https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/opensearch-job-scheduler/2.12.0.0-SNAPSHOT/opensearch-job-scheduler-2.12.0.0-20240206.180956-15.pom

Please note this could be a blocker for 2.12.0 release once the RC is finalized.
opensearch-project/opensearch-build#4115

Current alerting tool IT is rudimentary and does not test many cases. Add tests for both alerting tools to check:

• sample data in system indices is processed correctly
• filtering works correctly
• single piece of data is processed correctly
• multiple pieces of data is processed correctly

Received Error: Error building skills, retry with: ./build.sh manifests/2.13.1/opensearch-2.13.1.yml --component skills.
skills failed during the distribution build for version: 2.13.1.
Please see build log at https://build.ci.opensearch.org/job/distribution-build-opensearch/9729/display/redirect.
The failed build stage will be marked as unstable(!). Please see ./build.sh step for more details

Vulnerable Library - google-java-format-1.17.0.jar

Path to dependency file: /build.gradle

Path to vulnerable library: /home/wss-scanner/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/31.1-jre/60458f877d055d0c9114d9e1a2efb737b4bc282c/guava-31.1-jre.jar

Found in HEAD commit: 6261bb58c27f31fb187389465c943d4f2df507cf

Vulnerabilities

CVE	Severity	CVSS	Dependency	Type	Fixed in (google-java-format version)	Remediation Possible**
CVE-2023-2976	High	7.1	guava-31.1-jre.jar	Transitive	N/A*	❌

*For some transitive vulnerabilities, there is no version of direct dependency with a fix. Check the "Details" section below to see if there is a version of transitive dependency where vulnerability is fixed.

**In some cases, Remediation PR cannot be created automatically for a vulnerability despite the availability of remediation

Details

CVE-2023-2976

Vulnerable Library - guava-31.1-jre.jar

Guava is a suite of core and expanded libraries that include utility classes, Google's collections, I/O classes, and much more.

Path to dependency file: /build.gradle

Path to vulnerable library: /home/wss-scanner/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/31.1-jre/60458f877d055d0c9114d9e1a2efb737b4bc282c/guava-31.1-jre.jar

Dependency Hierarchy:

• google-java-format-1.17.0.jar (Root Library)
  • ❌ guava-31.1-jre.jar (Vulnerable Library)

Found in HEAD commit: 6261bb58c27f31fb187389465c943d4f2df507cf

Found in base branch: main

Vulnerability Details

Use of Java's default temporary directory for file creation in FileBackedOutputStream in Google Guava versions 1.0 to 31.1 on Unix systems and Android Ice Cream Sandwich allows other users and apps on the machine with access to the default Java temporary directory to be able to access the files created by the class.

Even though the security vulnerability is fixed in version 32.0.0, we recommend using version 32.0.1 as version 32.0.0 breaks some functionality under Windows.

Mend Note: Even though the security vulnerability is fixed in version 32.0.0, maintainers recommend using version 32.0.1 as version 32.0.0 breaks some functionality under Windows.

Publish Date: 2023-06-14

URL: CVE-2023-2976

CVSS 3 Score Details (7.1)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Local
  • Attack Complexity: Low
  • Privileges Required: Low
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: High
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-7g45-4rm6-3mm3

Release Date: 2023-06-14

Fix Resolution: com.google.guava:guava:32.0.1-android,32.0.1-jre

What is the bug?

The release notes for 2.13.0 (https://github.com/opensearch-project/skills/blob/main/release-notes/opensearch-skills.release-notes-2.13.0.0.md#dependencies) do not follow the standard guidelines mentioned here: https://github.com/opensearch-project/opensearch-plugins/blob/main/RELEASE_NOTES.md

Due to this the automation in https://github.com/opensearch-project/opensearch-build/tree/main/src/release_notes_workflow breaks and the release manager has to manually sort the PRs.

How can one reproduce the bug?

Clone build repo and run ./release_notes.sh compile manifests/2.13.0/opensearch-2.13.0.yml --output opensearch.md
See skills under NON-COMPLAINT banner:

<h2>NON-COMPLIANT</h2>
<h2>DEPENDENCIES</h2>
<h3>Opensearch Skills</h3>
<ul>
<li>Update mockito monorepo to v5.10.0 (#128) (#197)</li>
<li>Update dependency org.apache.commons:commons-lang3 to v3.14.0 (#47)</li>
<li>Update dependency org.apache.commons:commons-text to v1.11.0 (#62)</li>
<li>Update plugin io.freefair.lombok to v8.6 (#245) (#249)</li>
<li>Update plugin de.undercouch.download to v5.6.0 (#239) (#250)</li>
<li>Update plugin com.diffplug.spotless to v6.25.0 (#127) (#252)</li>
<li>Update dependency org.json:json to v20240205 (#246) (#251)</li>
</ul>

What is the expected behavior?

Should not be under non-complaint banner but appropriate headings

What is your host/environment?

Operating system, version.

Do you have any screenshots?

If applicable, add screenshots to help explain your problem.

Do you have any additional context?

Add any other context about the problem.

Received Error: Error building skills, retry with: ./build.sh manifests/2.14.0/opensearch-2.14.0.yml --component skills.
skills failed during the distribution build for version: 2.14.0.
Please see build log at https://build.ci.opensearch.org/job/distribution-build-opensearch/9769/display/redirect.
The failed build stage will be marked as unstable(!). Please see ./build.sh step for more details

Received Error: Error building skills, retry with: ./build.sh manifests/2.13.0/opensearch-2.13.0.yml --component skills.
skills failed during the distribution build for version: 2.13.0.
Please see build log at https://build.ci.opensearch.org/job/distribution-build-opensearch/9454/display/redirect.
The failed build stage will be marked as unstable(!). Please see ./build.sh step for more details

What is the bug?
When releasing OpenSearch 2.13, skills repo 2.13 branch build failed, more details can be found here: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/8053/pipeline.

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Checkout skills 2.13 branch with commit id: a7c8b44
2. Run ./gradlew build -Dos.arch=x86_64 under skills repo.
3. See error

What is the expected behavior?
Skills repo should be build successfully.

What is your host/environment?

• OS: MacOS
• 14.3.1

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

This is a component issue for 3.0.0.
Coming from opensearch-project/opensearch-build#3747. Please follow the following checklist.
Please refer to the DATES in that post.

Preparation

• Assign this issue to a release owner.
• Finalize scope and feature set and update the Public Roadmap.
• All the tasks in this issue have been reviewed by the release owner.
• Create, update, triage and label all features and issues targeted for this release with v3.0.0.
• Finalize the code and create the the release branch 3.0 from the 3.x branch.

CI/CD

• All code changes for 3.0.0 are complete.
• Ensure working and passing CI.
• Check that this repo is included in the distribution manifest.

Pre-Release

• Increment the version on the parent branch to the next development iteration.
• Gather, review and publish release notes following the rules and back port it to the release branch.git-release-notes may be used to generate release notes from your commit history.
• Confirm that all changes for 3.0.0 have been merged.
• Add this repo to the manifest for the next developer iteration.

Release Testing

• Find/fix bugs using latest tarball and docker image provided in parent release issue and update the release notes if necessary.
• Code Complete: Test within the distribution, ensuring integration, backwards compatibility, and performance tests pass.
• Sanity Testing: Sanity testing and fixing of critical issues found.
• File issues for all intermittent test failures.

Release

• Complete documentation.
• Verify all issued labeled for this release are closed or labelled for the next release.
• Verify the release date mentioned in release notes is correct and matches actual release date.

Post Release

• Prepare for an eventual security fix development iteration by incrementing the version on the release branch to the next eventual patch version.
• Add this repo to the manifest of the next eventual security patch version.
• Suggest improvements to this template.
• Conduct a retrospective, and publish its results.

This is a component issue for 2.12.0.
Coming from opensearch-project/opensearch-build#4115. Please follow the following checklist.
Please refer to the DATES in that post.

Preparation

• Assign this issue to a release owner.
• Finalize scope and feature set and update the Public Roadmap.
• All the tasks in this issue have been reviewed by the release owner.
• Create, update, triage and label all features and issues targeted for this release with v2.12.0.
• Finalize the code and create the the release branch 2.12 from the 2.x branch.

CI/CD

• All code changes for 2.12.0 are complete.
• Ensure working and passing CI.
• Check that this repo is included in the distribution manifest.

Pre-Release

• Increment the version on the parent branch to the next development iteration.
• Gather, review and publish release notes following the rules and back port it to the release branch.git-release-notes may be used to generate release notes from your commit history.
• Confirm that all changes for 2.12.0 have been merged.
• Add this repo to the manifest for the next developer iteration.

Release Testing

• Find/fix bugs using latest tarball and docker image provided in parent release issue and update the release notes if necessary.
• Code Complete: Test within the distribution, ensuring integration, backwards compatibility, and performance tests pass.
• Sanity Testing: Sanity testing and fixing of critical issues found.
• File issues for all intermittent test failures.

Release

• Complete documentation.
• Verify all issued labeled for this release are closed or labelled for the next release.
• Verify the release date mentioned in release notes is correct and matches actual release date.

Post Release

• Prepare for an eventual security fix development iteration by incrementing the version on the release branch to the next eventual patch version.
• Add this repo to the manifest of the next eventual security patch version.
• Suggest improvements to this template.
• Conduct a retrospective, and publish its results.

The integration test failed at distribution level for component skills
Version: 2.12.0
Distribution: tar
Architecture: x64
Platform: linux

Please check the logs: https://build.ci.opensearch.org/job/integ-test/7655/display/redirect

* Test-report manifest:*
- https://ci.opensearch.org/ci/dbc/integ-test/2.12.0/9350/linux/x64/tar/test-results/7655/integ-test/test-report.yml

Note: Steps to reproduce, additional logs and other files can be found within the above test-report manifest.
Instructions of this test-report manifest can be found here.

[Enquiry] How to get the documentation for this tool to do PPL from natural language and to do RAG

As i do not see any documentation on this tool

Thanks

RamDa

Is your feature request related to a problem?
The current PPL tool implementation will execute PPL as well

This can be useful for chat, where it would need the query response to reply to the user. But for query assist in event analytics observability plugin, the PPL tool is only used to generate the query, execution will be covered by event analytics. This allows UI to make any additional modifications to the query. It's ok to just ignore PPL execution results from PPL tool, but it's extra overhead and a bit unreliable since sometimes we need to extract the original PPL from error messages

What solution would you like?
Split PPL tool into two parts, one for generate, one for execute

What alternatives have you considered?
A clear and concise description of any alternative solutions or features you've considered.

Do you have any additional context?
cc @xinyual