We need to update Security plugin API path and settings.

• APIs are stilling using "opensearch", for example. We should use "plugins". The changes are in this PR.
• Settings need to be renamed to reflect the new names, for example. The changes are in this PR.

• Creating indices, all available settings
• Get index
• Updating index settings without reindexing, which can you update?
• Deleting indices
• Creating mappings, all available settings
• Get mapping
• Updating mappings, how does that work?
• Deleting mappings

The Helm for OpenSearch is released with Dashboards in approval.
https://github.com/opensearch-project/opensearch-devops/tree/main/Helm

We need a new section for Helm in docs just like back in ODFE:
https://opendistro.github.io/for-elasticsearch-docs/docs/install/helm/

Please let me know if there is anything I need to provide.

Thanks.

After bwc changes, update the setting and API to new formate (opendistro -> plugins)

We need to update AD plugin settings which still using "opendistro" https://docs-beta.opensearch.org/docs/ad/settings/, https://docs-beta.opensearch.org/docs/ad/ and https://docs-beta.opensearch.org/docs/ad/security/

And APIs are stilling using "opensearch" we should use "plugin", https://docs-beta.opensearch.org/docs/ad/api/

OpenSearch Dashboards contains items that isn't supported in other browsers.

For example, opensearch-project/OpenSearch-Dashboards#387 introduces animateTransform which isn't supported in IE11.

Browser support, as of 7.9, Kibana (and EUI) no longer supports IE11. See this matrix: https://www.elastic.co/support/matrix#matrix_browsers. This could be very important for people to know.

OpenSearch requires JDK 11 to build at a minimum per dev guide. It has been building with JDK14 (and will be building with 11 post 1.0 with this change). The combined artifact is packaging JDK 15.0.1.

• document the above
• potentially document what JDK(s) we testing with, core and plugins

Currently in Open-search we have the index names supporting the date math in the index pattern. We can create indices with the names consisting of the pattern described here. % Encoding is required while sending the request.

In alerting we had a request from the customer where we wanted to give them the support for defining these indices in the desired pattern while creating the monitors. The pattern would be exact similar to the one which is used while creating and indexing data into these indices.

opensearch-project/alerting#74
opensearch-project/alerting-dashboards-plugin#17

We have added this functionality and wanted to update our documentation for it. We wanted to link the document describing the index creation in the required pattern in open-search.

Thanks!

The file is missing from the documentation website:
https://opendistro.github.io/for-elasticsearch-docs/docs/security/configuration/yaml/

The only documentation available is inside the YAML file itself, in the repo:
https://github.com/opendistro-for-elasticsearch/security/blob/master/securityconfig/whitelist.yml

Details here: opensearch-project/security#1007

The documentation should be extended. There is a chance from elasticsearch to opensearch as an output plugin in the logstash-oss version which can be missed while setting up an opensearch environment

Affected documentation https://opensearch.org/docs/clients/logstash/index/#install-logstash

There should be a hint in step 4 like: "For users that migrate from Elastic or Amazon OpenDistro: The output plugin has changed from elasticsearch to opensearch"

Regards

This PR introduces the support for timeout based cancellation for search request. It adds request level parameter (cancel_after_time_interval) for search and msearch requests and cluster setting search.cancel_after_time_interval to control the timeout interval for cancelling the request. The feature is targeted for 1.1 release.

As a part of these issues:

opensearch-project/alerting#74
opensearch-project/alerting-dashboards-plugin#17

We have added the support for defining the query indices in the Data Math Pattern.

As a part of this issue, we want to update our documentation to aprise users of this functionality. Once the document for the required pattern is available for Opensearch, we will link it to our documentation for the plugin.

• Conceptual information on the basic goals here
• Paths from Elasticsearch OSS 5.x onward
• Rolling upgrade
• Cluster restart upgrade
• Kibana to OpenSearch Dashboards

Is your feature request related to a problem? Please describe.

Users don't have a place to find what's supported with what. For example, a user asks what the minimum JVM runtime environment is.

Describe the solution you'd like

Publish a set of support matrices in the product documentation, showing OS, JVM, browser configurations we've tested with, and inter-compatibilities between product versions.

via @julesg, Probably should cover "validated" or "tested" configurations rather than "supported" ones... since the project itself doesn't offer 'support' in the same way a commercial service might.

For example, https://opensearch.org/docs/opensearch/install/compatibility/.

What are we going to do when we release 1.1 or 2.0?

These should be https://opensearch.org/docs/opensearch/*1.0*/install/compatibility/

(coming from opensearch-project/OpenSearch#533)

it'd be cool if each API / feature in the documentation would state since which release it has been stable (i.e. part of the product and supported (non-beta)).

this is e.g. a cool feature of the documentation for the Rust programming language: their docs always state since when something has been stable. random example from their docs: https://doc.rust-lang.org/std/result/enum.Result.html (note the "1.0.0" in the top-right corner for the whole enum or "1.0.0 (const: 1.48.0)" further down for a few of the impl - the details are not relevant here, it's just about the fact that this information is present everywhere).

rationale: usually when you search for something online you end up on the latest version of the documentation. but equally often you might not be running on that specific version and might not want to / be able to upgrade to the latest version right now (the dream of running always-up-to-date bleeding edge software is often just that: a dream). it is then very helpful if the documentation immediately tells you starting from when a feature was available, so that you can judge whether you can use it (esp. if you're e.g. building something which needs to work with multiple versions of OpenSearch).
it might also be that the documentation on a newer release is more comprehensive (who will bother updating older documentation if it's already correct in a newer version?), so being able to check the latest version and still see that it's the same as in earlier versions helps.

there are of course some things to consider here, e.g. does the specified version state when the API / feature has been introduced or when the last breaking change happened to it (if a breaking change happened to it)?

adding this information right now would be fairly easy: just specify 1.0.0 for everything (i guess we can safely disregard the elasticsearch history before and presume that this information is mainly interesting for the future and everyone will be on opensearch 1.0 or newer). doing it later would be a lot more effort (things will have been introduced after 1.0 and will have to be found again to be correctly flagged accordingly).

Current Documentation here is out of date. Update the documentation to have the correct API paths as per this issue: opensearch-project/alerting#14

Repo -> https://github.com/opensearch-project/logstash-output-opensearch.
We can start with basic configuration as file/beats/http as input, grok as filter and opensearch as output with and without template

Docs says "This documentation remains in a beta state. It has content gaps and might contain bugs.", we need to remove that.

Currently the node_dn.yml is only mentioned in the YAML files.

The lag of documentation makes it hard to make use of this option.

This issue is to track the addition of documentation for remote reindex SPI extension(#547) once the public documentation is available.

Is your feature request related to a problem? Please describe.

Job scheduler has configuration, API, etc., but is not documented. I don't think having a supporting role is a good enough argument not to publish its documentation. Maybe it belongs in a different section.

Describe the solution you'd like

At least job scheduler settings documented.

Additional context

Coming from opensearch-project/opensearch-plugins#15 (comment) and opensearch-project/opensearch-plugins#18 (review).

Any link in the documentation with a trailing slash currently seems to be broken and redirects to the main site, e.g. links on
https://docs-beta.opensearch.org/security-plugin/configuration/index

https://docs-beta.opensearch.org/opensearch/install/docker-security/ redirects to https://docs-beta.opensearch.org/
vs.
https://docs-beta.opensearch.org/opensearch/install/docker-security which works.

I'm able to reproduce this both on Chrome and Firefox.

Please update the docs to reflect beyond wire compatibility, using beats beyond 7.10.x risks beats using logic that may not be compatible with the opensearch backend.

Example - beats often make use of ingest pipelines - and newer ingest pipelines may use features that require the opensearch backend be compatible with > 7.10.x

There is a quite well documented configuration settings for OpenID: https://opendistro.github.io/for-elasticsearch-docs/docs/security-configuration/openid-connect/ but it clearly misses the information that for openid the openid_connect_idp prefix is required:
https://github.com/opendistro-for-elasticsearch/security-advanced-modules/blob/e5d91955087c7365ac9a9ff33368521761976ee6/src/main/java/com/amazon/dlic/auth/http/jwt/keybyoidc/HTTPJwtKeyByOpenIdConnectAuthenticator.java#L57

So this is how it should be configured:

openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: preferred_username
            roles_key: roles
            openid_connect_idp:
              enable_ssl: true
              verify_hostnames: false

instead of:

openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            enable_ssl: true
            verify_hostnames: false
            subject_key: preferred_username
            roles_key: roles

A cluster setting was introduced in opensearch-project/OpenSearch#847. We removed the distribution in backwards-compatible mode in opensearch-project/OpenSearch#898. Set compatibility.override_main_response_version to put the cluster into a configuration that returns 7.10.2 as with ES OSS.

Currently https://docs-beta.opensearch.org/clients/agents-and-ingestion-tools/index/ point to "yes via a cluster setting" which links to opensearch-project/OpenSearch#693, but should be linking to the documentation that explains how to set this setting.

Update the query API endpoints and the perftop artifact name for PA plugin

For the tested clients, it would be helpful to link to compatible downloads. Could we add them?

It looks like the docs are incorrect on the security admin page - it references .opensearch_security as an index. Using RC1, the index is still .opendistro_security. To my knowledge this isn't changing prior to GA.

Looking back in history, this seems to be unchanged from the repo creation. Perhaps a copy/paste error?

Update the query/setting API endpoints and the setting names for SQL plugin

Add support of console for 1-click copying of the APIs in the curl format for easier consumption by developers.

Dark modes are—and this is beyond dispute—cool. We should have a dark mode toggle in the documentation, similar to Docusaurus and Docker.

Original issue here: opendistro/for-elasticsearch-docs#383

Issue
When setting up my Opensearch Cluster and the Dashboard, I got stuck with the option opensearch.ssl.certificateAuthorities in the Dashboard configuration file.

For the named option I can find two entries:
https://opensearch.org/docs/security-plugin/configuration/generate-certificates/#opensearch-dashboards

You have two options: disable SSL verification or add the root CA.

https://opensearch.org/docs/dashboards/install/tls/

[...]specify the full path (e.g. [ "/usr/share/opensearch-dashboards-1.0.0/config/root-ca.pem" ] to the certificate authority for your OpenSearch cluster.

To get the Dashboard working with SSL you need both the "signing CA" and the "root CA" if they are not the same.

Wish
I would wish that the docs get updates with a more clear definition of what CA(s) are needed.
Something like that:

You have two options: disable SSL verification or add the root CA and signing CA.

[...] to the root CA and signing CA for your OpenSearch cluster.

Coming from #39 we need documentation on migrating from Kibana OSS to OpenSearch Dashboards.

please add network and firewall requirements to the documentation?

In case OpenSearch is used in production on-premise, it would be great to get an overview of all the network requirements eg. firewall ports that needs to be opened.

Below a list of some ports (might not be complete) :

Firewall Ports
port: comment
443: OpenSearch Dashboards
5601: OpenSearch Dashboards
9000: Cerebro
9200: REST
9250: cross cluster search
9300: nodes communication / transport
9600: required for Performance Analyzer

Original issue here: opendistro/for-elasticsearch-docs#433

Ditto.

Related PR:
https://github.com/opensearch-project/OpenSearch/pull/875

As a part of this pull request, we want to update our documentation to notice users that default values in ${source.path.env} will not be loaded when they set custom OPENSEARCH_PATH_CONF.

Level up and cleanup markdown content.

See .github#8 for details and recommendations, and common-utils for an example.

• README.md
• CONTRIBUTING.md
• DEVELOPER_GUIDE.md
• MAINTAINERS.md
• ADMINS.md
• SECURITY.md
• RELEASING.md
• TESTING.md
• other .md files

The sidebar at https://docs-beta.opensearch.org/ states:

© Amazon Web Services, Inc. or its affiliates. All rights reserved.

That doesn't align with the license statement here. AFAIK no CLA is required, so you'll need something like:

© OpenSearch contributors, 2021. Released under Apache License 2.0.

The README is also less than clear:

Copyright Amazon.com, Inc. or its affiliates. All rights reserved.

Describe the bug
The link to docker hub on https://docs-beta.opensearch.org/docs/opensearch/install/docker/ is pointing to opensearch not opensearch project

To Reproduce
Steps to reproduce the behavior:

1. Go to 'https://docs-beta.opensearch.org/docs/opensearch/install/docker/'
2. Click on 'docker hub'
3. You see an empty repo

Expected behavior
I expect to end up on the repository for opensearchproject not opensearch

Plugins
N. A.

Screenshots
N. A.

Host/Environment (please complete the following information):
N. A.

Additional context
N. A.

Description

Original Issue: opensearch-project/OpenSearch-Dashboards#686

Developer was trying to setup OpenID Connect and following the directions here: https://opensearch.org/docs/security-plugin/configuration/openid-connect/#opensearch-dashboards-single-sign-on

And it failed because it's not the correct config path. The correct config path is called in the plugin as:
opensearch_security

Expected Solution

Update the documentation to display the correct configuration

In _opensearch/install/important-settings.md it says:

For production workloads, make sure the Linux setting vm.max_map_count is set to at least 262144. On the OpenSearch Docker image, this setting is the default.

The part about the OpenSearch Docker image is incorrect. This is something that has to be set on the host manually. See https://github.com/opensearch-project/opensearch-build/tree/c055f04fd11ab2e493ca2b390b1a2af7b451807c/tools/standard-test#requirements

https://docs-beta.opensearch.org/docs/security/configuration/security-admin/

In this page, not contain description of -rev parameter. This parameter makes available resolution enviroment variables before apply Opendistro Security configurations.

Example of use case enviroment variable in config.yml:

password: ${env.LDAP_PASSWORD}
Thanks!

Original issue here: opendistro/for-elasticsearch-docs#386

I found this Search Experience url display is not correct in opensearch.org. Right now it is displayed as
https://opensearch.org/docs/opensearch/ux/ which should be https://opensearch.org/docs/opensearch/search-experience/

We need to clarify that requests using wildcard expressions with the ISM API can affect system indices. This does not apply to ISM templates

Coming from opensearch-project/.github#21.

The correct copyright for open-source projects in opensearch-project is "Copyright OpenSearch Contributors". Please correct any places that say otherwise, especially where it says copyright Amazon. Make sure NOTICE.txt and README match. See opensearch-project/.github#24 for an example.

Cert Reload is a new feature introduced as part of https://github.com/opendistro-for-elasticsearch/security/pull/238. This feature and the API's related to it need to be added to odfe documentation.

Describe the bug

Missing documentation around RPM build.

To Reproduce
Steps to reproduce the behavior:

1. run command" yarn build --rpm
2. Failed to build the RPM.

Expected behavior
The RPM should be built and can be found in the target folder.

OpenSearch Version
1.0.0

Dashboards Version
Please list the version of OpenSearch Dashboards being used.
1.0.0
Plugins
N/A
Please list all plugins currently enabled.
None
Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.