Docs URLs are not versioned about documentation-website HOT 32 CLOSED

@AMoo-Miki I would prioritize this, we're gaining SEO points every day with an URL structure that will change; with additions like #127 we're going to cause a lot of churn in Google when we fix this

from documentation-website.

@dblock, @AMoo-Miki, and @aetter my instinct is that we should have a default as well that goes to the most current launched version. So when 1.1 launches /docs/<path> would go to /docs/1.1/<path>. Thoughts?

Yes, as long as it's a 302 redirect, since it will change with every release. We don't want anything actually indexed under docs without a version.

from documentation-website.

Agreed, let's get this fixed especially since there is already a 1.1 launch not to far down the road.

from documentation-website.

@dblock, @AMoo-Miki, and @aetter my instinct is that we should have a default as well that goes to the most current launched version. So when 1.1 launches /docs/<path> would go to /docs/1.1/<path>. Thoughts?

from documentation-website.

Hi @elfisher, there's a lot of value in showing the absolute best, latest version of the docs, so we should continue showing the most-recent commit on main by default. That commit will always be for the most-recent launched version, so it fills the same need without a redirect, which has the pleasant side effect of making bookmarks easier to understand (i.e. I can bookmark docs/current/page.html rather than getting redirected to docs/1.1/page.html and having to know that I can remove 1.1 from the URL and bookmark docs/page.html instead). The only real loss is that the URL doesn't include the version number, which doesn't seem like a big deal.

The 1.1, 1.2, 1.3 tags are all "here's what we had at time of release," so redirecting to them would always put readers days or weeks behind the latest fixes and improvements.

from documentation-website.

@aetter I don't think this is a good idea to show the latest and greatest without a version path or under current. Several problems:

1. Anyone with a bookmark or link from a blog post about an upcoming or current feature will link to documentation that will definitely become out of date, and that link will eventually point to incorrect documentation of a future version from the time it was written. We will never be able to rename or delete pages and people will be pointed to more and more of the wrong docs over time. Obviously broken links is bad. And remember it's not just about docs we author, it's also JavaDoc that is/will/should be published along that is impossible to track page-by-page.
2. The "current" documentation being the next release (right now) means it contains things that won't work for users (e.g. settings introduced in 1.1).
3. Google indexing current will suddenly find tons of duplicated content under 1.1 at the time when 1.2 ships. No building SEO for 1.1 ahead of time. A little bird tells me that Google understands versioned documentation under different paths.
4. We will need to do extra work of moving current -> 1.1 when we release 1.2.

This is how I expect it to work:

0. Everything /docs just redirects to the latest good release, today it's 1.0 and is stable.
1. As we develop 1.1 today, we publish daily current documentation to 1.1.
2. If we have improved docs, and need to make a change in 1.0 docs, we can backport those changes and publish them to /1.0 again.

WDYT?

from documentation-website.

The 1.1, 1.2, 1.3 tags are all "here's what we had at time of release," so redirecting to them would always put readers days or weeks behind the latest fixes and improvements.

Include a footer or header in every doc that says "you may not be reading the docs for the latest release of OpenSearch, click here for the latest"? I'd take it further, if the release is no longer supported that link would be "you're reading docs for an unsupported version of OpenSearch, ...", etc.

from documentation-website.

1. Actually, it's the opposite. If I go to docs/page.html, get redirected to docs/1.1/page.html, and then copy the URL for my blog post, I'm now redirecting every reader to 1.1 forever. If docs/ sends you to docs/current/page.html, there's no problem, because the content on that page will always be up-to-date.
2. We merge everything that's specific to the upcoming version on release day, so there's no problem.
3. N/A
4. current tracks the latest commit, and every version is a tag.

from documentation-website.

Include a footer or header in every doc that says "you may not be reading the docs for the latest release of OpenSearch, click here for the latest"? I'd take it further, if the release is no longer supported that link would be "you're reading docs for an unsupported version of OpenSearch, ...", etc.

That is an extremely poor user experience.

from documentation-website.

1. Actually, it's the opposite. If I go to docs/page.html, get redirected to docs/1.1/page.html, and then copy the URL for my blog post, I'm now redirecting every reader to 1.1 forever. If docs/ sends you to docs/current/page.html, there's no problem, because the content no that page will always be up-to-date.

Exactly. The former is the desired behavior. If I am reading a blog post about how to install logstash-opensearch-output on OpenSearch 1.0, and it says "for additional settings read docs[link]", why in the world would I want to read docs for OpenSearch 2.0?

from documentation-website.

That is an extremely poor user experience.

Seems like incorrect documentation is a worse user experience.

from documentation-website.

1. Actually, it's the opposite. If I go to docs/page.html, get redirected to docs/1.1/page.html, and then copy the URL for my blog post, I'm now redirecting every reader to 1.1 forever. If docs/ sends you to docs/current/page.html, there's no problem, because the content no that page will always be up-to-date.

Exactly. The former is the desired behavior. If I am reading a blog post about how to install logstash-opensearch-output on OpenSearch 1.0, and it says "for additional settings read docs[link]", why in the world would I want to read docs for OpenSearch 2.0?

If our hypothetical blog post author wanted to link specifically to version 1.0, it's available as a permalink at docs/1.0/page.html. Use the dropdown to select it or just modify the URL. If the author wanted to link to the best, most recent content (newest commit on main branch), it's available at docs/current/page.html, which is what the URL bar displays.

There's no ambiguity in what people are linking to or bookmarking, so this notion you're perpetuating that people will get constantly linked off to the wrong version of the documentation and be confused as to what they're looking at is not in line with reality.

from documentation-website.

Are we just arguing whether to have current during development of 1.1, and then copying current to 1.1 when 1.1 is relased, in the URL? I guess I'm ok with that, except that main is not current, it's the next release, but maybe I'm just being pedantic. Right now there's no current in the URL, though, moving everything from docs to docs/current and creating docs/1.0, would definitely be fixing my problem for the most part.

from documentation-website.

I'm hopeful that we're actually on the same page on the desired user experience, but I think there's a misunderstanding of how we develop the documentation:

• main isn't the next release. It's the most up-to-date content at any moment in time. For example, we have a tag for 1.0. It's probably 50 commits behind main. Directing people to documentation built off of the 1.0 tag (docs/1.0/), missing our 50 latest commits, is not good, especially when main is also for 1.0, because that's the latest version at this moment in time.

• Currently, we keep all content for the next version sitting in PRs and merge it into main on release day. There usually aren't very many PRs, and they rarely conflict. After we release and the site looks good, we tag. If this process ever gets too hectic, no problem, we can add a develop branch and use that as a staging ground for the next version, merging it into main on release day.

• We don't backport fixes to old versions of the documentation.

So there's no copying, and we shouldn't redirect to anything other than docs/current, which maps to main:

1. User visits https://opensearch.org/docs/ and gets sent to https://opensearch.org/docs/current/, which auto-builds off of main whenever a new commit hits the branch. This way, the default behavior is for users to be presented with the most accurate, up-to-date content. We never have to redo the redirect, and if someone creates a bookmark, there's an indication right in the URL that the content could indeed change over time.
2. If the user wants a different version, they use the dropdown menu (does not exist yet) to choose that version. The URL changes to https://opensearch.org/docs/1.1/, which is completely static and dependable over time (unless we really mess up and have to backport a critical fix).

The reason "current" isn't there at present is because, well, we only have the one version. You can see something similar in the Prometheus documentation and anything on ReadTheDocs (though both use "latest" instead of "current").

This strategy keeps things super simple and keeps us out of cherry picking hell. Everything goes on main, which gets auto-built to "current" within seconds, we tag for releases so that we can build point-in-time versions for the dropdown, and that's it. There's no ambiguity about where a PR should go or which branch we're building from.

Consider the alternative:

1. User visits https://opensearch.org/docs/ and gets sent to https://opensearch.org/docs/1.0/, because that happens to be today's most-recent release. Does the content have all the latest fixes (meaning it's a branch, cherry picking hell), or is it 1.0 at time of release (meaning it's a tag, likely out of date)?
2. If the user wants to bookmark the latest version of that page, the page that will stay up-to-date over time, rather than the 1.0 version, how do they do so?
3. We have the added complexity of changing the redirect for every release, along with changing where the automated build system posts content.

from documentation-website.

Appreciate the long explanation.

I still think we'll run into the problem I describe sooner than later with dead links, especially with Javadoc. I don't see how we can maintain forever stable documentation with no files being deleted, or content being completely superseded major release after major release, and I don't see those who link to the docs remember to pick and choose a link to a specific.

So disagree and commit from me! Thank you. If you feel like this issue should be closed, go for it, no hard feelings.

from documentation-website.

So disagree and commit from me! Thank you. If you feel like this issue should be closed, go for it, no hard feelings.

No hard feelings here at all -- having these sorts of discussions in the open leads to better outcomes, so I'm glad we got to have the back and forth. I think we absolutely want to keep this issue open so that @AMoo-Miki can weigh in on feasibility, discuss any issues during the implementation process, etc. We've still got a bit of a road before this becomes reality, but it's something we need sooner rather than later.

from documentation-website.

I don't think the version of OpenSearch will dictate the version of various plugins; will it? Will increments of plugins always and only be tied to releases of OpenSearch? While that is probably true for Dashboards, I have kept them all separate in my logic.

The way I have been developing this is by having specific versioned folders within each doc-component that wants to have versioning. OpenSearch, Dashboards, and each of the plugins will have their own versions that are not tied to each other.

As a user, the left menu will list the component and default to the latest version when expanding its submenu, while allowing me to choose a different version via a drop-down. Hence, manual navigation doesn't rely on non-versioned paths.

As a user looking at a page, I see an indicator of the component and its version right on top of the page; this too has a drop-down to change the version.

As a user, any bookmarks I make will be specifically to a versioned page and will have a versioned path. Documentation could have a visual indicator in the form of color or text to inform me that I am reading about an older version of the component.

Having versioned folders will give maintainers more flexibility and control over how the content is structured and when the number of pages across all versions becomes unmanageable, the documentation for the component can move out to its own repository and this repo will use Git's subtrees to reference various versions that are maintained across branches.

from documentation-website.

I don't think the version of OpenSearch will dictate the version of various plugins; will it? Will increments of plugins always and only be tied to releases of OpenSearch? While that is probably true for Dashboards, I have kept them all separate in my logic.

There will always be a release of OpenSearch as a bundle called "OpenSearch", the product with all the plugins. One day we might change this and release OpenSearch as a bundle with some plugins, then each other plugin might go have a life of its own, but we're not close to that.

The way I have been developing this is by having specific versioned folders within each doc-component that wants to have versioning. OpenSearch, Dashboards, and each of the plugins will have their own versions that are not tied to each other.

C.E.D., this is no longer our reality.

As a user, the left menu will list the component and default to the latest version when expanding its submenu, while allowing me to choose a different version via a drop-down. Hence, manual navigation doesn't rely on non-versioned paths.

As a user looking at a page, I see an indicator of the component and its version right on top of the page; this too has a drop-down to change the version.

As a user, any bookmarks I make will be specifically to a versioned page and will have a versioned path. Documentation could have a visual indicator in the form of color or text to inform me that I am reading about an older version of the component.

Cool. I was just pointing out to the fact that this is not the current experience.

from documentation-website.

One thing to contradict myself. Not everything in the docs is a plugin that ships with OpenSearch, the product. For example, logstash-output-opensearch is its own thing.

from documentation-website.

Hi @AMoo-Miki we chose a single repository for the documentation based on the decision that the core components and plugins would all share a common major and minor version number. In other words, the table here should never read OpenSearch 1.2 | Alerting 3.5.0.1, Index Management 5.2.0.0, etc. The common version numbers let us tag everything in one shot and keep the website to a single version switcher (rather than one switcher per component). It also leads to a simpler user experience, in that there's infinitely less complexity in terms of trying to parse what's compatible with what.

I'm not wildly in favor of an old version banner, because that probably means the version switcher isn't prominent enough. If it's highly visible, the version switcher fills that same need. That said, it's not a hill I'm particularly interested in defending, so I'll bow to the wisdom of the crowd if everyone else wants a banner.

from documentation-website.

Thanks for the clarifications. With a common version number for every component, the docsite will have to take a different approach.

I see 4 options for the structure of this repository:

1. A bunch of versioned root folders, one for each minor release:
   • Folders like 1.0, 1.1, and 2.0, residing in the root of the repo will each contain the complete documentation for that version.
   • Contributions to the main branch will trigger the release pipeline which will have all the versions built and released.
   • I don't believe the current website generation software can handle this structure; modifications or a new generator will be needed.
   • A drop-down above the left menu will allow switching between versions.
2. Branching to separate versions from each other:
   • Branches named like 1.0, 1.1, and 2.0 will contain the the documentation as well as the artifacts that make up the microsite.
   • For every version bump, a new release pipeline will have to be created to keep the automation for previous branches; this is because the existing ecosystem is branch-dependent. Alternately, we can look for a new build and release pipeline.
   • A drop-down above the left menu will allow switching between versions.
3. Tagging to separate versions from each other:
   • Tags like 1.1.0, 1.1.1, and 2.0.0 will be applied to commits and the greatest version within each minor will be the one that is selected for release: the greatest of 1.1.0, 1.1.1, and 1.1.2 will be 1.1.2 and it will be released as 1.1.
   • Changes to docs of a previous version will involve branching and tagging in that branch; other techniques will be prohibitively complicated for contributors.
   • Since the existing build system is based on branches, we will have to look for a different system.
   • A drop-down above the left menu will allow switching between versions.
4. We don't change anything with the structure of the docsite. Instead below the title of every feature, method, or property, we will have a label indicating its availability.
   • Those still available will say Introduced with 1.0.
   • Those no longer available will Available from 1.0 to 2.1.
   • A less wordy option which would reduce confusion is 1.0 ≤ Available and 1.0 ≤ Available ≤ 2.1.
   • A page could by default only show the headings that are available in the latest version and offer the user a toggle to show all headings. If the user arrives with a hash that points to a discontinued headings, the page will automatically expand to show everything. The choice of expanded or not can be remembered throughout that visit.

I would love to hear your thoughts on these or any other proposal you might have.

from documentation-website.

@AMoo-Miki Yep, 3 is what I'm proposing. If the current build system can't handle tags for whatever reason, we can branch at the same time that we tag.

from documentation-website.

I think (2) is how we develop code, isn't it? Which would then be my preference.
https://github.com/opensearch-project/.github/blob/main/RELEASING.md#branching

from documentation-website.

Thanks for your inputs. Here is my proposal:

• Documentation will be versioned at root of /docs to allow for /docs/1.0/, /docs/1.1/, and ...
• A file in the _data folder, _data/versions.yml, would indicate the current version and all other versions that should be made navigatable.
• The release mechanism will look at the main branch for updating the content of the folder named after the current version number found in _data/versions.yml.
• Before switching to a new version, a new branch will be cut named as the current version. Then, _data/release.yml will be updated to add the new version and move the current version to the navigatable list.
• A second pipeline will accept an entry for an admin to build and release a specific version from the past; this is for rare situations requiring changes to docs of a previous release.
• A version selector above the left menu will inform the visitors of the displayed version while allowing them to switch.
• Current and all previous versions will load a "component" from /docs to populate their version selector.
• /docs/latest/ will always point redirect/rewrite requests to the latest version.
• /docs will redirect to /docs/latest

Comments and queries are welcome.

from documentation-website.

Thanks for your inputs. Here is my proposal:

LGTM

• A second pipeline will accept an entry for an admin to build and release a specific version from the past; this is for rare situations requiring changes to docs of a previous release.

If you want a controlled release you don't have to have that control in some other build system - move it into source control by automating building these branches on every change. By deciding that the code/docs in release branches must always be in a publishable state you will thus save yourself a ton of future clicking. Publishing is then a merge into the release branch(es) and not a button that an admin has to click. The nice side effect of that is you don't need to give permissions to the build tool to your maintainers, you only need to manage merge rights on GitHub. This further increases transparency and trust in the process.

from documentation-website.

• /docs/latest/ will always point redirect/rewrite requests to the latest version.
• /docs will redirect to /docs/latest

Hey @AMoo-Miki, just to clarify, the goal is for latest to be its own directory that automatically builds off of the most-recent commit on main. So on the server, we'd have:

• /docs/latest
• /docs/1.1
• /docs/1.0
• etc.

And then /docs redirects to /docs/latest.

The reason is that if I come to the documentation this week, get redirected to /docs/1.0, and create a bookmark, my bookmark will refer to that old version forever, and it wouldn't be clear how to create a bookmark for the latest version.

I'm totally fine with /docs/latest and /docs/1.1 showing the exact same content and in fact being duplicative builds for the duration of 1.1 being the latest version, but readers need an easy way of saying "give me the latest version of this page" that doesn't involve manually changing the URL.

Everything else looks great. 👍

from documentation-website.

The reason is that if I come to the documentation this week, get redirected to /docs/1.0, and create a bookmark, my bookmark will refer to that old version forever, and it wouldn't be clear how to create a bookmark for the latest version.

If latest is the latest release, and there's no 1.0 (like right now), I can't get a stable link to 1.0 docs.

We could have current or next that people would bookmark, but latest would be a 302 to 1.0?

from documentation-website.

Thanks for your inputs.

If you want a controlled release you don't have to have that control in some other build system - move it into source control by automating building these branches on every change.

Since our current release channel is tied to a single branch, to maintain the full automation, we will have to create new pipelines for every new version. With this proposed approach of having one pipeline for main and one for manually selecting the branch for older versions, we will simply reduce the effort until we move to a more flexible release pipeline.

Hey @AMoo-Miki, just to clarify, the goal is for latest to be its own directory that automatically builds off of the most-recent commit on main

I was thinking of having a urlrewrite rule to rewrite requests on /docs/latest to /docs/<latest-version>. Upon each release, this would get updated to point to the new release folder. I could also simply copy the content of latest-version into the /docs/latest folder.

We could have current or next, and latest would be a 302 to 1.0?

With this proposal, we will not have /docs/current/ or /docs/next. The only non-numeric entry point will be /docs/latest. If the current version was 1.2, we would have /docs/1.2 and /docs/latest serving it. We would also have /docs/1.0 and /docs/1.1 serving previous content. When a version 1.3 is released, we modify _data/versions.yml, setversions.current to 1.3, and add 1.2 to versions.available. The release will upload the artifacts to /docs/1.3 while /docs/1.2 will be untouched. Then, the new content will be copied into /docs/latest or the url-rewrite rule will be updated to serve the new content when it is requested.

but latest would be a 302 to 1.0

If the website does a 302 or 307, the URL in the browser will change and bookmarking will be tied to version. Serving the content behind the scene, with a url-rewrite or by just duplicating the content, will prevent that. Despite its challenges, I prefer the url-rewrite due to the flexibility of handling a scenario where a bookmarked page no longer exists in the new versions; the url-rewrite could perhaps send the browser one level up, instead of showing an unhelpful 404.

from documentation-website.

If the website does a 302 or 307, the URL in the browser will change and bookmarking will be tied to version. Serving the content behind the scene, with a url-rewrite or by just duplicating the content, will prevent that. Despite its challenges, I prefer the url-rewrite due to the flexibility of handling a scenario where a bookmarked page no longer exists in the new versions; the url-rewrite could perhaps send the browser one level up, instead of showing an unhelpful 404.

Perfect, yeah, the goal is for the default case to be "I enter the website, and my URL reads latest." If someone wants a permalink for a particular version of the documentation, they can use the dropdown, choose it, and now the URL reads 1.1 (or whatever). If you can achieve those cases without duplicating the content, even better. 👍

from documentation-website.

* Current and all previous versions will load a "component" from `/docs` to populate their version selector.

This should read:

• Current and all previous versions will load a "component" from /docs/latest to populate their version selector.

from documentation-website.

If you want a controlled release you don't have to have that control in some other build system - move it into source control by automating building these branches on every change.

Since our current release channel is tied to a single branch, to maintain the full automation, we will have to create new pipelines for every new version. With this proposed approach of having one pipeline for main and one for manually selecting the branch for older versions, we will simply reduce the effort until we move to a more flexible release pipeline.

Where's that pipeline? If you were to use GHA, everything can work automatically as it picks up workflows in newly created branches without any work.

With this proposal, we will not have /docs/current/ or /docs/next. The only non-numeric entry point will be /docs/latest. If the current version was 1.2, we would have /docs/1.2 and /docs/latest serving it.

OK, this is clear. I felt odd to have two places serving the same content, but I understand why you would want that (to enable both links).

from documentation-website.

Thanks for your inputs; I am on it.

from documentation-website.