What happened:
RSS feeds in blog are outdated

What you expected to happen:
The feed generation seems broken. Also, a button with a link to the feed on the blog webpage would enable others to also find the feed much easier.

• Use docforge to pull documentation versions
  - pull documentation
  - execute docforge for each version of the documentation manifest in that repo that we need to publish docs for, and target - a corresponding vX.Y folder. Use GITHUB oauth token from concourse in CI builds, and locally provided token in local builds.
• Remove the fetching of remote links form node/index.js. This is performed by docforge.
• Integrate building GitHub (commits) history and contributors in docforge as optional step.
• Adapt layouts to use original document location: edit link/footer issues link/github_contributors partials using the document contentSource. Consider the case for documents with multiple contentSources (created from multiple sources).
• Promote the test pipeline definition to productive.
• Use environment variables or configuration file to set number of support documentation versions published by the build.
• Add documentation for CI/CD pipeline setup for the website-generator

Documentation issue at https://gardener.cloud/docs/gardener/concepts/

Shouldn't the concepts page be more navigable. For someone new, I can't tell how to browse this.

What would you like to be added:
Documentation version support on the website.

Why is this needed:
Currently, it is not possible to pin documentation state go gardener state and answer e.g. "Is this documentation valid for my Gardener version x.y.z?". It is necessary to be able to map documentation version to a Gardener version.

Documentation Engineering

Milestone stage in #51

Publish Versioned Gardener Documentation

Provide the toolset and establish the process and the delivery pipeline for having the Gardener Documentation published with versions on https://gardener.cloud in a manner consumable for other documentation shipments too.

Process

Releases from the gardener/gardener repository trigger automatic pull requests for update of the latest Gardener release version maintained in gardener/documentation repository. Once all supplementary material, such as tutorials and cross-cutting concerns, is ready for documentation release, the release is triggered in Concourse. The gardener/documentation and gardener/gardener release versions match.

Releasing the gardener/documentation triggers a release PR to the gardener/website-generator repo. This automatically starts the process for building a set of documentation bundles, one for each version supported on the website, and the rest of the website assets into a publishable bundle.

Finally, the bundle is pushed to the gardener/website where it is taken over and published by GitHub pages as website content.

The described process applies to the documentation only. Changes to blogs or other sections of the website, beyond documentation are still reflected immediately with an automated build and publish and do not require gardener/documentation release.

Versions scope

The published Gardener documentation versions are up to the 3 latest minor versions, the latest patch number of each.

Tools

To support the process we employ the docforge tool. It is intended to takeover completely from the current variety of tools and scripts employed for site building. Using it, the whole intended website structure can be pulled and forge into source suitable for building and publishing with Hugo, all that regardless of the physical locations of the original sources. The intended website structure is described with yaml manifests. The gardener/documentation repository maintains one and its versioned state is used at build time to build a documentation bundle in a version with the website accordingly.

The Landscape Documentation (LD) UI

Is supposed to gather multiple microsites that consist of documentation separated by the components that the landscape is built from.

As a first step we would like to have a mockup version of the website UI and the way it is structured. For example how are the documentation components defined and how do we navigate trough each. Another thing for example is how the API references for each component is displayed. What is left to be decided is either we create a LD UI for each landscape instance or provide the possibility to navigate between them by a dropdown menu (even between versions that are "to be released"). From one side this depends on how the LDUI will be consumed from the Dashboard UI for example and other possible ways to get to the site. Such things should be taken in consideration.

What would you like to be added:
When the documentation repository releases a hotfix, the website-generator should build a new version of the website with the latest documentation patch version.

Normally, repositories are configured to auto-assign code owners from the repo to PR reviews to that repo. When it comes to documentation, some processes require a documentation expert to be involved too. However, not every PR is strictly code or strictly documentation and normally documentation experts are not code-owners but rather play a cross repository function.

It would be beneficial to make auto-assign of reviewers smarter and if there are markdown documents in a PR, to assign a list of predefined documentation experts automatically too.

What happened:
The websites with Docsy theme does not format correctly the short codes, e.g. notice note in gardener/documentation#123

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know:

Environment:

Documentation Engineering

Summary

This project approaches the problem of documentation development by approximating it to the problems in source code development and addressing it with software and process engineering. It provides a high level of automation and software support, comparable to the one in a best practice, complex end-to-end development processes, particularly in OSS projects.

In a nutshell it:

• fuses the documentation process into the software development process
• utilises the same process automation infrastructure - CICD, containers, bots, automatic PR checks
• supports separation of concerns between documentation providers (domain experts creating content) and documentation publishers (experts in design and delivery of documentation content)
• frees capacity for creative work, automating procedural tasks in the documentation process from development of material to publishing it on a publishing channel
• supports complex, multicomponent projects spread across multiple repositories that have established end-to-end processes

Motivation

Documentation development and delivery is traditionally seen as a separate domain that eventually has cross points with software in the production process. But paradigm shifts in software engineering like agile processes and design thinking bring closer the previously distinct roles in the process and they start to exchange and adopt process models and tools in a more unified approach. That is particularly visible in open source projects, where dynamism and collaboration but also the "do-more-with-less" approach are survival factors. These are some reasons why we see a new paradigm at documentation development called documentation-as-code making its way more and more and particularly in open source and especially in complex projects.

Often this approach is described as keeping your documentation close to code and using the same tools as developers to work on it. While this is true there is a lot between the lines here. Considering that this means having a docs folder with markdowns in your repo and using git to check in and out and pull requests to collaborate over changes is a good start, but it is almost certainly an oversimplification of the potential of the documentation-as-code paradigm. It wouldn't bring significant benefits to a complex projects with CICD-orchestrated automatic processes, consisting of multiple repositories, each being a component that needs documentation, like Gardener or Kubernetes for example. In fact most claims for implementation of documentation-as-a-code in OSS projects that we see stop short at the tool support in the initial development material phase on a per repo basis. When documentation material moves close to code for good, and there are multiple repositories to consider, we face a distinct issue. How the documentation content from multiple repositories can be aggregated in a consistent, publishable documentation bundle and qualify it. And then how to release and deploy documentation with software releases, considering product and component versions.

Applying the documentation-as-code paradigm implies a high degree of automation and software support, comparable to the one in a development process, which is a software engineering problem. In fact, the documentation development process resembles significantly the software development process in that way. Coming from this angle, this project addresses the documentation process problems with software engineering means not only focusing on the initial phase of documentation development but all the way along the whole process down to a delivery on a publishing channel.

Proposal

The proposal is to define the practices and deliver the software necessary to support a documentation process that shares the same infrastructure as software development process for multicomponent, multi-repository, GitHub-based projects, that can be used to deliver documentation on multiple publish channels from the same source.

Formally such documentation process can look like this:

Or if we apply this concept to the delivery of documentation by the Gardener OSS project:

Material Contribution

A documentation material contribution, being new material or existing one pulled for update, is the usual GitHub-driven process with change commits, collaboration and reviews until a final agreement, known from contributing code.

Developing documentation on this stage can be supported by local editor, containerized build and preview of changes, or the same provided by an online CMS, such as netlify CMS. Running the same build as CICD and previewing changes as they will be seen by end users on a publish channel, or at least the documentation bundle that will be transformed for that, introduces a transparency that significantly reduces the turntime for a change.

Introducing formal enforcement of guidelines such as markdown formatter, or early and formal quality check, including dead links, spelling and expression checks is a significant improvement reducing review times and removing the burden of trivial checks.

Finally, merging documentation changes to a repository is subject to automatic formal checks that guard the quality of the contributed material. These are mostly the same quality checks that are available to be run while developing material.

Build

Building a documentation bundle has several stages. The common one is the initial - aggregating source content into a desired structure. The next build stages depend on the purpose of the bundle and the publishing technology and may require technology-specific transformation. Therefore they can be customized as needed for the specific use-case.

Building starts from a manifest file that describes the desired documentation bundle and its structure. It is convenient to manage this file in a repository dedicated to documentation or a root component of the project.

The product of the build is a documentation bundle publishable on a particular platform.

Continuous Integration and Continuous Deploy (CICD)

Projects that have CICD setup to orchestrate change management will have the documentation reusing it for coherent release and deploy cycles. When a documentation change is merge a CICD job is triggered to build the documentation bundle and run quality checks. Upon release the build result, that is the documentation bundle and source are added to the release artifacts. Continuous deploy uses those artifacts to publish them on a target publish channel.

There can be more than one publishing channel and normally that will require also different documentation bundle possibly structured precisely for that channel and optionally built for a specific publishing platform. Publishing on each channel can be performed by dedicated CICD pipelines.

Summary of topics

The project supports documentation development process similar to software development contributing software support in the following directions:

• Material change contribution
  • Sandboxed build
  • Change Previews
  • Quality and compliance control
  • Pull requests quality control gardener/docs-toolbelt#8
• Build documentation bundle gardener/docforge#23
  • Pull documentation sources from multiple locations and storage system types (incl. GitHub and file system) as instructed by a documentation bundle manifest
  • Reconcile linked resources in documents - download as appropriate and rewrite links gardener/docforge#7
  • Applying automatic processing on source documents, including for a particular build system (e.g. Hugo) gardener/docforge#20
• CICD integration guidelines

Validation Scenarios

OSS project

The scenario validates the extremely popular among OSS project process organization with GitHub-based dev process, CICD integration, build with Hugo, publish on GitHub Pages. The project to validate it will be Gardener and the expected result is versioned documentation bundles published at https://gardener.cloud upon Gardener release.

Internal project

The scenario validates the versatility of the concept and the implementation in a different environment and process. For example, publishing documentation in a DITA system. A stakeholder with more concrete scenario is to be defined.

Definition of DONE

• Validation scenarios accomplished
• Documented
  Documentation for adopters, and developers
• Tested
  75%+ avg code coverage with tests, integration tests
• Stable
  The project concepts and core implementations are stable enough to adopt and extend with new tools in the given scope without significant changes.
  • 1+ more alternative(s) to a supplied solution in the scope

In the graphic kubernetes/documentation, kubernetes/gardener-generator, and kubernetes/gardener are used.

kubernetes should be gardener.

Unfortunately I don't have a sketch license, so I can't adapt it.

This is an umbrella issue that is meant to track the different steps to setup the project infrastructure.

• Create a dedicated repository that is going to manage the orchestration of the LDUI
• Add owners and contribution guide
• Manage permissions
• Setup a pipeline definition for the CI/Cd
• Make the repository a component

Documentation Engineering

Milestone stage in #51

Landscape Deployment Documentation

Each Gardener landscape consists of component in a certain version that collectively constitute a BoM for that landscape. Operators and users need to be able to access the documentation for the exact version of the components in the landscape, preferably efficiently, without searching for it among other components and versions not relevant for that landscape.

The documentation bundle for a landscape should be built automatically e.g. using the static Gardener component model.

The documentation should be published on a dedicated website and accessible via the landscape Dashboard UI.

This epic can be considered a special case and a subset of #52 .

Why is this needed:
As we have discussed, the default theme should be switched to Docsy, because most of the features we want to implement,
are already existing in Docsy.

Changes needed to be done before switching to Docsy:

• Update docforge manifest with reorganised blogs.
  • Sort blogs by date (latest on top).
  • Do not use section files.
• Make adopters page in Markdown format
• Remove home folder from the content
• Integrate a search field
• Remove type field from the frontmatter
• Handle docsy submodule in the CI/CD
• Add documentation for the .gitmodules

In parallel to #54, we can start developing a microsite that is able to be imported as a component along side with other microsites.
In order to achieve this we need a common theme and layout that has to be used by those microsites.
As done criteria we expect to have multiple micro websites that use the same layout and theme and are imported into a
a website (LDUI for example) that is combining them into one.

Documentation Engineering

Milestone stage in #51

Multi-Components Documentation Publishing Platform (DocHub)

Leveraging #50, the next stage is to turn the documentation on the website into a multi-component documentation publishing platform.

• As "New to Gardener", I want to explore, which are the components that the project supports and learn more about them
• As a Gardener Stakeholder, I want to pin the documentation for the particular release of the components that constitute my landscape deployment
• As a "New to Gardener"/Gardener stakeholder, I want to explore components documentation to plan upgrade or adoption of a component in a particular version
• As a Gardener Component Owner, I want to register my component documentation to be published automatically to the website upon its every release
• As a Gardener Component Owner, I want to be responsible for authoring, structuring and compliance of my material with the publishing platform
• As a Gardener Component Owner, I want to preview changes to my component documentation early and before published
• As a Gardener Documentation Website Master, I want to have new component documentation and its releases published in coherence with the established website framework automatically and efficiently

Motivation

The documentation on gardener.cloud currently is Gardener-centric, which serves most of the cases, but it leaves out other components, such as extensions. Their documentation is dispersed all round the org repos and one needs to leave the website to exploring it, which considering 80+ repos (as of now) in different state and purpose is not easy task. Instead, we would like to have a single Documentation Hub (DocHub), a one stop shop for people to find documentation on anything in the project. And we want it to scale and stay future-proof.

Concerns

Process Transparency

While the integration with the publishing platform requires some insight into how to comply with the website rules, so that the result looks coherent, technically it needs to be as transparent as possible. The role of a component owner needs to keep most of its focus on providing documentation structured according to the rules. To put it simply, they do not need to know how their content makes its way from the repo to the website as long as it's there.

Matching website and content versions

Website can and will evolve. Layout and configuration will change. Considering a multitude of versions of component documentation bundles that will quickly explode. A breaking change in the rendering such as deprecation of a shortcode in favor of another will then become an overkill to apply.

It should be possible to maintain older content without changes, if that's acceptable, or stage the migration by changing the site reference when it is convenient to make the move.

UX

Working on the website interface with a multitude of components each with unlimited number of documentation versions is a challenge in terms of user experience. The site layouts need to account for that and present effective pattern for locating a component and a concrete version of a document among many.

Scalable site building

We need a predictable and controlled load on the build process, and within a reasonable timeframe range, and storage volume.

Search index

The component documentation bundle versions need to be ready for indexing by a local search solution or external indexing service such as algolia or google.com.

Change Preview

A concept for preview of changes is necessary to minimize turn round times and provide sufficient idea of how the final change will look like, even if it's partial.

What would you like to be added:
During website publishing Community Page shall be retrieved from documentation repo
Why is this needed:

Currently it is taken from hugo/layouts/community/list.html but exists also in documentation repo: https://github.com/gardener/documentation/tree/master/website/community

To accomplish a Landscape Documentation UI that gathers multiple components (like Gardener, Gardener Dashboard, Machine Controller Manager and etc.) in the form of microsites, most simply we need a frame (header with navigation and footer) and we need a mechanism to combine those multiple microsites into one. In the end we should have a website that has different documentation components built separately.