What would you like to be added?

Provide a search API, this API's handler invoke storage layer interfaces to search multi-cluster resources from underlying storage.

Topological relationship:

flowchart TB
  subgraph "Karbour"
    A[Search API] --> B[Storage Layer]
  end
  B[Storage layer] --> C[Strorage Backend 1]
  B[Storage layer] --> D[Strorage Backend 2]
  B[Storage layer] --> E[Strorage Backend 3]
  style A stroke-width:4px

Why is this needed?

No response

Support the kubeconfig upload api

What would you like to be added?

Add uniresource type to obtain multi-cluster resources from storage

e.g.

curl -k  https://127.0.0.1:7443/apis/search.karbour.com/v1beta1/uniresources?query=name%3Dtest
{
  "kind": "UniResource",
  "apiVersion": "search.karbour.com/v1beta1",
  "objects": [
    {
      "apiVersion": "v1",
      "kind": "Namespace",
      "metadata": {
        "name": "test1",
        "namespace": ""
      }
    },
    {
      "apiVersion": "v1",
      "kind": "Namespace",
      "metadata": {
        "name": "test2",
        "namespace": ""
      }
    },
    {
      "apiVersion": "apps.cafe.cloud.alipay.com/v1alpha1",
      "kind": "CafeDeployment",
      "metadata": {
        "name": "test1",
        "namespace": "test1"
      }
    },
    {
      "apiVersion": "apps.cafe.cloud.alipay.com/v1alpha1",
      "kind": "CafeDeployment",
      "metadata": {
        "name": "test2",
        "namespace": "test2"
      }
    },
    {
      "apiVersion": "v1",
      "kind": "Pod",
      "metadata": {
        "name": "test1",
        "namespace": "test1"
      }
    },
    {
      "apiVersion": "v1",
      "kind": "Pod",
      "metadata": {
        "name": "test2",
        "namespace": "test2"
      }
    }
  ]
}

Why is this needed?

No response

What would you like to be added?

List the document outline on the product's official website.

Outline:

• Getting Started
  • Installation
  • Quick Start
• Concepts
  • Architecture
    • Components
    • How Works
• User Guide
  • Multi-Cluster Management
    • Register Cluster
    • Edit Cluster
    • Rotate Certificate
    • Remove Cluster
• References
  • CLI Commands
  • OpenAPI
  • Search Ways

Why is this needed?

No response

What would you like to be added?

ClusterSyncer is automatically installed when registering the cluster

Why is this needed?

So that Karpor can perform search and insight capabilities based on the data synced.

What would you like to be added?

The design for the resource syncing component that collects data on resources in the member cluster and store them into Karbour's storage layer.
User should be able to customize the rules for the resource sync.

Why is this needed?

This is the design doc for implementing ClusterSyncer.
Customized resource sync rules are to give user some flexibility in terms of configuring Karbour's behavior.

What would you like to be added?

The design for the data transforming component that transforms resource data before storing them into Karbour's storage layer. The transformation should happen based on user's customized configuration.

Why is this needed?

This is the design doc for implementing the Transformer.
Transformers are to enrich the data being collected into Karbour's storage so it can provide additional, advanced search and insight capabilities.

What would you like to be added?

Add an issue-based score to insight details which appears on the:

• resource detail page
• cluster detail page

Source of the issues is kubeaudit: https://github.com/Shopify/kubeaudit#auditors
Design is located here (will move to github eventually): https://yuque.antfin-inc.com/antcloud-paas/ar858o/uyxg590vqsba8xfa
Should also include the list of issues if found any.

Why is this needed?

This implements one of the core insight functionalities that Karbour presents.
Karbour provides resource scores (for each individual resource) and a cluster score, which represent the extent the resource or the cluster complies with security policies and best practices.
In the future Karbour should also provide capabilities to address the issues identified.

What would you like to be added?

Add the kubernetes object analysis to insight page with recommendations for improved reliability and security.

Can refer to:

• kube-score

Why is this needed?

No response

What would you like to be added?

This work should happen after the search/insight APIs are properly designed.
This tracks work to have the Karbour portal properly integrated with the search/insight APIs mentioned in #64 #65 #66

Why is this needed?

This completes the product and customer-facing layer of Karbour Search and Insight capabilities.

What would you like to be added?

Apart from the topology API for a single resource (#65), aggregated topology (Cluster/Namespace) should include GVK and its count

Why is this needed?

So that aggregated topology are displayed properly by frontend

What would you like to be added?

support setting cross-domain options.
e.g.

--cors-allowed-origins=.*

Why is this needed?

Frontend cross domain call server reporting error

What would you like to be added?

Add a relationship topology graph to resource detail page, which describes all the resources that are related to the current one.
User should be able to define custom relationships between resources to accommodate CRDs.
Without any customization, Karbour should have a built-in topology graph describing relationships between Kubernetes-native resources such as (but not limited to):

• Deployments and ReplicaSets and Pods
• Services and Pods
• Ingresses and service
• PVCs and Pods
• PVCs and PVs
• etc

Why is this needed?

This implements one of the core insight functionalities that Karbour presents.
For each resource in the cluster, Karbour provides the ability to locate all the relevant Kubernetes resources and display them on a single graph for user to navigate.

What would you like to be added?

APIs for user to create/read/update/delete Cluster CR in order to create/modify/delete registered clusters

Why is this needed?

Cluster can be added, modified or deleted

What would you like to be added?

End to end verification of Karbour installation including a working ResourceSyncer

Why is this needed?

So that installation works as expected. Installation is step 0 of Karbour adoption.

What would you like to be added?

Embedding front-end static files into karbour binary, local installation & used out of the box.

Local startup:

$ karbour start [Startup_Parameter]

Open 127.0.0.1:7443 in browser, enjoy!

What would you like to be added?

Documentations that describes how to register an existing cluster so that it's managed by Karbour.
The content should include:

• What information is needed when registering a cluster
• What connectivity does Karbour need when registering the cluster
• The RBAC requirements (read-only) when registering the cluster
• Examples

Why is this needed?

Users should be able to refer themselves to the documentation when they need to register an existing cluster to Karbour.

What would you like to be added?

A unified scoring API for single or multiple (aggregated) resources that takes the list of issues as input, and produces a score as output

Why is this needed?

Resource score is properly calculated both for a single resource and for aggregated resources

What would you like to be added?

add search api group

Why is this needed?

No response

What would you like to be added?

Create a design for the core workflows of Karbour

Why is this needed?

No response

What would you like to be added?

Karbour installation should include a centralized version of ResourceSyncer.

Why is this needed?

ResourceSyncer needs to be installed to collect data from all the registered clusters.

What would you like to be added?

an API that accepts a kubeconfig file upload and respond with the content of the kubeconfig to the frontend

Why is this needed?

The frontend will display a diff based on the response

What would you like to be added?

Create the Karbour website
Discussion:

• Does it live under Kusionstack.io?

Including but not limited to:

• Product overview
• Product introduction
• User stories
• Documentations
• Quickstart example
• etc

Why is this needed?

This provides the channels for end users to understand the product and find samples.

What would you like to be added?

Make it possible for Karbour to run without relying on a kube-apiserver.

Why is this needed?

Currently, karbour relies on a kube-apiserver for delegate authentication and authentication at startup, we hope that karbour to be able to run independently.

What would you like to be added?

The design documentation for the query language Karbour is using when searching for resources.
Karbour need to provides a full Search DSL (Domain Specific Language) based on JSON to define queries.

flowchart TB
  subgraph Karbour
    subgraph SearchAPI
      DSL[Search DSL] --> Convert[DSL Converter: DSL to Query]
    end 
    subgraph StorageLayer
      Query[Query Converter:  Query to ES/ZinC/... DSL]
    end
    Dashboard --> SearchAPI[Search API]
    SearchAPI[Search API] --> StorageLayer[Storage layer]
  end
  StorageLayer[Storage layer] --> C[Search Engine 1]
  StorageLayer[Storage layer] --> D[Search Engine 2]
  StorageLayer[Storage layer] --> E[Search Engine 3]
  style DSL stroke-width:4px

Why is this needed?

So that Karbour can provide advanced search capabilities.

Support sanitize specified KubeConfig by ClusterManager.

Screenshots:

What would you like to be added?

Initialize the code, github action configurations, makefile, configuration files, etc of repository.

TODO:

• Repository Settings
  • Issue Labels
  • Security Settings
  • Code Review Settings
    • Allow squash merging
    • Allow auto-merge
    • Automatically delete head branches
• Configurations
  • CodeOnwers, SECURITY.md, CONTRIBUTING.md, CODE_OF_CONDUCT.md files
  • PR、Issue templates
  • EditorConfig
  • GolangciLint Configuration
  • GoReleaser Configuration
  • Licenserc Configuration
• CICD
  • CI
    • UnitTest
    • GolangLint
    • CommitLint
    • ClaCheck
    • LicenseCheck
  • CD
    • Automation build and push image
    • Automation generate release note and push to github release
    • Automation build and push artifacts

Why is this needed?

1. Standardize the R&D process
2. Automation
3. Consistency

What would you like to be added?

The ability to sync resource information to Karbour's backend storage

Why is this needed?

Karbour can perform search and insight capability based on the data in the backend storage

No response

Task list

• #55
• #131
• #57
• #58

What happened?

The cluster's status field can only be modified by accessing the status subresource of the cluster.

What did you expect to happen?

No response

How can we reproduce it (as minimally and precisely as possible)?

No response

Anything else we need to know?

No response

Karbour version

$ karbour -V
# paste output here

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Install tools

What would you like to be added?

Add the storage layer interface and provide the implementation of elasticsearch type.

Topological relationship:

flowchart TB
  subgraph "Karbour"
    A[Search API] --> B[Storage Layer]
  end
  B[Storage layer] --> C[Strorage Backend 1]
  B[Storage layer] --> D[Strorage Backend 2]
  B[Storage layer] --> E[Strorage Backend 3]
  style B stroke-width:4px

Storage Layer Interface:

type Storage interface {
	Get(ctx context.Context, cluster string, obj runtime.Object) error
	Create(ctx context.Context, cluster string, obj runtime.Object) error
	Update(ctx context.Context, cluster string, obj runtime.Object) error
	Delete(ctx context.Context, cluster string, obj runtime.Object) error
}

type Searcher interface {
	Search(ctx context.Context, queries []Query) (*SearchResult, error)
}

Why is this needed?

The Karbour server supports various types of storage, including ES, and provides search api users. To enable unified access to the storage layer, an interface for the storage layer is provided to shield the differences between different types of storage.

What would you like to be added?

Add the essential information to the resource and cluster details page, including but not limited to:
For resource:

• Name
• Namespace
• Cluster
• Health Status (when applicable)
• CPU and memory usage (when applicable)
• YAML
• etc

For cluster:

• Cluster health status
• Node status
• Node count
• Node CPU and memory usage
• etc

Why is this needed?

This implements one of the core insight functionalities that Karbour presents.
For each resource in the cluster, Karbour displays the essential information for user to read from.

What would you like to be added?

A cluster registration page in the Karbour portal

Why is this needed?

Users should be able to register/update/delete a cluster via Karbour's front end.

What would you like to be added?

The implementation for the transforming component that transforms the data (based on user customization) before store them into Karbour's storage layer.

Why is this needed?

So that user can perform certain data transformation activities during resource sync to provide advanced search and insight capability.

What would you like to be added?

Provide an API that returns the YAML of a given resource

Why is this needed?

The yaml should be available to the user

What would you like to be added?

Currently Karbour registers/updates/deletes new cluster by installing a CR that looks like this in the Karbour apiserver:

apiVersion: cluster.karbour.com/v1beta1
kind: Cluster
metadata:
  name: [local-cluster-nickname]
spec:
  provider: [local-cluster-provider] #optional
  access:
    endpoint: [local-cluster-endpoint]
    caBundle: [local-cluster-CABundle]
    credential:
      type: ServiceAccountToken #SAToken or x509
      serviceAccountToken: [local-cluster-credential]

This is a very manual process. The ability to register/update/delete clusters should be exposed to the end user with a recommended workflow.

Why is this needed?

User should be able to register new cluster via a consistent workflow without having to install Custom Resources manually.

What would you like to be added?

add resources related to sync, two kinds of resource are added: TransformRule and SyncClustersResources

Why is this needed?

No response

What would you like to be added?

We need to develop the Karbour Dashboard component to provide a user-friendly interface. The first step is to develop the search homepage and cluster management page.

TODO:

• search page
  • homepage
  • search result page
• cluster management page
  • cluster list page

Why is this needed?

Provide a user-friendly interface.

What would you like to be added?

Generate cluster CR from name, description and kubeconfig

Why is this needed?

so that cluster can be registered properly

What would you like to be added?

Properly designed search homepage and result page.

Why is this needed?

So that user can easily find what they are looking for when performing a search using Karbour.

What would you like to be added?

The design documentation for the query language Karbour is using when searching for resources.

Why is this needed?

So that Karbour can provide advanced search capabilities.

What would you like to be added?

Karbour concepts and overview

Why is this needed?

Part of Karbour documetations

What would you like to be added?

Detailed documentation on how to perform basic and advanced search using Karbour, including examples.

Why is this needed?

Users can self-service based on the documentation when using Karbour to search for resources.

What would you like to be added?

Integrate with a lightweight web server framework
Choice of framework: go-chi/chi

Why is this needed?

To serve HTTP requests that are not k8s-like

What would you like to be added?

Draft for what Karbour product page looks like.

Why is this needed?

This is a pre-requisite before the Karbour frontend pages can be developed.

What would you like to be added?

A page that lets user configure cluster syncer and transformer behavior.
User should be able to customize the kind of resource to sync, the specific fields to include/exclude, the kind of transformation to perform on raw data, etc.

Why is this needed?

To enrich the data being collected into Karpor's storage so it can provide additional, advanced search and insight capabilities.
Also to give user some flexibility in terms of configuring Karpor's behavior.

What would you like to be added?

Support the all-in-one of kabrour(server) and dashboard(ui) usage.

Old usage:

$ docker run -p 8080:8080 -d kusionstack/karbour:v0.1.2
$ docker run -p 9871:9871 -d kusionstack/karbour-dashboard:v0.1.2

New usage:

$ docker run -p 8080:8080 -d kusionstack/karbour:v0.1.2

or:

Why is this needed?

Simpify user usage.

What would you like to be added?

Provide an out-of-the-box configuration for the centralized ResourceSyncer

Why is this needed?

So that ResourceSyncer can collect data from registered clusters

What would you like to be added?

The ability to search for resources based on specific keywords.
Should also consider the case where the key is not provided, for example instead of providing "Name=hello-world-abcde,Kind=pod", simply providing "hello-world-abcde" should also return similar results.

Why is this needed?

So that Karbour can perform basic search capability.

What would you like to be added?

The main feature of the current version is the addition of a Karbour Dashboard component, which includes code, CICD, build targets, etc. The Karbour Dashboard provides a user-friendly frontend interface that makes it easy for users to search for and gain insights into Kubernetes resources.

Usage

$ docker pull kusionstack/karbour-dashboard:v0.1.2
$ docker run -p 9871:9871 -d kusionstack/karbour-dashboard:v0.1.2

You can visit to http://127.0.0.1:9871, enjoy!

Why is this needed?

No response

What would you like to be added?

All local clusters managed by Karbour should have their health status checked on a regular basis.
Any failure to reach the cluster should be reported to the user.
This includes but not limited to:

• Permission issues due to invalid/expired credential
• Network connectivity issues when Karbour tries to reach the APIServer endpoint
• Local cluster issues that cause /healthz or /readyz endpoint to return an error
• etc

Why is this needed?

Users should be aware when one or some of the clusters managed by Karbour has service interruptions