YDB Operator allows you to deploy your own YDB cluster in Kubernetes
License: Other
The YDB Kubernetes operator deploys and manages YDB resources in a Kubernetes cluster.
For steps on how to deploy and use YDB Kubernetes Operator, please refer to documentation.
Refer to the operator development docs.
YDB Kubernetes Operator version:
latest
Environment
kubernetes
Current behavior:
click operator + ydb operator
Expected behavior:
click and ydb work fine
Steps to reproduce:
deploy click
deploy ydb
Related code:
2024-03-04T15:05:59Z INFO running step checkStorageFrozen for Storage {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z INFO running step handlePauseResume for Storage {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z INFO running step handleResourcesSync {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z INFO resource sync complete {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z INFO running step syncNodeSetSpecInline {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z INFO syncNodeSetSpecInline complete {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z INFO running step setInitialStatus {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z INFO running step waitForStatefulSetToScale for Storage {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z INFO running step getYDBCredentials {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z INFO running step runSelfCheck {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z INFO running step runInitScripts {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z DEBUG events SelfCheck result: GOOD, issues found: 8 {"type": "Normal", "object": {"kind":"Storage","namespace":"stage","name":"valun-ydb","uid":"be602273-3b00-4e61-b1cc-3f723a488666","apiVersion":"ydb.tech/v1alpha1","resourceVersion":"85119616"}, "reason": "SelfCheck"}
2024-03-04T15:05:59Z ERROR unexpected Sync error {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2", "error": "failed to stream execution results back, stdout:\n\"\"stderr:\n\"\": container ydb-storage is not valid for pod chi-backend-shard-0-0-0", "errorVerbose": "container ydb-storage is not valid for pod chi-backend-shard-0-0-0\nfailed to stream execution results back, stdout:\n\"\"stderr:\n\"\"\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/exec.InPod\n\t/workspace/internal/exec/exec.go:59\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).initializeStorage\n\t/workspace/internal/controllers/storage/init.go:167\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).handleFirstStart\n\t/workspace/internal/controllers/storage/sync.go:579\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).Sync\n\t/workspace/internal/controllers/storage/sync.go:61\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).Reconcile\n\t/workspace/internal/controllers/storage/controller.go:71\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"}
github.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).Reconcile
/workspace/internal/controllers/storage/controller.go:73
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:122
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:323
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235
2024-03-04T15:05:59Z ERROR Reconciler error {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2", "error": "failed to stream execution results back, stdout:\n\"\"stderr:\n\"\": container ydb-storage is not valid for pod chi-backend-shard-0-0-0", "errorVerbose": "container ydb-storage is not valid for pod chi-backend-shard-0-0-0\nfailed to stream execution results back, stdout:\n\"\"stderr:\n\"\"\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/exec.InPod\n\t/workspace/internal/exec/exec.go:59\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).initializeStorage\n\t/workspace/internal/controllers/storage/init.go:167\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).handleFirstStart\n\t/workspace/internal/controllers/storage/sync.go:579\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).Sync\n\t/workspace/internal/controllers/storage/sync.go:61\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).Reconcile\n\t/workspace/internal/controllers/storage/controller.go:71\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235
Logs from 04.03.2024, 14:12:38
Other information:
Пытаюсь создать таблицу НО
database doesn't have storage pools at all to create tablet channels to storage pool binding by profile id
Describe the Feature Request
Operator could enable day-2 workflows declaratively.
Some of the issues are called out in the docs with [1] but they seem to be fairly limited. I have been using CrunchyData's postgres operator, and looked into CloudNativePG operator as well, and they offer a pretty good blueprint of what could work.
backups:
pgbackrest:
configuration:
- secret:
name: pgo-gcs-creds
image: registry.developers.crunchydata.com/crunchydata/crunchy-pgbackrest:ubi8-2.47-0
manual:
options:
- --type=full
repoName: repo1
repos:
- name: repo1
s3:
bucket: <BUCKET>
endpoint: <ENDPOINT>
region: <REGION>
schedules:
full: 0 7 */2 * *
Storage size increase
Crunchy allows for increasing # of replicas and storage size for each of the instances. Based on YDB docs, it's not possible to update the manifest and apply changes. I have not tested if manually increasing PersistentVolumeClaim size works, but doing this at manifest would be idea.
Affinity
I believe operator supports node affinity / through the CRD, but it's not called in documentation, so one has to check the CRD definition. An example would probably also suffice.
Users/Databases
This is not a strict must, and Database is a separate CRD object, so it may be moot somewhat, but crunchy encapsulates database and users in same CRD.
Similar to (4), Crunchy encapsulates monitoring (exporter) and administraive UI (pgadmin) in a single CRD object, could work better.
Describe Preferred Solution
More capable CRD would help. CLI is fine, but hard with devops.
If the feature request is approved, would you be willing to submit a PR?
No. Not sure i know the tool enough to build the operator on it.
Describe the Feature Request
At this point, operator is unable to connect to the database if enforce_static_user_credentials in Storage config is set to true
Relevant error:
2022-12-23T11:00:29.252Z ERROR controller.storage GetSelfCheckResult error {"reconciler group": "ydb.tech", "reconciler kind": "Storage", "name": "main", "namespace": "ydb", "error": "rpc error: code = Unauthenticated desc = unauthenticated, Access denied without user token"}
github.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).Reconcile
/workspace/internal/controllers/storage/controller.go:63
Additional Context
Internal ticket YDBOPS-6056
YDB Kubernetes Operator version:
Environment
Current behavior:
Blobstorage init job is created with no specified resource limits. While resources are limited by quota init job's pod is not created
Expected behavior:
ydb operator should specify resource limits when creating the job
Steps to reproduce:
Related code:
insert short code snippets here
Other information:
Need to check if prometheus CRDs such as servicemonitors.monitoring.coreos.com exist at actions beginning. Now it is causing ydb-operator errors.
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes
YDB Kubernetes Operator version:
image: 'cr.yandex/yc/ydb-kubernetes-operator:0.4.12'
Environment
Managed k8s in Yandex
Current behavior:
Storage status is Initializing
status:
conditions:
- lastTransitionTime: '2022-09-16T07:29:53Z'
message: Storage initialization in progress
reason: InProgress
status: 'False'
type: StorageInitialized
- lastTransitionTime: '2022-09-16T07:29:53Z'
message: InitStorageStep is required
reason: InProgress
status: 'False'
type: InitStorageStep
state: Initializing
2022-09-16T07:53:27.522Z DEBUG events Warning {"object": {"kind":"Storage","namespace":"ydb","name":"storage-sample","uid":"4bed2162-36ff-4a3f-b63b-b61f5bbdd4e0","apiVersion":"ydb.tech/v1alpha1","resourceVersion":"50214544"}, "reason": "SelfCheck", "message": "SelfCheck result: EMERGENCY, issues found: 34"}
2022-09-16T07:54:57.703Z ERROR controller.storage unexpected Sync error {"reconciler group": "ydb.tech", "reconciler kind": "Storage", "name": "storage-sample", "namespace": "ydb", "error": "failed to stream execution results back, stdout:\n\"\"stderr:\n\"MP-0130 Tablet request timed out Marker# MBT4\n\": command terminated with exit code 1", "errorVerbose": "command terminated with exit code 1\nfailed to stream execution results back, stdout:\n\"\"stderr:\n\"MP-0130 Tablet request timed out Marker# MBT4\n\"\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/exec.ExecInPod\n\t/workspace/internal/exec/exec.go:49\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*StorageReconciler).runInitScripts\n\t/workspace/internal/controllers/storage/init.go:67\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*StorageReconciler).Sync\n\t/workspace/internal/controllers/storage/sync.go:86\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*StorageReconciler).Reconcile\n\t/workspace/internal/controllers/storage/controller.go:64\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1581"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227
Expected behavior:
Storage status is Ready
Steps to reproduce:
Link
Step after install Storage
My spec
Other information:
YDB Kubernetes Operator version:
helm list
CHART
operator-0.4.5
APP VERSION
0.4.9
Environment
GKE autopilot, GKE
cloud shell
OS – Debian GNU/Linux 11 (bullseye)
kubectl version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:58:47Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.6-gke.1503", GitCommit:"2c7bbda09a9b7ca78db230e099cf90fe901d3df8", GitTreeState:"clean", BuildDate:"2022-02-18T03:17:45Z", GoVersion:"go1.16.9b7", Compiler:"gc", Platform:"linux/amd64"}
helm
version.BuildInfo{Version:"v3.5.0", GitCommit:"32c22239423b3b4ba6706d450bd044baffdcf9e6", GitTreeState:"clean", GoVersion:"go1.15.6"}
Current behavior:
I followed managed k8s steps here
https://ydb.tech/en/docs/deploy/orchestrated/yc_managed_kubernetes
helm repo add ydb https://charts.ydb.tech/
helm repo update
helm install ydb-operator ydb/operatorThe last one produced this error
Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
Hello
I am trying use linkerd for ydb, but it need add annotations "linkerd.io/inject: enabled" to pods/deployments.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.