Consider how to handle shared instance with VC about cluster-api-provider-nested HOT 8 CLOSED

@gyliu513 yea, check out the readme in https://github.com/kubernetes-sigs/cluster-api-provider-nested/tree/main/virtualcluster/experiment it's a way of doing multi-supercluster virtualcluster deployments with regards to the workload scheduling, not related to the control plane provisioning aspects.

from cluster-api-provider-nested.

@gyliu513 do you feel like you've gotten the right information now and we can close this issue?

from cluster-api-provider-nested.

Yeah, to add to that…

Some other example of areas where I could see more of this is a centralized logging or metrics system so everyone didn't need to maintain their own.

All that being said there is a downside to this to consider, VC does a lot to help make the super cluster really lightweight and easily replaceable, the more you break that model the more VC get's harder and less flexible. For example if you look at the experimental namespace two tier scheduler this is continuing to focus on making the tenant cluster the owner of the data and the implementation which allows us to have N light weight pod scheduling domains if one goes down the tenant cluster can still exist and can easily transfer workloads to another cluster in the fleet.

from cluster-api-provider-nested.

https://github.com/kubernetes-sigs/cluster-api-provider-nested/blob/main/virtualcluster/doc/customresource-syncer.md for CRDs that we expose only the implementation at the super cluster but want tenant clusters to be able to CRUD them.

@christopherhein , thanks, this is the feature that I want. If we can expose some CRDs only on super cluster, then how about extending this to support other k8s resources :)

from cluster-api-provider-nested.

Yup, we do this as well. Some are just to create random actions for example because we don't have CAPI's ClusterResourceSet we use a "checker" within the confines as how the vc-syncer defines it to automatically make sure there is always X or Y admission webhook deployed into the tenant control plane.

from cluster-api-provider-nested.

The struggle is you have to write a lot of those extensions yourself cause of how vc syncs things you need make sure it supports that flow, we package our syncer w/ vc-syncer as a single code base but extend it to have the other custom components.

from cluster-api-provider-nested.

/kind support

from cluster-api-provider-nested.

For example if you look at the experimental namespace two tier scheduler this is continuing to focus on making the tenant cluster the owner of the data and the implementation which allows us to have N light weight pod scheduling domains if one goes down the tenant cluster can still exist and can easily transfer workloads to another cluster in the fleet.

@christopherhein For VC, I think there is only one scheduler in super cluster, but no in tenant cluster. Can you share more detail for what do you mean by experimental namespace two tier scheduler?

from cluster-api-provider-nested.