kubevirt / cloud-provider-kubevirt Goto Github PK

Kubernetes cloud-provider for KubeVirt

License: Apache License 2.0

Makefile 1.35% Dockerfile 0.47% Go 84.72% Shell 13.46%

cloud-provider-kubevirt's Introduction

KubeVirt

KubeVirt is a virtual machine management add-on for Kubernetes. The aim is to provide a common ground for virtualization solutions on top of Kubernetes.

Introduction

Virtualization extension for Kubernetes

At its core, KubeVirt extends Kubernetes by adding additional virtualization resource types (especially the VM type) through Kubernetes's Custom Resource Definitions API. By using this mechanism, the Kubernetes API can be used to manage these VM resources alongside all other resources Kubernetes provides.

The resources themselves are not enough to launch virtual machines. For this to happen the functionality and business logic needs to be added to the cluster. The functionality is not added to Kubernetes itself, but rather added to a Kubernetes cluster by running additional controllers and agents on an existing cluster.

The necessary controllers and agents are provided by KubeVirt.

As of today KubeVirt can be used to declaratively

Create a predefined VM
Schedule a VM on a Kubernetes cluster
Launch a VM
Stop a VM
Delete a VM

To start using KubeVirt

Try our quickstart at kubevirt.io.

See our user documentation at kubevirt.io/docs.

Once you have the basics, you can learn more about how to run KubeVirt and its newest features by taking a look at:

To start developing KubeVirt

To set up a development environment please read our Getting Started Guide. To learn how to contribute, please read our contribution guide.

You can learn more about how KubeVirt is designed (and why it is that way), and learn more about the major components by taking a look at our developer documentation:

Architecture - High-level view on the architecture
Components - Detailed look at all components
API Reference

Useful links

The KubeVirt SIG-release repo is responsible for information regarding upcoming and previous releases.

KubeVirt to Kubernetes version support matrix - Verify the versions of KubeVirt that are built and supported for your version of Kubernetes
Noteworthy changes for the next KubeVirt release - Pre-release notes for the upcoming release
Release schedule - For our current and previous releases

Community

If you got enough of code and want to speak to people, then you got a couple of options:

Follow us on Twitter
Chat with us on Slack via #virtualization @ kubernetes.slack.com
Discuss with us on the kubevirt-dev Google Group
Stay informed about designs and upcoming events by watching our community content

Related resources

Submitting patches

When sending patches to the project, the submitter is required to certify that they have the legal right to submit the code. This is achieved by adding a line

Signed-off-by: Real Name <[email protected]>

to the bottom of every commit message. Existence of such a line certifies that the submitter has complied with the Developer's Certificate of Origin 1.1, (as defined in the file docs/developer-certificate-of-origin).

This line can be automatically added to a commit in the correct format, by using the '-s' option to 'git commit'.

License

KubeVirt is distributed under the Apache License, Version 2.0.

Copyright 2016

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

FOSSA Status

cloud-provider-kubevirt's People

Contributors

Stargazers

Watchers

cloud-provider-kubevirt's Issues

Create E2E tests

In order to gain more code maturity, need to add integration tests

Those tests would be done using cluster-api clusters

For more info about cluster-api, see:

cluster-api github repository
cluster-api-provider-kubevirt repository
The cluster-api can be initiated using clusterctl cli (quick start)
kubevirt platform command: clusterctl init --infrastructure kubevirt

Add external IP update mechanism for LBs

The kubvirt-cloud-provider currently tries to set the external IP of a LoadBalancer service only at creation time (see https://github.com/kubevirt/cloud-provider-kubevirt/blob/master/pkg/cloudprovider/kubevirt/loadbalancer.go#L94,L108). In case that fails or the external IP changes, the cloud-provider will not reconcile the service.
A mechanism in the EnsureLoadbalancer method (https://github.com/kubevirt/cloud-provider-kubevirt/blob/master/pkg/cloudprovider/kubevirt/loadbalancer.go#L81,L87) is needed to update the external IP.

Add e2e tests

Add integration covering al the use cases, one solution would be to copy mechanism from capk.

avoid requiring RBAC for node access by using zone/region info on vmi

once kubevirt implements kubevirt/kubevirt#8063 we should have an alternative way of retrieving the zone/region information that doesn't require access to the VMI's node. We'll be able to read the zone/region off of the VMI itself.

Will there be a new cloud-provider-kubevirt release this year

Hi Team,

The last release for this repository is in the month of Oct 2022. Will there be a new release of cloud-provider-kubevirt in the near future ?

Is there a document that refers to the timelines for the release of this repository.

Thanks
Sharath

Fetch all reported ip addresses by vms and detect their types

Currently we pass to node the interface name "default". We have to pass all reported ip addresses with appropriate types.
https://github.com/kubevirt/cloud-provider-kubevirt/blob/main/pkg/provider/instances_v2.go#L138

Define k8s compatibility policy

We should add information about supported k8s versions or policy to the main README document

Error adding new node to cluster: Node is invalid with 'spec.providerID' forbidden error

We encountered the following error when attempting to add a new node to a tenant cluster:

Node "xxx" is invalid: spec.providerID: Forbidden: node updates may not change providerID except from "" to valid, requeuing

Initially, we used another cloud provider to add nodes to the cluster. However, when we attempted to re-add them, we received the above error message. It appears that when updating the node manifest, the cloud-provider-kubevirt attempts to update the providerID field value with providerID: kubevirt://nodeName

Is it possible to disable updates to this field if it already exists?"

Lookup for VMI by hostname

The current implementation doesn't work as expected, it fails with:

that match label selector "", field selector "spec.Hostname=testmachine": field label not supported: spec.Hostname

Add a possibility to read namespace from CCM config

A ticket to track an upstream work in our Zenhub:
#ref: #78

Use Ginkgo for all unit-tests

Support Local ExternalTrafficPolicy

As mentioned in this ticket: #15, EnsureLoadBalancer can only support an ExternalTrafficPolicy=Cluster.
It would also require:

Do not label the proxy pod/VMI where actual pod does not exist
React on endpoint updates (if the actual pod is rescheduled on a different node)

Is there a plan to fix this issue to have ExternalTrafficPolicy=Local work ?
We for example have this ticket opened kubermatic/kubermatic#9022 by our customer that would like to use the Local externalTrafficPolicy.

Node labels remain the same on a tenant cluster after KubeVirt VM is migrated to another zone

Like described in README for our set up we use infrastructure cluster that hosts KubeVirt VMs (previously known as UnderKube) and tenant cluster that uses those VMs as kubernetes nodes (OverKube).

Nodes that host UnderKube VMs are located in two different availability zones and labeled accordingly.

Recently we were testing VM migration scenarios and found out that labels (such as zone) on OverKube nodes always remain the same despite VMs were migrated between racks, meaning different zones.

What we expect: labels are being changed dynamically when KubeVirt VMs on an UnderKube cluster are migrated between availability zones and correctly propagated to kubernetes nodes on a tenant cluster.

I can provide more info if needed about migration steps etc

Retrieving the zone/region information should be optional

In cases the cloud-controller-manager can't access the nodes, the user would need to have option to disable this functionality

Add a possibility to read namespace from CCM config

By default a namespace is passed to the cloud provider from a kubeconfig. We should allow to read namespace from the CCM configuration as well.

Separate infrastructure kubeconfig from the CCM configuration

The KubeVirt CCM deployment should be able to mount a KubeVirt kubeconfig as a secret and a CCM configuration as a config map.

Add comprehensive documentation

We should add a better documentation for the cloud provider with architecture diagrams.

Use distroless base image rather than alpine

https://github.com/kubernetes/cloud-provider-gcp/blob/c2a51fc35cbf856dc4bab2a0d83ae46993ebe7f2/WORKSPACE#L78
https://github.com/kubernetes/cloud-provider-aws/blob/master/Dockerfile#L25
https://github.com/kubernetes/kubernetes/blob/7c46f40bdf89a437ecdbc01df45e235b5f6d9745/CHANGELOG/CHANGELOG-1.22.md#no-really-you-must-read-this-before-you-upgrade

Many of Kubernetes community folks decided to use Distroless as their base image. Alpine is a great distro but what about using Distroless? :)

Revendor to k/k 1.17.x or newer

In order to support adding the new topology labels to overkube nodes, as opposed to the deprecated failure-domain labels (see also #11), the CCM should be revendored to a version of kubernetes newer than 1.16. Unfortunately, this is hard to do since kubevirt.io/client-go is still based on 1.16 and there are conflicts, see also #10.

There are two ways this issue could be resolved:

Wait until kubevirt.io/client-go is revendored to a newer kubernetes (or actively contribute to this).
Remove dependency to kubevirt.io/client-go/kubecli and use a different client for reading / writing Kubevirt resources. A good alternative could be sigs.k8s.io/controller-runtime/pkg/client. In this case, only the dependency to kubevirt.io/client-go/api/v1 would remain, which should not be an issue.

The second approach requires more changes to the CCM itself, but can be done without depending on any changes to kubevirt.io/client-go. As an additional bonus, this would remove the dependency to glog described in #10 and would enable vendoring of kubevirt.io/client-go newer than 0.26.5.

@afritzler @gonzolino What do you think?

Need workaround for MetalLB service backend mismatch

#16 copies annotations from proxied service to proxy. It was primarily created for load balancers that use service annotations to provide metadata for endpoint wiring, for instance an external IP address.

This works in general, but the allow-shared-ip of MetalLB is a special case. There are likely to be many of this nature. The value provided in the annotation is a lookup key for other services that an IP address should be allowed to be shared with. If two services have the same IP address request but do not contain the same sharing key, the services will not be allowed to share the address.

A nuance of this check is the services must also be managed by the same load balancer. This only makes sense.

When deployed in Gardener, the load balancer of a shoot is not the same as the external load balancer of the cluster. So blindly copying allow-shared-ip will fail because there are two load balancers using the key.

There are several ways to make the key unique, what I am concerned with is how to do this without adding special cases for unrelated projects to the code.

Any thoughts, @gonzolino or @stoyanr?