This operator can be used to deploy additional networking components on top of Kubernetes cluster.
Configuration of desired network addons is done using NetworkAddonsConfig object:
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
multus: {}
multusDynamicNetworks: {}
linuxBridge: {}
kubeMacPool: {}
ovs: {}
macvtap: {}
kubeSecondaryDNS: {}
kubevirtIpamController: {}
imagePullPolicy: AlwaysThe operator allows administrator to deploy multi-network
Multus plugin. It is done using multus
attribute.
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
multus: {}Additionally, container image used to deliver this plugin can be set using
MULTUS_IMAGE environment variable in operator deployment manifest.
This controller
allows hot-plug and hot-unplug of additional Pod intefaces. It is done using
multusDynamicNetworks attribute.
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
multus: {}
multusDynamicNetworks: {}Additionally, container image used to deliver this plugin can be set using
MULTUS_DYNAMIC_NETWORKS_CONTROLLER_IMAGE environment variable in operator
deployment manifest.
The operator allows administrator to deploy Linux Bridge CNI plugin
simply by adding linuxBridge attribute to NetworkAddonsConfig.
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
linuxBridge: {}Additionally, container image used to deliver this plugin can be set using
LINUX_BRIDGE_IMAGE environment variable in operator deployment manifest.
The bridge marker image used to deliver a bridge marker detecting the availability
of linux bridges on nodes can be set using the LINUX_BRIDGE_MARKER_IMAGE environment
variable in operator deployment manifest.
Following snippets can be used to configure linux bridge on your node.
# create the bridge using NetworkManager
nmcli con add type bridge ifname br10
# allow traffic to go through the bridge between pods
iptables -I FORWARD 1 -i br10 -j ACCEPTThe operator allows administrator to deploy the Kubemacpool. This project allow to allocate mac addresses from a pool to secondary interfaces using Network Plumbing Working Group de-facto standard.
Note: Administrator can specify a requested range, if the range is not
requested a random range will be provided. This random range spans from
02:XX:XX:00:00:00 to 02:XX:XX:FF:FF:FF, where 02 makes the address local
unicast and XX:XX is a random prefix.
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
kubeMacPool:
rangeStart: "02:00:00:00:00:00"
rangeEnd: "FD:FF:FF:FF:FF:FF"The operator allows administrator to deploy OVS CNI plugin
simply by adding ovs attribute to NetworkAddonsConfig. Please note that
in order to use this plugin, openvswitch have to be up and running at nodes.
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
ovs: {}Note: The cluster-network-addons-operator is no longer installing kubernetes-nmstate, refer to its own operator release notes to install it.
Note: This feature is experimental. Macvtap-cni is unstable and its API may change.
The operator allows the administrator to deploy the
macvtap CNI plugin, simply by
adding macvtap attribute to NetworkAddonsConfig.
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
macvtap: {}Macvtap-cni must be explicitly configured by the administrator, indicating the interfaces on top of which logical networks can be created.
A simple example on how to do so, the user must deploy a ConfigMap, such as in this example.
Currently, this configuration is not dynamic.
This controller allows to support FQDN for VMI's secondary networks.
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
kubeSecondaryDNS:
DOMAIN: ""
NAME_SERVER_IP: ""Additionally, container image used to deliver this plugin can be set using
KUBE_SECONDARY_DNS_IMAGE environment variable in operator
deployment manifest.
This controller allows to support IPAM for user defined networks.
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
multus: {}
kubevirtIpamController: {}Additionally, container image used to deliver this plugin can be set using
KUBEVIRT_IPAM_CONTROLLER_IMAGE environment variable in operator
deployment manifest.
Note: This component requires certificates mounted on the controller pods for the webhook to work. On non OpenShift clusters, the user should manually install a certificate library (e.g. cert-manager). It is done for convenience as part of the helper scripts.
Administrator can specify image pull policy
for deployed components. Default is IfNotPresent.
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
imagePullPolicy: AlwaysAdministrator can specify webhook self signed certificates configuration
for deployed components. Default is caRotateInterval: 168h, caOverlapInterval: 24h, certRotateInterval: 24h, certOverlapInterval: 8h
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
selfSignConfiguration:
caRotateInterval: 168h
caOverlapInterval: 24h
certRotateInterval: 24h
certOverlapInterval: 8hThe selfSignConfiguration parameters has to be all or none set, setting some of them fails at validation. They have to conform to golang time.Duration string format. Additionally the following checks are done at validation:
- caRotateInterval >= caOverlapInterval && caRotateInterval >= certRotateInterval && certRotateInterval >= certOverlapInterval
This parameters are consumed by Kubemacpool component.
CNAO deploys two component categories: infra and workload. Workload components manage node configuration and need to be scheduled on the nodes where actual user workload is scheduled. Infra components provide cluster-wide service and do not need to be running on the same nodes as user workload.
Administrator can specify placement preferences for deployed infra and workload components by defining affinity, nodeSelector and tolerations.
By default, infra components are scheduled on control-plane nodes and workload components are scheduled on all nodes.
To adjust this behaviour, provide custom placementConfiguration to the NetworkAddonsConfig.
In the following example, nodeAffinity is used to schedule infra components to control-plane nodes and nodeSelector
to schedule workloads on worker nodes.
Note that worker nodes need to be labeled with node-role.kubernetes.io/worker label.
apiVersion: networkaddonsoperator.network.kubevirt.io/v1
kind: NetworkAddonsConfig
metadata:
name: cluster
spec:
placementConfiguration:
infra:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
workloads:
nodeSelector:
node-role.kubernetes.io/worker: ""First install the operator itself:
kubectl apply -f https://github.com/kubevirt/cluster-network-addons-operator/releases/download/v0.95.0/namespace.yaml
kubectl apply -f https://github.com/kubevirt/cluster-network-addons-operator/releases/download/v0.95.0/network-addons-config.crd.yaml
kubectl apply -f https://github.com/kubevirt/cluster-network-addons-operator/releases/download/v0.95.0/operator.yamlThen you need to create a configuration for the operator example CR:
kubectl apply -f https://github.com/kubevirt/cluster-network-addons-operator/releases/download/v0.95.0/network-addons-config-example.cr.yamlFinally you can wait for the operator to finish deployment:
kubectl wait networkaddonsconfig cluster --for condition=AvailableIn case something failed, you can find the error in the NetworkAddonsConfig Status field:
kubectl get networkaddonsconfig cluster -o yamlYou can follow the deployment state through events produced in the default namespace:
kubectl get eventsEvents will be produced whenever the deployment is applied, configured or failed. The expected events are:
| Event type | Reason |
|---|---|
| Progressing | When operator had started deploying the components |
| Failed | When one or more components failed to deploy |
| Available | When all components finished to deploy |
| Modified | When the configuration was modified or applied for the first time |
For more information about the configuration format check configuring section.
Starting with version 0.89.0, this operator supports upgrades to any newer
version. If you wish to upgrade, remove old operator (operator.yaml) and
install new, operands will remain available during the operator's downtime.
Make sure you have either Docker >= 17.05 / podman >= 3.1 installed.
# run code validation and unit tests
make check
# perform auto-formatting on the source code (if not done by your IDE)
make fmt
# generate source code for API
make gen-k8s
# build images (uses multi-stage builds and therefore requires Docker >= 17.05 / podman >= 3.1)
make docker-build
# or build only a specific image
make docker-build-operator
make docker-build-registry
# bring up a local cluster with Kubernetes
make cluster-up
# bridge up a local cluster with kubernetes 1.25
export KUBEVIRT_PROVIDER=k8s-1.25
make cluster-up
# build images and push them to the local cluster
make cluster-operator-push
# install operator on the local cluster
make cluster-operator-install
# run workflow e2e tests on the cluster, requires cluster with installed operator,
# workflow covers deployment of operands
make test/e2e/workflow
# run lifecycle e2e tests on the cluster, requires cluster without operator installed,
# lifecycle covers deployment of operator itself and its upgrades
make test/e2e/lifecycle
# access kubernetes API on the cluster
./cluster/kubectl.sh get nodes
# ssh into the cluster's node
./cluster/cli.sh ssh node01
# clean up all resources created by the operator from the cluster
make cluster-clean
# delete the cluster
make cluster-downFor developing at an external cluster:
export KUBEVIRT_PROVIDER=external
export KUBECONFIG=[path to external's cluster kubeconfig]
# This is the registry used to push and pull the dev image
# it has to be accessible by the external cluster
export DEV_IMAGE_REGISTRY=quay.io/$USER
# Then is possible to follow normal dev flow
make cluster-operator-push
make cluster-operator-install
make test/e2e/workflow
make test/e2e/lifecycle- Checkout a public branch
- Call
make prepare-patch|minor|majorand prepare release notes - Open a new PR
- Once the PR is merged, create a new release in GitHub and attach new manifests
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent