Git Product home page Git Product logo

cloud-provider-proxmox's Introduction

Kubernetes Proxmox Cloud Provider

Proxmox Cloud Controller Manager

This repository contains the Kubernetes cloud-controller-manager for Proxmox VE. Proxmox Cloud Controller Manager is compatible with cluster-api-provider-proxmox so that machine controller of CAPI can match the nodes with their machine object. (See machine-controller for how it works)

Feature

Node lifecycle controller

The nodes initialized by proxmox-ccm look like below

apiVersion: v1
kind: Node
metadata:
  labels:
    node.kubernetes.io/instance-type: proxmox-qemu.cpu-2.mem-4.0G
  name: worker-1
spec:
  ...
  providerID: proxmox://smbios-system-uuid-ab012345678

Configuration

cloud config looks like below. See sample manifest

proxmox:
    url: https://X.X.X.X:8006/api2/json
    tokenID: 'root@pam!api-token-id'
    secret: "aaaaaaaa-bbbb-cccc-dddd-ee12345678"
    # user: user and
    # password: password is also available

Developing

Integration Testing

export PROXMOX_URL='http://localhost:8006/api2/json'

# tokenid & secret
export PROXMOX_TOKENID='root@pam!your-token-id'
export PROXMOX_SECRET='aaaaaaaaa-bbb-cccc-dddd-ef0123456789'

# or username & password
# export PROXMOX_USERNAME='root@pam'
# export PROXMOX_PASSWORD='password'

export PROXMOX_TEST_UUID='something-actual-uuid-here'

go test ./... -v -run ^TestSuiteIntegration

License

Copyright 2023 Teppei Sudo.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

cloud-provider-proxmox's People

Contributors

sp-yduck avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

cloud-provider-proxmox's Issues

No error message in log when ccm does not set the Provider ID?

Hi @sp-yduck,

first really nice project(s) i follow every commit via e-mail in your proxmox project(s)๐Ÿค“.
At the moment i just use your software i also try to dip into golang and especially into your code but atm i do not have the time/motivation to learn so much new stuff.

Today i tried to get up and running your newest version but i have to problem that the ccm do not set the providerID in the node.spec.providerID. Even with CNI and CCM installed. I tried different CNIs (i don't get cilium to work "Load overlay network failed" error="program cil_from_overlay: replacing clsact qdisc for interface cilium_vxlan: operation not supported") but calico (good old friend) worked.

Node is healthy, all pods are up (coredns not because of the node.cloudprovider.kubernetes.io/uninitialized taint after i remove the taint manually coredns also came up) and the log of the ccm also looks okay but i the providerID under node.spec. is although missing.

Below are some snippets of my setup.
I would assume a log message in the ccm log if the ccm is not able to set the providerID (or remove the taint).

Is this maybe a hint?

$ k --kubeconfig=kubeconfig.yaml logs -n kube-system kube-controller-manager-cappx-test-controlplane-gj5b9 | grep err
I0802 19:09:40.753236       1 resource_quota_monitor.go:223] "QuotaMonitor created object count evaluator" resource="controllerrevisions.apps"
I0802 19:09:50.872722       1 controllermanager.go:638] "Started controller" controller="clusterrole-aggregation"
I0802 19:09:50.872868       1 clusterroleaggregation_controller.go:189] "Starting ClusterRoleAggregator controller"
I0802 19:09:50.888903       1 actual_state_of_world.go:547] "Failed to update statusUpdateNeeded field in actual state of world" err="Failed to set statusUpdateNeeded to needed true, because nodeName=\"cappx-test-controlplane-gj5b9\" does not exist"

Some snippets of the status and logs of the cluster

$ k --kubeconfig=kubeconfig.yaml get pods -A
NAMESPACE     NAME                                                    READY   STATUS    RESTARTS       AGE
kube-system   calico-kube-controllers-68df4c59b7-przqw                1/1     Running   0              2m56s
kube-system   calico-node-l25bf                                       1/1     Running   0              5m59s
kube-system   coredns-5d78c9869d-lqhp6                                0/1     Pending   0              10m
kube-system   coredns-5d78c9869d-pw2q8                                0/1     Pending   0              10m
kube-system   etcd-cappx-test-controlplane-gj5b9                      1/1     Running   1 (11m ago)    11m
kube-system   kube-apiserver-cappx-test-controlplane-gj5b9            1/1     Running   1 (11m ago)    11m
kube-system   kube-controller-manager-cappx-test-controlplane-gj5b9   1/1     Running   2 (5m7s ago)   11m
kube-system   kube-proxy-jp476                                        1/1     Running   0              10m
kube-system   kube-scheduler-cappx-test-controlplane-gj5b9            1/1     Running   2 (5m7s ago)   11m
kube-system   kube-vip-cappx-test-controlplane-gj5b9                  1/1     Running   2 (5m8s ago)   11m

$ k --kubeconfig=kubeconfig.yaml get node cappx-test-controlplane-gj5b9 -o=jsonpath={.spec}
{"podCIDR":"10.244.0.0/24","podCIDRs":["10.244.0.0/24"],"taints":[{"effect":"NoSchedule","key":"node-role.kubernetes.io/control-plane"},{"effect":"NoSchedule","key":"node.cloudprovider.kubernetes.io/uninitialized","value":"true"}]}

$ k --kubeconfig=kubeconfig.yaml get node
NAME                            STATUS   ROLES           AGE   VERSION
cappx-test-controlplane-gj5b9   Ready    control-plane   18m   v1.27.3

$ k --kubeconfig=kubeconfig.yaml get events
LAST SEEN   TYPE      REASON                    OBJECT                               MESSAGE
20m         Normal    Starting                  node/cappx-test-controlplane-gj5b9   Starting kubelet.
20m         Warning   InvalidDiskCapacity       node/cappx-test-controlplane-gj5b9   invalid capacity 0 on image filesystem
20m         Normal    NodeAllocatableEnforced   node/cappx-test-controlplane-gj5b9   Updated Node Allocatable limit across pods
20m         Normal    NodeHasSufficientMemory   node/cappx-test-controlplane-gj5b9   Node cappx-test-controlplane-gj5b9 status is now: NodeHasSufficientMemory
20m         Normal    NodeHasNoDiskPressure     node/cappx-test-controlplane-gj5b9   Node cappx-test-controlplane-gj5b9 status is now: NodeHasNoDiskPressure
20m         Normal    NodeHasSufficientPID      node/cappx-test-controlplane-gj5b9   Node cappx-test-controlplane-gj5b9 status is now: NodeHasSufficientPID
19m         Normal    Starting                  node/cappx-test-controlplane-gj5b9   Starting kubelet.
19m         Warning   InvalidDiskCapacity       node/cappx-test-controlplane-gj5b9   invalid capacity 0 on image filesystem
19m         Normal    NodeHasSufficientMemory   node/cappx-test-controlplane-gj5b9   Node cappx-test-controlplane-gj5b9 status is now: NodeHasSufficientMemory
19m         Normal    NodeHasNoDiskPressure     node/cappx-test-controlplane-gj5b9   Node cappx-test-controlplane-gj5b9 status is now: NodeHasNoDiskPressure
19m         Normal    NodeHasSufficientPID      node/cappx-test-controlplane-gj5b9   Node cappx-test-controlplane-gj5b9 status is now: NodeHasSufficientPID
19m         Normal    NodeAllocatableEnforced   node/cappx-test-controlplane-gj5b9   Updated Node Allocatable limit across pods
19m         Normal    RegisteredNode            node/cappx-test-controlplane-gj5b9   Node cappx-test-controlplane-gj5b9 event: Registered Node cappx-test-controlplane-gj5b9 in Controller
19m         Normal    Starting                  node/cappx-test-controlplane-gj5b9
13m         Normal    NodeReady                 node/cappx-test-controlplane-gj5b9   Node cappx-test-controlplane-gj5b9 status is now: NodeReady
12m         Normal    RegisteredNode            node/cappx-test-controlplane-gj5b9   Node cappx-test-controlplane-gj5b9 event: Registered Node cappx-test-controlplane-gj5b9 in Controller

I would appreciate your reply very much!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.