Comments (19)
for anyone that somehow end up here, the CSI driver is moved to https://github.com/digitalocean/csi-digitalocean
from digitalocean-cloud-controller-manager.
@klausenbusk yeap, I've officially started working on it. Follow the repo or the csi
folder for more info. Eventually we're going to split it out into it's own repository later.
from digitalocean-cloud-controller-manager.
@klausenbusk volume support on external cloud controller manager requires some work to be done in both the kube-controller-manager and the kubelets since both are responsible for attaching/detaching/mounting/unmounting volumes. From my understanding we're waiting on CSI (container storage interface) to be ready to integrate into kubelets before adding that feature to external cloud controller managers.
I have brought this up before if you want to follow up on those discussions:
kubernetes/enhancements#88 (comment)
kubernetes/kubernetes#48690 (comment)
from digitalocean-cloud-controller-manager.
To clarify, there's no set ETA, but you can follow up with sig-storage to see where the CSI project is
from digitalocean-cloud-controller-manager.
Thanks for the fast response, I will stick with my "bash" flexvolume script solution for the time being.
Just a side question: Is DigitalOcean working on any anything else related to k8s? I just saw that you are working on DO support in kops.
DO support in https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler could also be nice (mostly if DO lose a region, auto-scaling in the remaining region could be nice), maybe I should take a stab at that myself.
from digitalocean-cloud-controller-manager.
kops support for DO is more of a side project for me.
Re: cluster-autoscaler I would love to see it working with DO, I haven't had a chance to work on it since I figured supporting cloud controller manager is probably more important. Do let me know if you take a stab at it, would love to give feedback where I can :)
from digitalocean-cloud-controller-manager.
Would it make sense to add a flexvolume plugin to this repo?
I have a 167 line bash script currently which work as a flexvolume plugin, and I think that can be stripped down even further. As we can just use the default mountdev
/unmountdevice
call (I think).
Reimplementing it in Go shouldn't be that hard. In the end I just need to implement the init
/attach
/detach
/waitforattach
and isattached
call. It could properly work without the last 2.
I did a bit of research, and with this proposal kubernetes/community#833 , installing the flexvolume plugin will be pretty easy, the needed changes was just merged a few days ago: kubernetes/kubernetes#50031 .
With the flexvolume in place, we/I could then create a external provisioner with https://github.com/kubernetes-incubator/external-storage . I the end it would properly would at least as good as kubernetes/kubernetes#50044 , one thing I can't figure out through, is how to ensure that k8s does not schedule more than 5 volumes to a node.
from digitalocean-cloud-controller-manager.
@klausenbusk thanks for doing some research and digging into this! I didn't know the dynamic discovery of flex volume plugins was in already, that's going to be super helpful.
For flex volumes plugins, there's a project called rexray that integrates DO block storage with Kubernetes flex volumes written in Go. You can read up on how to setup here. One of my biggest issues with rexray's flexvolume plugin is that there's some host configuration required to make it work with Kubernetes, but I think we can do some work to make that easier.
As you mentioned there isn't a good story right now to provision the volumes. So even though we can use rexray (or another plugin) for attach/detach/mount/unmount, we still have to create those volumes manually. I think creating an external provisioner is going in the right direction and something we'll need eventually to better integrate with flex volumes plugins. I'm not sure if external storage provisioner should be a separate project or outside of CCM, that's something I'd have to discuss with sig cluster lifecycle.
For the scheduling concern regarding having more than 5 volumes to a node, I think there are some ways to work around this, but it's something we can work that out afterwards. I think it's more important to have a better story for the install process of flex volume plugins and an external provisioner.
from digitalocean-cloud-controller-manager.
from digitalocean-cloud-controller-manager.
For flex volumes plugins, there's a project called rexray that integrates DO block storage with Kubernetes flex volumes written in Go.
I have used rexray
before (back when my cluster was at 1.5), but rexray
does not currently support the attach/detach
controller, which is one of the reason I wrote my own flexvolume plugin in bash.
I took a stab on implementing the same plugin in Go: https://gist.github.com/klausenbusk/422e2a9ebc2ac0856cc5ffab2fe90334 , all the logic is there, but the code quality could be better.
Note: I haven't tested it.
from digitalocean-cloud-controller-manager.
@klausenbusk I'm not sure I follow you completely. When you say rexray does not support attach/detach
controller you mean it doesn't support controller managed attach/detach right? When using rexray you are enabling attach/detach controller on the individual kubelets in which case rexray's flex volume plugin does have the ability to attach/detach volumes.
from digitalocean-cloud-controller-manager.
When you say rexray does not support attach/detach controller you mean it doesn't support controller managed attach/detach right?
Correct (--enable-controller-attach-detach
).
When using rexray you are enabling attach/detach controller on the individual kubelets in which case rexray's flex volume plugin does have the ability to attach/detach volumes.
You mean --enable-controller-attach-detach=false
? The flexrex
plugin should work without issue if --enable-controller-attach-detach=false
is set.
from digitalocean-cloud-controller-manager.
Gotcha, so what you're requesting originally is to have a flexvolume plugin that is controller managed, which rexray does not currently support.
from digitalocean-cloud-controller-manager.
I'm not sure if external storage provisioner should be a separate project or outside of CCM
The code can go into the external-storage project, per kubernetes-retired/external-storage#318 .
from digitalocean-cloud-controller-manager.
@klausenbusk cool, looking forward to that!
from digitalocean-cloud-controller-manager.
With 1.9 just released with a alpha implementation of CSI, does DigitalOcean plan on providing a CSI plugin?
from digitalocean-cloud-controller-manager.
@klausenbusk over at stackpoint.io we built the official DO flex volume, so likely we'll look at re-creating it as a CSI plugin, but probably won't get going on that until March.
from digitalocean-cloud-controller-manager.
@klausenbusk over at stackpoint.io we built the official DO flex volume, so likely we'll look at re-creating it as a CSI plugin, but probably won't get going on that until March.
Cool :)
BTW: digitalocean-provisoner
was just merged upstream: kubernetes-retired/external-storage#470
from digitalocean-cloud-controller-manager.
Lets close this issue. @fatih is working on a CSI driver (and provisioner?)
from digitalocean-cloud-controller-manager.
Related Issues (20)
- do-loadbalancer-protocol: 'http2' results in 'http2' --> 'http' HOT 4
- Extending Loadbalancer timeout duration HOT 1
- Misconfigured cloud-controller-manager.yml (HA deployment that uses daemonset) HOT 1
- Change release pipeline to promote dev manifests
- Protect load balancer from being deleted HOT 7
- udp loadbalancer failing to create HOT 8
- Controller manual mode HOT 17
- Allow the region to be explicitly specified instead of using the Region metadata API HOT 5
- K8 annotations for load balancer name / id do not work as expected HOT 4
- Typos in README.md
- IPv6 address missing in nodes status HOT 9
- Prevent duplicate do-loadbalancer-name annotation from changing LB ownership
- do-loadbalancer should accept a certificate name as an alternative to the certificate ID
- Wrong validation regex for service.beta.kubernetes.io/do-loadbalancer-allow-rules HOT 2
- Feature Request: Create a Helm chart for DO CCM HOT 1
- Cloud Controller Manager doesn't add droplets to Load Balancer HOT 6
- `k8s.gcr.io` is no longer used HOT 2
- CI: Bypass branch protection on release workflow execution
- do-loadbalancer-allow-rules doesn't work (firewall is not configured) HOT 1
- Confusion with do-loadbalancer-hostname HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from digitalocean-cloud-controller-manager.