virtuslab / kubedrainer Goto Github PK
View Code? Open in Web Editor NEWSimple Kubernetes Node Drainer
License: Apache License 2.0
Simple Kubernetes Node Drainer
License: Apache License 2.0
Is there way to reduce the api call limit to prevent the following? It looks like sleep time is 10s, so every minute it makes 6 api calls.
W0207 23:20:14.956666 1 handler.go:28] Can not get instance status and auto scaling group name, will try again: Throttling: Rate exceeded
status code: 400, request id: e29ab693-33b8-401a-b73a-85567b8559bd
Attempting to spin this up in both a non-ASG managed cluster and one that is resulted in the same error.
│ {"level":"warn","time":1610478473,"message":"No config file found"} │
│ {"level":"info","time":1610478473,"message":"Running as server"} │
│ Error: Can't match expression '^/(?P<Region>[a-zA-Z0-9-]+)[a-z]/(?P<InstanceID>[a-zA-Z0-9-]+)$' to 'us-west-2b/i-0af01948154f55129' │
│ Usage: │
│ kubedrainer serve [flags] │
│ Flags: │
│ --cache-dir string Default HTTP cache directory (default "/.kube/http-cache") │
│ --certificate-authority string Path to a cert file for the certificate authority │
│ --client-certificate string Path to a client certificate file for TLS │
│ --client-key string Path to a client key file for TLS │
│ --cluster string The name of the kubeconfig cluster to use │
│ --context string The name of the kubeconfig context to use │
│ --delete-local-data Continue even if there are pods using emptyDir (local data that will be deleted when the node is drained). (default true) │
│ --force Continue even if there are pods not managed by a ReplicationController, ReplicaSet, Job, DaemonSet or StatefulSet. │
│ --grace-period int Period of time in seconds given to each pod to terminate gracefully. If negative, the default value specified in the pod will be used. (default -1) │
│ -h, --help help for serve │
│ --ignore-daemonsets Ignore DaemonSet-managed pods. (default true) │
│ --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure │
│ --instance-id string AWS EC2 instance ID to terminate │
│ --kubeconfig string Path to the kubeconfig file to use for CLI requests. │
│ --node string Kubernetes node name to drain │
│ --pod-selector string Label selector to filter pods on the node │
│ --profile string AWS Profile to use │
│ --region string AWS Region to use │
│ --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0") │
│ -l, --selector string Selector (label query) to filter on │
│ -s, --server string The address and port of the Kubernetes API server │
│ --timeout duration The length of time to wait before giving up, zero means infinite (default 1m0s) │
│ --token string Bearer token for authentication to the API server │
│ --user string The name of the kubeconfig user to use │
│ Global Flags: │
│ --config string config file (default is $HOME/.kubedrainer.yaml) │
│ -d, --debug sets log level to debug │
│ {"level":"fatal","time":1610478473,"message":"Can't match expression '^/(?P<Region>[a-zA-Z0-9-]+)[a-z]/(?P<InstanceID>[a-zA-Z0-9-]+)$' to 'us-west-2b/i-0af01948154f55129'"}
Looks like this just needs a backslash in the expression with the starting slash removed?
kubedrainer/cmd/kubedrainer/serve.go
Line 204 in 9a7d7d7
^(?P<Region>[a-zA-Z0-9-]+)[a-z]\/(?P<InstanceID>[a-zA-Z0-9-]+)$
Like 'draino' did:
kubernetes/node-problem-detector#199
I'm trying to import the drainer
package into my Go project. But, I got errors. Did I miss something?
go get github.com/VirtusLab/kubedrainer/pkg/drainer
go: downloading github.com/VirtusLab/kubedrainer v0.0.9
go: found github.com/VirtusLab/kubedrainer/pkg/drainer in github.com/VirtusLab/kubedrainer v0.0.9
go: downloading github.com/rs/zerolog v1.19.0
go: downloading github.com/VirtusLab/go-extended v0.0.11
go: downloading github.com/gregjones/httpcache v0.0.0-20190212212710-3befbb6ad0cc
go: downloading google.golang.org/appengine v1.6.1
# github.com/VirtusLab/kubedrainer/pkg/kubernetes/node
../../go/pkg/mod/github.com/!virtus!lab/[email protected]/pkg/kubernetes/node/node.go:25:46: not enough arguments in call to n.Client.CoreV1().Nodes().List
have ("k8s.io/apimachinery/pkg/apis/meta/v1".ListOptions)
want (context.Context, "k8s.io/apimachinery/pkg/apis/meta/v1".ListOptions)
../../go/pkg/mod/github.com/!virtus!lab/[email protected]/pkg/kubernetes/node/node.go:49:44: not enough arguments in call to n.Client.CoreV1().Nodes().Get
have (string, "k8s.io/apimachinery/pkg/apis/meta/v1".GetOptions)
want (context.Context, string, "k8s.io/apimachinery/pkg/apis/meta/v1".GetOptions)
I feel like I'm missing something.
I've deployed the daemonset and IAM roles, but without the ASG lifecycle hook, the daemonset won't be triggered right?
So what should my lifecycle hook/action look like?
Granted I'm new to lifecycle hooks, but I feel like that's the critical part I'm missing.
Thanks in advance! @pawelprazak
I have issue with cloudformation template polices are not applied to the role
https://github.com/VirtusLab/kubedrainer/blob/master/examples/kubernetes.yaml
In our clusters we use fluentbit to send logs to CloudWatch, which then allows us to use CloudWatch Logs Insights to query for patterns, either for one-time debugging or auditing, or in some cases, even creating Grafana dashboards. Having the option to output the logs in json format allows for improved searching on specific fields or patterns.
Taking this on would likely mean using a different logging package, since it doesn't look like glog supports this. I've had good success in the past with logrus, but it looks like even they now recommend some other improved packages, such as: Zerolog, Zap, and Apex.
@pawelprazak It would be nice if kubedrainer supported Pod Disruption Budget. I haven't tested it, but I assume in its current state, kubedrainer will fail to evict pods that have a PDB. I assume there should be some sort of retry mechanism. Or even watch the PDB to see when the budget is refreshed.
What do you think?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.