[Termination Handler] Explore graceful termination for EC2 Instances about karpenter-provider-aws HOT 4 CLOSED

This may be a complete solution https://github.com/aws/aws-node-termination-handler with no work on our side.

from karpenter-provider-aws.

I think the aws-node-termination-handler would integrate with no additional work required for Karpenter. NTH does require a quite a bit of customer setup (creating the lifecycle hooks, eventbridge rules, and SQS queue), but after the initial setup, it can respond to a lot of events very easily.

Since Karpenter is already managing node groups, I'm curious if some of that setup (at least the lifecycle hooks) could be abstracted away and then NTH could just plugin to handle them or receive them from karpenter? The plan for NTH to ease the setup burden is to integrate with ACK but it's still early and most of the resources we need are not available yet.

Knowledge of the actual event does come in handy when processing events. NTH takes slightly different actions depending on the event. For example, ASG lifecycle terminations have different post draining actions than an EC2 Status Change event since the lifecycle hook does not need to be completed. Also EC2 scheduled maintenance event reboots are handled differently in the NTH IMDS processor since the node can be labeled and automatically brought back into service after the reboot.

from karpenter-provider-aws.

Also worth investigating this guy: https://github.com/pusher/k8s-spot-termination-handler

from karpenter-provider-aws.

My current stance on this is that we should recommend that users rely on https://github.com/aws/aws-node-termination-handler for at least v0.4. We should explore building a karpenter native interruption handler, but this use case (rebalance) should be scoped into the defragmentation design.

from karpenter-provider-aws.