sideshowbandana / k8s-sqs-autoscaler Goto Github PK

View Code? Open in Web Editor NEW

71.0 3.0 27.0 3 KB

Kubernetes pod autoscaler based on queue size in AWS SQS

Dockerfile 3.18% Makefile 3.90% Python 90.83% Shell 2.10%

kubernetes sqs autoscaler

k8s-sqs-autoscaler's Introduction

k8s-sqs-autoscaler

Kubernetes pod autoscaler based on queue size in AWS SQS

Usage

Create a kubernetes deployment like this:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-k8s-autoscaler
spec:
  revisionHistoryLimit: 1
  replicas: 1
  template:
    metadata:
      labels:
        app: my-k8s-autoscaler
    spec:
      containers:
      - name: my-k8s-autoscaler
        image: sideshowbandana/k8s-sqs-autoscaler:1.0.0
        command:
          - ./k8s-sqs-autoscaler
          - --sqs-queue-url=https://sqs.$(AWS_REGION).amazonaws.com/$(AWS_ID)/$(SQS_QUEUE) # required
          - --kubernetes-deployment=$(KUBERNETES_DEPLOYMENT)
          - --kubernetes-namespace=$(K8S_NAMESPACE) # optional
          - --aws-region=us-west-2  #required
          - --poll-period=10 # optional
          - --scale-down-cool-down=30 # optional
          - --scale-up-cool-down=10 # optional
          - --scale-up-messages=20 # optional
          - --scale-down-messages=10 # optional
          - --max-pods=30 # optional
          - --min-pods=1 # optional
        env:
          - name: K8S_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        resources:
          requests:
            memory: "64Mi"
            cpu: "250m"
          limits:
            memory: "1512Mi"
            cpu: "500m"
        ports:
        - containerPort: 80

k8s-sqs-autoscaler's People

Contributors

Stargazers

Watchers

k8s-sqs-autoscaler's Issues

Add license file

Could you please add a license file to this repo? I would like to use this but without a license this is under copyright by default.

When you make a creative work (which includes code), the work is under exclusive copyright by default. Unless you include a license that specifies otherwise, nobody else can copy, distribute, or modify your work without being at risk of take-downs, shake-downs, or litigation. Once the work has other contributors (each a copyright holder), “nobody” starts including you.

Source

I personally would recommend the MIT license.

Adding this would be greatly appreciated!

Rolling upgrades

Hi there,

I have this use case here:

We have haproxy running in container which gets their configuration generated when the pod starts. The configuration is generated by values published to consul server. The haproxy configuration loads SSL certs in its memory ( this was gotten from the consul server)
Assuming that a new customer is added new certificates are pushed to consul which are gotten only when the pad starts up.
Can a rolling upgrade be triggered when certain consul keys change( fyi, we have the consul watcher written in place)?
I would need your help if we can extend or use apis to trigger rolling upgrade similar to what you have done for sqs

Any help is appreciated on this!

Thanks
Abhay

RBAC errors

Not sure if this is the best way to set RBAC, but I got the API permission errors fixed by applying the following:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: sqs-scaler
  namespace: ${K8S_NAMESPACE}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: sqs-scaler
  namespace: ${K8S_NAMESPACE}
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: sqs-scaler
subjects:
- kind: ServiceAccount
  name: sqs-scaler
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: sqs-scaler
  namespace: ${K8S_NAMESPACE}
rules:
- apiGroups:
  - extensions
  - apps
  resources:
  - deployments
  - replicasets
  verbs:
  - get
  - list
  - watch
  - patch

(In k8s-sqs-autoscaler deployment)

...
    spec:
      serviceAccountName: sqs-scaler
...

Active Pod being deleted while scaling-down.

I have a minimum 2 replicas of my pod running at a time. If the SQS message count exceeds 2, let's say it becomes 5, the autoscaler should spin up 3 more replicas, in order to serve the excess 3 messages. This part (scale-up) is working fine.

Now once there are 5 replicas running, the autoscaler sometimes kills the pods that is actively running a process on the sqs message.

Please help. Thanks in advance.

deployments.items[0] -> IndexError: list index out of range

I found your SQS autoscaler from the Wattpad issues discussion (Wattpad/kube-sqs-autoscaler#7). Unfortunately, your tool is apparently suffering from it's own issues.

I ran into the region problem (#1), and was able to fix it with the suggested env var. However, I'm not faced with the following:

{'sqs_queue_url': '<sqs url>', 'kubernetes_deployment': '<deployment>', 'kubernetes_namespace': <namespace>', 'aws_region': '<region>', 'poll_period': 5, 'scale_down_cool_down': 300, 'scale_up_cool_down': 30, 'scale_up_messages': 2, 'scale_down_messages': 0, 'max_pods': 5, 'min_pods': 1}
Starting poll for <sqs url> every 5s
Waiting for scale up cooldown
Waiting for scale up cooldown
Waiting for scale up cooldown
Waiting for scale up cooldown
Waiting for scale up cooldown
Waiting for scale up cooldown
loading deployment: <deployment> from namespace: <namepsace>
Traceback (most recent call last):
  File "./k8s-sqs-autoscaler", line 71, in <module>
    sqs.run(options)
  File "/usr/src/app/sqs/sqs.py", line 95, in run
    SQSPoller(options).run()
  File "/usr/src/app/sqs/sqs.py", line 88, in run
    self.poll()
  File "/usr/src/app/sqs/sqs.py", line 35, in poll
    self.scale_up()
  File "/usr/src/app/sqs/sqs.py", line 50, in scale_up
    deployment = self.deployment()
  File "/usr/src/app/sqs/sqs.py", line 74, in deployment
    return deployments.items[0]
IndexError: list index out of range

The error isn't really all that clear. Is this due to a lack of RBAC permissions? Or just some bad logic in your code? I can't really tell, unfortunately.

You must specify a region .. though I have it specified `--aws_region=us-west-2`

I am getting following error though I have specified the region with --aws_region=us-west-2 parameter

{'sqs_queue_url': 'https://sqs.******.amazonaws.com/********/********', 'kubernetes_deployment': 'torpid-alligator', 'kubernetes_namespace': 'default', 'aws_region': 'us-west-2', 'poll_period': 10, 'scale_down_cool_down': 30, 'scale_up_cool_down': 10, 'scale_up_messages': 20, 'scale_down_messages': 10, 'max_pods': 5, 'min_pods': 1}
Traceback (most recent call last):
  File "./k8s-sqs-autoscaler", line 71, in <module>
    sqs.run(options)
  File "/usr/src/app/sqs/sqs.py", line 95, in run
    SQSPoller(options).run()
  File "/usr/src/app/sqs/sqs.py", line 16, in __init__
    self.sqs_client = boto3.client('sqs')
  File "/usr/local/lib/python3.6/site-packages/boto3/__init__.py", line 83, in client
    return _get_default_session().client(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/boto3/session.py", line 263, in client
    aws_session_token=aws_session_token, config=config)
  File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 861, in create_client
    client_config=config, api_version=api_version)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 76, in create_client
    verify, credentials, scoped_config, client_config, endpoint_bridge)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 295, in _get_client_args
    verify, credentials, scoped_config, client_config, endpoint_bridge)
  File "/usr/local/lib/python3.6/site-packages/botocore/args.py", line 45, in get_client_args
    endpoint_url, is_secure, scoped_config)
  File "/usr/local/lib/python3.6/site-packages/botocore/args.py", line 111, in compute_client_args
    service_name, region_name, endpoint_url, is_secure)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 368, in resolve
    service_name, region_name)
  File "/usr/local/lib/python3.6/site-packages/botocore/regions.py", line 122, in construct_endpoint
    partition, service_name, region_name)
  File "/usr/local/lib/python3.6/site-packages/botocore/regions.py", line 135, in _endpoint_for_partition
    raise NoRegionError()
botocore.exceptions.NoRegionError: You must specify a region.```

sideshowbandana / k8s-sqs-autoscaler Goto Github PK

k8s-sqs-autoscaler's Introduction

k8s-sqs-autoscaler

Usage

k8s-sqs-autoscaler's People

Contributors

Stargazers

Watchers

Forkers

k8s-sqs-autoscaler's Issues

Add license file

Rolling upgrades

RBAC errors

Active Pod being deleted while scaling-down.

deployments.items[0] -> IndexError: list index out of range

You must specify a region .. though I have it specified `--aws_region=us-west-2`

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent