ianlewis / memcached-operator Goto Github PK

View Code? Open in Web Editor NEW

151.0 3.0 18.0 33.46 MB

A Kubernetes operator for memcached

License: Apache License 2.0

Go 97.24% Shell 2.14% Makefile 0.63%

kubernetes-operator kubernetes memcached

memcached-operator's Introduction

Memcached Operator

memcached-operator is a Kubernetes Operator for deploying and managing a cluster of Memcached instances.

memcached-operator provides a single Service endpoint that memcached client applications can connect to to make use of the memcached cluster. It provides this via a memcached proxy which is automatically updated whenever memcached instances are added or removed from the cluster.

memcached-operator supports sharded and replicated pools of servers as well as combinations of both strategies.

See the documentation for more information.

Project Status

Project status: alpha

memcached-operator is still under active development and has not been extensively tested yet. Use at your own risk. Backward-compatibility is not supported for alpha releases.

Prerequisites

Version >= 1.10 of Kubernetes.

memcached-operator relies on subresources for CRDs which is in 1.10+

Quickstart

You can install the memcached-operator using the included helm chart. Check out the git repository and run this in the root directory.

$ helm install --name memcached-operator charts/memcached-operator

The easiest way to create a memcached cluster is using the memcached helm chart:

$ helm install --name sharded stable/memcached

You can then create a memcached proxy to connect to the cluster.

apiVersion: ianlewis.org/v1alpha1
kind: MemcachedProxy
metadata:
  name: sharded-example
spec:
  rules:
    type: "sharded"
    service:
      name: "sharded-memcached"
      port: 11211

$ kubectl apply -f docs/sharded-example.yaml

You can then access your memcached cluster via thesharded-memcachedservice. Check the documentation for more information.

Removal

You can remove the memcached-operator by deleting the helm release.

$ helm delete --purge memcached-operator

Development

Check out memcached-operator to your GOPATH

Building

memcached-operator can be built using the normal Go build tools. This will build a binary dynamically linked to glibc.

$ go build

You can build a fully statically linked binary as well:

$ make build

Similar projects

Disclaimers

This is not an official Google product

memcached-operator's People

Contributors

Stargazers

Watchers

Forkers

sun363587351 mcspx atul9 hidapple maduhu yuong2015 funkemediengruppe hiteshchaudhari1 shuinoo naidu-kjml dinesh nigel914 naemono kiranmenon agsenthil devkmsg fearlesschenc figoxu

memcached-operator's Issues

Fix golint issues

https://goreportcard.com/report/github.com/ianlewis/memcached-operator#golint

Support scaling mcrouter

Support scaling mcrouter pods. mcrouter is not likely CPU bound, but rather I/O bound. It should probably be scaled using custom/metrics like concurrent connection count or I/O throughput.

mcrouter deployment sometimes not created

If a memcachedproxy is created that points to a service that doesn't exist yet, the memcached-operator will fail to create the mcrouter configmap, and thus will fail to create the mcrouter Deployment.

When the memcached cluster service is created, the configmap is then created but the deployment is never created because the proxydeployment controller doesn't watch for configmap changes.

Issue when running in a namespace.

Hey,
I'm trying to deploy memcached-operator in a namespace called memcached where my memcached chart is deployed. When I'm trying to create a memcachedproxies.ianlewis.org resource, it's not generating a service(Which I suppose how it should work).
I'm getting this error on the prometheus-operator pod
E1015 05:47:22.856956 1 reflector.go:205] github.com/ianlewis/memcached-operator/vendor/github.com/ianlewis/controllerutil/informers/informers.go:99: Failed to list *v1beta1.Deployment: deployments.extensions is forbidden: User "system:serviceaccount:memcached:memcached-operator-memcached-operator" cannot list deployments.extensions at the cluster scope: Unknown user "system:serviceaccount:memcached:memcached-operator-memcached-operator"
I'm using this command to install the helm chart
helm install --name memcached-operator memcached-operator/charts/memcached-operator --namespace memcached

Thanks in advance. :)

Immutable configmaps

I should use immutable ConfigMaps. Each change to mcrouter's configuration should generate a new ConfigMap and trigger a rolling-update on the mcrouter Deployment. ConfigMaps should be garbage collected so they don't accumulate.

ConfigMap is created with owner reference pointing to the owning MemcachedProxy. That way if the MemcachedProxy is deleted before any action is taken by the controller, the built-in garbage collector will garbage collect the ConfigMap
The new ConfigMap name is set to the Deployment triggering a rolling-update
Once a new ReplicaSet is created for the Deployment the owner reference for the ConfigMap is changed from the MemcachedProxy to the ReplicaSet. That way the ConfigMap is garbage collected by the built-in garbage collector when the ConfigMap is not needed by any ReplicaSet. How long data stays around can be adjusted using the Deployment's clean up policy

Use generation/observedGeneration

generation is supported in CRDs since 1.10. Use that instead of hashing the MemcachedProxySpec

Sometimes hangs on startup

Create our own mcrouter Docker image

Memcached operator should have it's own mcrouter docker image and Dockerfile that folks can use to customize, build, and deploy to their own private registries.

Store mcrouter config in Deployment

Instead of storing config for mcrouter in a configmap, it could be passed to the application through an environment variable or command line parameter on the Deployment and written to the config file by an init container.

Support for managing memcached clusters

I want to add the ability for memcached operator to create and manage the cluster itself.

Some ideas:

Should have the ability to break the cluster into multiple pools across failure domains (zones). Sharding and replication can be done across pools of servers using MemcachedProxy
Should allow setting the image used by memcached servers to allow use of private registries etc.