Track progress for Kubernetes 1.8 / kubernetes-kafka v3.0.0 about kubernetes-kafka HOT 6 CLOSED

• rename manifests from NNname.yml to NN-name.yml?

from kubernetes-kafka.

Found an explanation for why selector: needs to be added to workload API manifests:
From daemonset but I guess it applies to other resources as well:
"The pod selector will no longer be defaulted when left empty. Selector defaulting was not compatible with kubectl apply."

from kubernetes-kafka.

An update on testing, based on #79 (comment). I lack the Kafka experiment to interpret current results, so I think I want 3 tests that do essentially the same but with different clients.

What we want to assert is basically "uptime" in the face of re-scheduled broker and zk pods, caused by things like node downtime, cluster upgrades or zone outages. Tests continuously "bootstrap" + consume from a topic with 2 replicas (b8bfda8) + regularly produce messages + assert that those messages get consumed.

Measurements of throughput etc will have to wait until we have Prometheus monitoring up and running (#49 + Yolean/kubernetes-monitoring + ServiceMonitors + rules).

The three tests are:

• The kafka image's console producer + consumer (test-produce-consume)
• Kafkacat (#79)
• https://github.com/Yolean/kafka-test-failover (i.e. the java client with some typical usage pattern/config)

As a complement it'll be interesting to have kafkacat with new bootstrap for every assert run, i.e. https://github.com/Yolean/kubernetes-kafka/blob/master/test/basic-with-kafkacat.yml prior to #79)

from kubernetes-kafka.

Killing pods and watching test readiness I tend to need a tab that does a human readable variant of #60. My new favorite oneliner 😄 (with alias from solsson/kubectx#1 (comment)):

while :; do k get pods --all-namespaces -w; done | gawk '{ print strftime("%FT%T"), $0; fflush() }'

from kubernetes-kafka.

Tests have been put into practice in #79 (comment)

Huge improvement that we don't spin up 2 JVMs every 10s :)

from kubernetes-kafka.

Closing this messy ticket in favor of https://github.com/Yolean/kubernetes-kafka/milestone/1, with scope reductions -> https://github.com/Yolean/kubernetes-kafka/milestone/2.

from kubernetes-kafka.