Git Product home page Git Product logo

Comments (3)

everesio avatar everesio commented on June 3, 2024

Hi,
could you provide your current kind and strimzi setup ?
Kind regards,
Michal

from kafka-proxy.

jarojasm95 avatar jarojasm95 commented on June 3, 2024

Hey @everesio!, here's my configuration:
Kind:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 30005  # trying to use this port to expose kafka-proxy
    hostPort: 30005

strimzi:

apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
  name: kueue
spec:
  kafka:
    version: 2.6.0
    replicas: 1
    listeners:
      plain: {}
      tls: {}
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      log.message.format.version: "2.6"
      default.replication.factor: 1
    storage:
      type: ephemeral
  zookeeper:
    replicas: 1
    storage:
      type: ephemeral
  entityOperator:
    topicOperator: {}
    userOperator: {}

from kafka-proxy.

everesio avatar everesio commented on June 3, 2024

Hi @jarojasm95,

please try this out

  1. pods after kafka strimzi operator and kafka CRD were installed in kafka namespace
$ kubectl get pods -A                         
NAMESPACE            NAME                                                        READY   STATUS    RESTARTS   AGE
kafka                kueue-entity-operator-6b9d789c88-zl9vk                      3/3     Running   0          21h
kafka                kueue-kafka-0                                               1/1     Running   0          21h
kafka                kueue-zookeeper-0                                           1/1     Running   0          21h
kafka                strimzi-cluster-operator-54ff55979f-tsdsl                   1/1     Running   0          21h
kube-system          coredns-f9fd979d6-h9jvb                                     1/1     Running   0          22h
kube-system          coredns-f9fd979d6-rbsv4                                     1/1     Running   0          22h
kube-system          etcd-strimzi-kafka-proxy-control-plane                      1/1     Running   0          22h
kube-system          kindnet-pbx44                                               1/1     Running   0          22h
kube-system          kube-apiserver-strimzi-kafka-proxy-control-plane            1/1     Running   0          22h
kube-system          kube-controller-manager-strimzi-kafka-proxy-control-plane   1/1     Running   0          22h
kube-system          kube-proxy-nwkgm                                            1/1     Running   0          22h
kube-system          kube-scheduler-strimzi-kafka-proxy-control-plane            1/1     Running   0          22h
local-path-storage   local-path-provisioner-78776bfc44-zdvrk                     1/1     Running   0          22h
  1. check kafka config
$ kubectl exec -it -n kafka kueue-kafka-0 -- ps -ef
UID          PID    PPID  C STIME TTY          TIME CMD
kafka          1       0  0 Dec12 ?        00:00:01 /usr/bin/tini -w -e 143 -- /opt/kafka/bin/kafka-server-start.sh /tmp/strimzi.properties
...

$ kubectl exec -it -n kafka kueue-kafka-0 -- cat /tmp/strimzi.properties
...
listeners=REPLICATION-9091://0.0.0.0:9091,PLAIN-9092://0.0.0.0:9092,TLS-9093://0.0.0.0:9093
advertised.listeners=REPLICATION-9091://kueue-kafka-0.kueue-kafka-brokers.kafka.svc:9091,PLAIN-9092://kueue-kafka-0.kueue-kafka-brokers.kafka.svc:9092,TLS-9093://kueue-kafka-0.kueue-kafka-brokers.kafka.svc:9093
listener.security.protocol.map=REPLICATION-9091:SSL,PLAIN-9092:PLAINTEXT,TLS-9093:SSL
...

PLAINTEXT listener is advertised as kueue-kafka-0.kueue-kafka-brokers.kafka.svc:9092

  1. deploy kafka-proxy and use 30005 as node port which in you kind config is exposed as host port
---
apiVersion: apps/v1
kind: Deployment
metadata:
   name: kafka-proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka-proxy
  template:
    metadata:
      labels:
        app: kafka-proxy
    spec:
      containers:
        - name: kafka-proxy
          image: grepplabs/kafka-proxy:v0.2.8
          args:
            - 'server'
            - '--log-level=debug'
            - '--bootstrap-server-mapping=kueue-kafka-0.kueue-kafka-brokers.kafka.svc:9092,0.0.0.0:30005'
---
apiVersion: v1
kind: Service
metadata:
  name: kafka-proxy
spec:
  type: NodePort
  selector:
    app: kafka-proxy
  ports:
    - protocol: TCP
      port: 30005
      targetPort: 30005
      nodePort: 30005
  1. Now you can use kafka consumer / producer from your host
bin/kafka-console-producer.sh --broker-list 127.0.0.1:30005 --topic my-topic
bin/kafka-console-consumer.sh --bootstrap-server 127.0.0.1:30005 --topic my-topic --from-beginning

from kafka-proxy.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.