Comments (3)
Hi,
could you provide your current kind and strimzi setup ?
Kind regards,
Michal
from kafka-proxy.
Hey @everesio!, here's my configuration:
Kind:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 30005 # trying to use this port to expose kafka-proxy
hostPort: 30005
strimzi:
apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
name: kueue
spec:
kafka:
version: 2.6.0
replicas: 1
listeners:
plain: {}
tls: {}
config:
offsets.topic.replication.factor: 1
transaction.state.log.replication.factor: 1
transaction.state.log.min.isr: 1
log.message.format.version: "2.6"
default.replication.factor: 1
storage:
type: ephemeral
zookeeper:
replicas: 1
storage:
type: ephemeral
entityOperator:
topicOperator: {}
userOperator: {}
from kafka-proxy.
Hi @jarojasm95,
please try this out
- pods after kafka strimzi operator and kafka CRD were installed in
kafka
namespace
$ kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kafka kueue-entity-operator-6b9d789c88-zl9vk 3/3 Running 0 21h
kafka kueue-kafka-0 1/1 Running 0 21h
kafka kueue-zookeeper-0 1/1 Running 0 21h
kafka strimzi-cluster-operator-54ff55979f-tsdsl 1/1 Running 0 21h
kube-system coredns-f9fd979d6-h9jvb 1/1 Running 0 22h
kube-system coredns-f9fd979d6-rbsv4 1/1 Running 0 22h
kube-system etcd-strimzi-kafka-proxy-control-plane 1/1 Running 0 22h
kube-system kindnet-pbx44 1/1 Running 0 22h
kube-system kube-apiserver-strimzi-kafka-proxy-control-plane 1/1 Running 0 22h
kube-system kube-controller-manager-strimzi-kafka-proxy-control-plane 1/1 Running 0 22h
kube-system kube-proxy-nwkgm 1/1 Running 0 22h
kube-system kube-scheduler-strimzi-kafka-proxy-control-plane 1/1 Running 0 22h
local-path-storage local-path-provisioner-78776bfc44-zdvrk 1/1 Running 0 22h
- check kafka config
$ kubectl exec -it -n kafka kueue-kafka-0 -- ps -ef
UID PID PPID C STIME TTY TIME CMD
kafka 1 0 0 Dec12 ? 00:00:01 /usr/bin/tini -w -e 143 -- /opt/kafka/bin/kafka-server-start.sh /tmp/strimzi.properties
...
$ kubectl exec -it -n kafka kueue-kafka-0 -- cat /tmp/strimzi.properties
...
listeners=REPLICATION-9091://0.0.0.0:9091,PLAIN-9092://0.0.0.0:9092,TLS-9093://0.0.0.0:9093
advertised.listeners=REPLICATION-9091://kueue-kafka-0.kueue-kafka-brokers.kafka.svc:9091,PLAIN-9092://kueue-kafka-0.kueue-kafka-brokers.kafka.svc:9092,TLS-9093://kueue-kafka-0.kueue-kafka-brokers.kafka.svc:9093
listener.security.protocol.map=REPLICATION-9091:SSL,PLAIN-9092:PLAINTEXT,TLS-9093:SSL
...
PLAINTEXT listener is advertised as kueue-kafka-0.kueue-kafka-brokers.kafka.svc:9092
- deploy kafka-proxy and use 30005 as node port which in you kind config is exposed as host port
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: kafka-proxy
spec:
replicas: 1
selector:
matchLabels:
app: kafka-proxy
template:
metadata:
labels:
app: kafka-proxy
spec:
containers:
- name: kafka-proxy
image: grepplabs/kafka-proxy:v0.2.8
args:
- 'server'
- '--log-level=debug'
- '--bootstrap-server-mapping=kueue-kafka-0.kueue-kafka-brokers.kafka.svc:9092,0.0.0.0:30005'
---
apiVersion: v1
kind: Service
metadata:
name: kafka-proxy
spec:
type: NodePort
selector:
app: kafka-proxy
ports:
- protocol: TCP
port: 30005
targetPort: 30005
nodePort: 30005
- Now you can use kafka consumer / producer from your host
bin/kafka-console-producer.sh --broker-list 127.0.0.1:30005 --topic my-topic
bin/kafka-console-consumer.sh --bootstrap-server 127.0.0.1:30005 --topic my-topic --from-beginning
from kafka-proxy.
Related Issues (20)
- IAM auth passthrough to MSK with encryption in transit?
- MSK TLS Errors HOT 1
- TLS extract CN and lookup on LDAP for SASL
- cannot connect to aws kafka from on-prem server with SASL_SSL enabled HOT 2
- Simple use case to connect docker compose container to remote vpn kafka cluster over socks5a proxy via ssh HOT 1
- Pod startup issue after version 0.3.3-all HOT 3
- CVE-2023-37788 - github.com/elazarl/goproxy HOT 3
- AWS MSK Serverless - had error: api key -13567 is invalid HOT 4
- updating to tag 0.3.7-all from 0.3.3-all getting error auth-local-command HOT 1
- [Question] Can I attach 3 bootstrap server endpoints to a single port? HOT 1
- [Question] If my Kafka brokers are running version 2.8.1, should I be using kafka-proxy version 0.2.9? HOT 1
- "Metadata" request (ApiKey=3 and ApiVersion=5) in the Kafka Proxy is not following the protocol structure defined by Kafka protocol guide HOT 1
- [Question] is there a plan to release a Java implementation of Kafka Proxy ? HOT 1
- [Need Help] Sending Custom METADATA response through Kafka Proxy
- [Question] is there a plan to support HTTPS proxy ?
- will there be an update to resovle 7 vulnerabilitys
- tls: failed to parse private key AWS MSK HOT 6
- bad performance when executing kafka-producer-perf-test.sh HOT 3
- Can not use grepplabs/kafka-proxy ARM image as base image HOT 1
- one port mapping to 6 broker HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kafka-proxy.