Use case different than sidecar about kafka-proxy HOT 16 CLOSED

You can configure kafka-proxy to run also on remote server
e.g.

 kafka-proxy server --bootstrap-server-mapping "kafka-0.broker:9092,0.0.0.0:32401,kafka-proxy.remote:32401" \
                   --bootstrap-server-mapping "kafka-1.broker:9092,0.0.0.0:32402,kafka-proxy.remote:32402" \
                   --bootstrap-server-mapping "kafka-2.broker:9092,0.0.0.0:32403,kafka-proxy.remote:32403" \
		   --dynamic-listeners-disable	

kafka-proxy.remote would be the host name where proxy runs. The kafka-proxy.remote:32501, kafka-proxy.remote:32502 and kafka-proxy.remote:32503 could be used as kafka bootstrap servers for the clients.

from kafka-proxy.

BTW. if you use older kafka clients e.g (kafka version 2.3), it should work

from kafka-proxy.

@gustavomcarmo please check the latest image. local auth should work with kafka 2.5 clients as well.

from kafka-proxy.

kafka proxy can intercept the sasl user/password and perform authentication against plugin e.g. LDAP.

make clean build plugin.auth-ldap && build/kafka-proxy server \
                             --auth-local-enable \
                             --auth-local-command build/auth-ldap \
                             --auth-local-param "--url=ldaps://ldap.example.com:636" \
                             --auth-local-param "--user-dn=cn=users,dc=exemple,dc=com" \
                             --auth-local-param "--user-attr=uid" \
                             --bootstrap-server-mapping "192.168.99.100:32400,127.0.0.1:32400"

LDAP plugin will perform user bind with DN.
Client jaas is not different than standard jaas used for SASL

KafkaClient {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="alice"
  password="alice-secret";
};

from kafka-proxy.

1. I've just added SaslAuthenticateRequestV1, so you must use latest image with the 21f0ff9.
2. Set sasl.mechanism=PLAIN in you client.properties
3. Your user has a user-attr cn not uid, you must disable tls

      - server
      - --bootstrap-server-mapping=kafka-0:9092,0.0.0.0:32400,localhost:32400
      - --bootstrap-server-mapping=kafka-1:9092,0.0.0.0:32401,localhost:32401
      - --bootstrap-server-mapping=kafka-2:9092,0.0.0.0:32402,localhost:32402
      - --dynamic-listeners-disable
      - --auth-local-enable
      - --auth-local-command=/auth-ldap
      - --auth-local-param=--start-tls=false
      - --auth-local-param=--url=ldap://openldap:389
      - --auth-local-param=--user-dn=ou=people,dc=example,dc=org
      - --auth-local-param=--user-attr=cn  

from kafka-proxy.

ok. thanks for reporting. this kafka api change is not implemented yet. I will notify you when it is done

from kafka-proxy.

release v0.2.4 is out.

LDAP group filtering (and maybe caching) would be a nice feature, but I will not have time to implement it soon.
Feel free to contribute.

from kafka-proxy.

Thanks a lot, @everesio!

I'm going now into LDAP authentication configuration, trying to figure out the right Kafka client settings to use, unfortunately with no success. Could you please give me an example of jaas.conf and other eventual configuration required for the client?

I'm eager to make it work and contribute with the kafka-proxy at least improving the documentation 😄

from kafka-proxy.

I'm using scripts from the bin folder of the Kafka installation (kafka-topics.sh, kafka-console-producer.sh and kafka-console-consumer.sh) for testing and I'm facing some issues. I'll provide more details about that in a new comment soon.

Have you ever used this Kafka client implementation for testing the kafka-proxy LDAP authentication?

Thanks, @everesio!

from kafka-proxy.

Hi @everesio,

I've just created the kafka-proxy-test project with all the details regarding my tests. I appreciate any help 😃

from kafka-proxy.

Hi @everesio,

After your latest instructions, I'm still getting errors. Please take a look once more at the kafka-proxy-test.

Thanks!

from kafka-proxy.

Yes, you're right! Thanks a lot! Check it out here.

Isn't it something to have in the README? The Kafka client versions supported. If you don't mind, I can submit a PR with this in the documentation.

from kafka-proxy.

Hi @everesio,

Now it works, thanks! When are you planning to create a new release?

BTW, the kafka-proxy LDAP authentication plugin doesn't support group filtering yet, right? This would be a nice enhancement, while not all of the LDAP users should access Kafka, but only those who would be in a specific LDAP group. What are your thoughts about that?

from kafka-proxy.

Hi @everesio,

I'll find some time to contribute with the LDAP group filtering feature.

BTW, has kafka-proxy some limitation in terms of number of applications connected? I've tried to connect a second application to kafka-proxy, from a different machine, with no success.

Thanks!

from kafka-proxy.

Hi @everesio,

Is there a logo for kafka-proxy? I would like to add it to a project diagram.

Thanks!

from kafka-proxy.

Hi @gustavomcarmo

1. If you want to add LDAP group filtering feature, please create a PR.
2. There are no limitations in proxy in terms of number of applications connected. If you cannot connect from a different machine, there is probably some misconfiguration. Due kafka protocol specifics, the setup can be sometimes tricky as brokers' advertised listeners, proxy settings and client config must "match".
3. There is no logo for kafka-proxy. If you want you can propose something ;-)
   Kind regards,
   Michal

from kafka-proxy.