sasl passthrough about kafka-proxy HOT 8 OPEN

this is already supported. if neither local sasl nor sasl by proxy is configured, sasl will passed to broker.

from kafka-proxy.

Thank you for your response. It appears this works the way you described if the kafka proxy's client chooses SASL_PLAINTEXT as the security protocol. Its failing with following error with SASL_SSL

Reading data from local connection on 127.0.0.1:3904 from 127.0.0.1:51253 (xxxx.xx.xx.com:9093) had error: api key -19711 is invalid

Port 9093 is SSL listener port on the Kafka broker.

If security protocol setting is not set at all in the client code, it results in following

Reading data from xxxx.xx.xx.com:9093 had error: unexpected EOF

from kafka-proxy.

The same issue, api key -19711 is invalid
security.protocol on broker side is SASL_SSL
@everesio is there any plan for the future releases to implicitly set security protocol setting?

from kafka-proxy.

I've had the same experience, what are we missing from the client or proxy command?

from kafka-proxy.

Same here, setting no sasl config in the proxy leads to no connection at all.

from kafka-proxy.

@bitkill , I already spend more then 30 hours to setup this but due to combination of various configuration on client, cluster and proxy end, It was long to setup but your one comment "no connection at all without sasl config on proxy" end my struggle. Thanks,

from kafka-proxy.

Is this issue still a problem? I am a bit confused by the discussion.

My situation is that I am able to use the kafka utilities to connect using a config like this:

client.id=event-hub
ssl.protocol=TLSv1.3   
ssl.enabled.protocols=TLSv1.3  
security.protocol=SASL_SSL 
sasl.mechanism=OAUTHBEARER 
sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required; 
sasl.login.callback.handler.class=com.adobe.core.pipeline.kafka.security.client.auth.ImsClientCallbackHandler  
sasl.ims.url=https://ims-na1.adobelogin.com
sasl.ims.secret.type=DIRECT
sasl.ims.secret.client.id=event-hub
sasl.ims.secret.client.secret=<redacted>
sasl.ims.secret.client.code=<redacted>
metric.reporters=org.apache.kafka.common.metrics.JmxReporter

And I can then run commands like:

./bin/kafka-console-consumer.sh --consumer.config config/consumer.properties --topic project-management_outbox_fullstate --bootstrap-server kafka-1-az1-or2-secure.prd.pipeline.adobedc.net:9097 --from-beginning

./bin/kafka-broker-api-versions.sh --bootstrap-server kafka-1-az1-or2-secure.prd.pipeline.adobedc.net:9097 --command-config config/consumer.properties

./bin/kafka-topics.sh --bootstrap-server kafka-1-az1-or2-secure.prd.pipeline.adobedc.net:9097 --command-config config/consumer.properties --list

./bin/kafka-get-offsets.sh --bootstrap-server kafka-1-az1-or2-secure.prd.pipeline.adobedc.net:9097 --command-config config/consumer.properties

However, I am not sure how to connect with kafka-proxy in the same way. Does it support passing through SASL parameters? Does it support a custom callback handler function (which I have to put a JAR file in the libs folder for)?

from kafka-proxy.

Hi @everesio,

I can only confirm what the previous writers say. It seems that when the kafka brokers are configured in a way that sasl is not running over plaintext and it's configured to do only forwarding the requests seem to hang.

Even though I try to set some sasl-method or enable sasl (which doesn't work as it now requires sasl.username and sasl.password) the mechanism seems not to work. The only exception that I see is this one:

{"@level":"info","@message":"New connection for kafka-cluster-1-kafka-0.kafka-cluster-1-kafka-brokers.kafka-cluster-1.svc.cluster.local:9093","@timestamp":"2023-09-22T10:56:46Z"}
{"@level":"debug","@message":"Kafka request key 17, version 0, length 40","@timestamp":"2023-09-22T10:56:46Z"}
{"@level":"debug","@message":"Kafka response key 17, version 0, length 25","@timestamp":"2023-09-22T10:56:46Z"}
{"@level":"info","@message":"Reading data from kafka-cluster-1-kafka-0.kafka-cluster-1-kafka-brokers.kafka-cluster-1.svc.cluster.local:9093 had error: open request is missing","@timestamp":"2023-09-22T10:56:51Z"}

from kafka-proxy.