Comments (10)
@Kotsubinsky Cruise Control should support all the protocol supported by Apache Kafka clients. Can you try set the property sasl.mechanism=GSSAPI?
from cruise-control.
@Kotsubinsky Actually it looks that we are not passing the corresponding configurations to the clients properly. I'll fix that.
from cruise-control.
@Kotsubinsky Can you try the latest master branch again? It should have been fixed.
from cruise-control.
@becketqin , Thank you for you update,
but i have same error when i run cruise-control:
security.protocol = SASL_PLAINTEXT
(com.linkedin.kafka.cruisecontrol.config.KafkaCruiseControlConfig)
Exception in thread "main" java.lang.IllegalArgumentException: `clientSaslMechanism` must be non-null in client mode if `securityProtocol` is `SASL_PLAINTEXT`
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:59)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:85)
at com.linkedin.kafka.cruisecontrol.common.MetadataClient.<init>(MetadataClient.java:43)
at com.linkedin.kafka.cruisecontrol.monitor.LoadMonitor.<init>(LoadMonitor.java:94)
at com.linkedin.kafka.cruisecontrol.KafkaCruiseControl.<init>(KafkaCruiseControl.java:75)
at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlMain.main(KafkaCruiseControlMain.java:47)
I add to config only 2 line
$ egrep "sasl.mechanism|security.protocol" config/cruisecontrol.properties
security.protocol=SASL_PLAINTEXT
sasl.mechanism=GSSAPI
Do you add support of 'sasl.mechanism' parameter in code:
https://github.com/linkedin/cruise-control/blob/master/cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/config/KafkaCruiseControlConfig.java
from cruise-control.
@Kotsubinsky Sorry for the late reply. Yeah, you are right. I added the sasl configs in #45 . We usually call config.originals()
when configure the producers and consumers, but in the metadata client, we actually was calling configs.values()
.
from cruise-control.
@Kotsubinsky I am closing this issue as #45 has been checked in. Feel free to reopen this if it does not work for you.
from cruise-control.
@becketqin , yes sasl mechanism is passed from parameters, but i have one exception, while run cruise-control
[2017-10-18 14:30:42,634] INFO Client will use GSSAPI as SASL mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2017-10-18 14:30:42,635] INFO Opening socket connection to server ns-hd06/172.25.26.6:2181. Will attempt to SASL-authenticate using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn)
[2017-10-18 14:30:42,636] INFO Socket connection established to ns-hd06/172.25.26.6:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2017-10-18 14:30:42,685] INFO Session establishment complete on server ns-hd06/172.25.26.6:2181, sessionid = 0x15f2e4a04030007, negotiated timeout = 30000 (org.apache.zookeeper.ClientCnxn)
[2017-10-18 14:30:42,687] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2017-10-18 14:30:42,838] INFO zookeeper state changed (SaslAuthenticated) (org.I0Itec.zkclient.ZkClient)
Exception in thread "main" kafka.common.KafkaException: Failed to parse the broker info from zookeeper: {"jmx_port":-1,"timestamp":"1508310423499","endpoints":["PLAINTEXTSASL://ns-hd07:6667"],"host":null,"version":3,"port":-1}
at kafka.cluster.Broker$.createBroker(Broker.scala:125)
at kafka.utils.ZkUtils.getBrokerInfo(ZkUtils.scala:843)
at kafka.utils.ZkUtils$$anonfun$getAllBrokersInCluster$2.apply(ZkUtils.scala:286)
at kafka.utils.ZkUtils$$anonfun$getAllBrokersInCluster$2.apply(ZkUtils.scala:286)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at kafka.utils.ZkUtils.getAllBrokersInCluster(ZkUtils.scala:286)
at com.linkedin.kafka.cruisecontrol.monitor.sampling.KafkaSampleStore.ensureTopicCreated(KafkaSampleStore.java:131)
at com.linkedin.kafka.cruisecontrol.monitor.sampling.KafkaSampleStore.configure(KafkaSampleStore.java:104)
at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:207)
at com.linkedin.kafka.cruisecontrol.monitor.task.LoadMonitorTaskRunner.<init>(LoadMonitorTaskRunner.java:86)
at com.linkedin.kafka.cruisecontrol.monitor.task.LoadMonitorTaskRunner.<init>(LoadMonitorTaskRunner.java:61)
at com.linkedin.kafka.cruisecontrol.monitor.LoadMonitor.<init>(LoadMonitor.java:130)
at com.linkedin.kafka.cruisecontrol.monitor.LoadMonitor.<init>(LoadMonitor.java:93)
at com.linkedin.kafka.cruisecontrol.KafkaCruiseControl.<init>(KafkaCruiseControl.java:75)
at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlMain.main(KafkaCruiseControlMain.java:47)
Caused by: java.lang.IllegalArgumentException: No security protocol defined for listener PLAINTEXTSASL
at kafka.cluster.EndPoint$$anonfun$securityProtocol$1$1.apply(EndPoint.scala:48)
at kafka.cluster.EndPoint$$anonfun$securityProtocol$1$1.apply(EndPoint.scala:48)
at scala.collection.MapLike$class.getOrElse(MapLike.scala:128)
at scala.collection.AbstractMap.getOrElse(Map.scala:59)
at kafka.cluster.EndPoint$.securityProtocol$1(EndPoint.scala:47)
at kafka.cluster.EndPoint$.createEndPoint(EndPoint.scala:56)
at kafka.cluster.Broker$$anonfun$2.apply(Broker.scala:115)
at kafka.cluster.Broker$$anonfun$2.apply(Broker.scala:115)
at scala.collection.immutable.List.map(List.scala:284)
at kafka.cluster.Broker$.createBroker(Broker.scala:115)
... 19 more
from cruise-control.
@Kotsubinsky What is the Kafka broker version you are running?
from cruise-control.
@becketqin
[2017-10-18 18:51:27,818] INFO Kafka version : 0.10.2.1 (org.apache.kafka.common.utils.AppInfoParser
from cruise-control.
i get information from zookeeper, and it the same string. Probably cruise-control can't parse this string...
get /brokers/ids/1001 {"jmx_port":-1,"timestamp":"1508310423499","endpoints":["PLAINTEXTSASL://ns-hd07:6667"],"host":null,"version":3,"port":-1} cZxid = 0x700000024 ctime = Wed Oct 18 14:07:03 KRAT 2017 mZxid = 0x700000024 mtime = Wed Oct 18 14:07:03 KRAT 2017 pZxid = 0x700000024 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x15f2e4a04030002 dataLength = 139 numChildren = 0
from cruise-control.
Related Issues (20)
- Document units in capacityJBOD.json HOT 2
- Partition load result shows discrepancy with the actual kafka partition
- A potential bug in taking action to optimize goal for kafka assignment.
- CVE-2023-43642 and CVE-2023-31582
- Allow specifying prometheus.server.endpoint without port
- Add metrics about total and remaining data to rebalance
- We should update the README for kafka-version support
- cruise control rebalance rack aware
- Error log missing for tasks completedWithError
- compile fails with java 21
- multiple fixable vulnerabilities
- Cruise Control with MSK HOT 2
- Slow task execution due to long retry backoff in ReplicationThrottleHelper
- Does CC support MSK/Kafka 2.8.1? - NoSuchMethodError: 'void kafka.zk.AdminZkClient.<init> (CC 2.5.137) HOT 4
- log4j not working when synced to latest main branch HOT 2
- Is there a final jar available with all dependency jars in jfrog HOT 5
- How do I enable DEBUG logging for Cruise Control? HOT 1
- Failed to describe Kafka cluster configs when using KafkaAdminTopicConfigProvider HOT 8
- Get cluster load API (sometimes) ignores `start` and `end` arguments
- CC reporting less disk for partition load
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cruise-control.