Configuration in Zookeeper (/config/brokers/{id}) isn't refreshed after changes made to KafkaCluster.spec.listenersConfig.externalListeners about koperator HOT 4 OPEN

@play-io Can you provide detailed steps for reproducing this issue?

Based on the given information, I can only tell that the new broker never gets registered to ZooKeeper, but can't tell what causes this without seeing the detailed steps and some logs

from koperator.

Dear @play-io !
Are you sure about you wanted to change the containerPort for the external listener?
Kafka listens on the specified containerPort in the pod but it is not the same as the exposed port which is available outside of the Kubernetes cluster through envoy. You have to configure the externalStartingPort to modify the advertised.listener port number. Currently the exposed port will be the externalStartingPort + brokerID.
More details:

from koperator.

Hi @panyuenlau

but can't tell what causes this without seeing the detailed steps and some logs

Makes sense. Steps to reproduce:

1. Put incorrect (config options start with -) configuraiton into brokers template config
   brokerConfigGroups:
   default:
   config: |-
   • "auto.create.topics.enable=false"
   • "cruise.control.metrics.topic.auto.create=true"
   • "cruise.control.metrics.topic.num.partitions=1"
   • "cruise.control.metrics.topic.replication.factor=1"
2. Assign the template to all brokers
   brokers:

• id: 0
  brokerConfigGroup: "default"
  ...

3. Apply the new config to a cluster
4. If after the new config is applied and brokers accessible then make a change to containerPort in externalListeners configuration.
   Given you have
   externalStartingPort: 19090
   containerPort: 9095
   Then change to
   externalStartingPort: 19090
   containerPort: 9096

Observations:

• Incorrect configuration somehow hasn't broken cluster from the beginning
• Just after making changes in externalListener ports.

Troubleshooting:
After the change my cluster was in Running state, but I could not access it:

envoy had correct (updated) configuration in the configmap
brokers exposed correct (updated) container port
BUT zookeeper. get /config/brokers/0 had advertised.listeners still pointing to port 9094
BUT zookeeper. get /config/brokers/1 did not exist

from koperator.

H @bartam1

Are you sure about you wanted to change the containerPort for the external listener?

Yes an I'm aware about its sense. Thanks for your comment

from koperator.