thomas-george-t / kafka-producer-consumer-group-message-passing Goto Github PK

Java code for creating custom Producer and Consumer group demonstrating message passing in Kafka.

• Java JDK 8 (1.8)
• Zookeeper
• Kafka >= 2.0
• Linux/Windows
• Maven

Ensure that Zookeeper & Kafka servers are up and running ( Open in separate terminal windows if necessary ).

Command to start Zookeeper

zookeeper-server-start.sh config/zookeeper.properties

Command to start Kafka server

kafka-server-start.sh config/server.properties

After cloning this repo,

1. Run the maven clean install command

mvn clean install

Maven will now generate a target directory with the jar kafka-1.0-shaded.jar

2. Move into the target directory

cd target

1. Run the kafka console consumer in another terminal window with the following topic and group parameters

kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic firstTopic --group my-third-application

2. Execute the ProducerDemo class from the shaded jar

java -cp kafka-1.0-shaded.jar com.github.thomas.kafka.ProducerDemo

3. You should now be able to see the output in your Kafka console consumer terminal.

1. Execute the ConsumerDemo class from the shaded jar

java -cp kafka-1.0-shaded.jar com.github.thomas.kafka.ConsumerDemo

2. Run the kafka console producer in another terminal window with the topic parameter.

kafka-console-producer.sh --broker-list localhost:9092 --topic firstTopic

3. You should now be able to see the output in your Kafka consumer terminal. ( Step 1 )

1. Execute the ProducerDemo class from the shaded jar

java -cp kafka-1.0-shaded.jar com.github.thomas.kafka.ProducerDemo

2. Execute the ConsumerDemo class from the shaded jar

java -cp kafka-1.0-shaded.jar com.github.thomas.kafka.ConsumerDemo

3. You should now the see the output in your Consumer terminal.

Producer

ConsumerGroup

This repository is under Apache License 2.0 - see License for more details

This was inspired by Stephane Maarek. Check out his Apache Kafka Series course