Git Product home page Git Product logo

testcontainers-kafka-avro's Introduction

Production/Consumption of Apache Kafka AVRO messages with testcontainers library

This is a showcase on how to test the serialization/deserialization of AVRO messages over Apache Kafka with the help of docker containers (via testcontainers library).

The testcontainers library already offers a Kafka module for interacting with Apache Kafka, but there is not, at the moment, a testcontainers module for the whole Confluent environment (Confluent Schema Registry container support is missing from the module previously mentioned). As a side note, the containers used do not use the default ports exposed by default in the artifacts (e.g. : Apache Zookeeper 2181, Apache Kafka 9092, Confluent Schema Registry 8081), but rather free ports available on the test machine avoiding therefor possible conflicts with already running services on the test machine.

This project provides a functional prototype on how to setup the whole Confluent environment (including Confluent Schema Registry) via testcontainers.

For the test environment the following containers will be started:

  • Apache Zookeeper
  • Apache Kafka
  • Confluent Schema Registry

Once the test environment is started, a BookmarkEvent value type will be registered on the Confluent Schema Registry container and also a Apache Kafka topic called BookmarkEvents will be created.

The test demo will simply verify whether a message serialized in AVRO format can be successfully serialized and sent over Apache Kafka in order to subsequently be deserialized and read by a consumer.

Use

mvn clean install

for trying out the project.

The file testcontainers.properties can be used for overriding the default docker images used for the containers needed in setting up the Confluent test environment.

testcontainers-kafka-avro's People

Contributors

findinpath avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.