Git Product home page Git Product logo

kafka-dockeree's Introduction

Introduction

This is a development-only container of kafka based on instructions here: https://dzone.com/articles/running-apache-kafka-on-windows-os.

It includes zookeeper and a single kafka instance. All storage is ephemeral (but you could change that if needed).

Its mainly tested on Windows Docker EE but works on CE too.

Getting Started

On Windows Server, follow standard instructions for setting up Docker EE.

E.g. instructions for setting up windows docker here: https://docs.docker.com/install/windows/docker-ee/

On Windows 10, install Docker CE: https://hub.docker.com/editions/community/docker-ce-desktop-windows On Windows 10 1809 onwards you can install Docker EE as well: https://www.kauffmann.nl/2019/03/04/how-to-install-docker-on-windows-10-without-hyper-v/

On CE I've previously had error "The operation timed out because a response was not received from the Virtual Machine Container. (0xc0370109)" during container build - just retrying the build fixed it.

Build and Test

Build and run the container via build.ps1. It takes a very long time on first run.

Once running, run with run.ps1. Note the first 3 digits of the container id and plug into the following powershell commands to test the container is working internally:

  $id=700
  # this command creates a named topic with simple replication options
  docker exec -i $id /kafka/bin/windows/kafka-topics.bat --create --zookeeper localhost:2181 --topic test --partitions 1 --replication-factor 1
  # start this producer in one shell:
  docker exec -i $id /kafka/bin/windows/kafka-console-producer.bat --broker-list kafka_dockeree_server:9092 --topic test 
  # start this consumer in another:
  docker exec -i $id /kafka/bin/windows/kafka-console-consumer.bat --bootstrap-server kafka_dockeree_server:9092 --topic test
  # messages typed in the producer shell get echoed to the consumer

You should be then able to connect to the cluster on kafka_dockeree_server:9092.

Contribute

If you want to, feel free!

kafka-dockeree's People

Contributors

melohenrique avatar peterwishart avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

kafka-dockeree's Issues

Up to date modifications

Hello,

First off, thanks for all the hard work you've put into this! I get that in a perfect world, Kafka would be running on Linux with or without Docker. But let's face it, sometimes that's just not possible without spending a ton of time migrating and configuring other stuff.

In my case, I can't justify spending all that time to migrate everything right now. One of my servers doesn't even support Hyper-V, which means no Linux subsystems, and installing a bare-metal Linux distro would be a pretty big change for the client.

So, I think it makes sense to keep this up-to-date and maybe even publish the images to Docker Hub. I've made some modifications to the script (and removed Kafka-Manager), but if you want to keep it, just let me know and I'll create a PR.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.