Git Product home page Git Product logo

kafka-topic-viewer's Introduction

kafka-topic-viewer

A simple web-based viewer to monitor messages on topic(s) in Kafka.

Connects to Kafka via a WebSockets based proxy (using kafka-proxy-ws).

Screenshots

Features

  • Monitor multiple Kafka topics
  • Receive Kafka messages in real-time

Quick Start

  1. Ensure docker is installed and running.

  2. Run the following commands:

$ docker pull jchen86/kafka-topic-viewer
$ docker run --rm -it -p 8080:8080 -p 9999:9999 \
             -e KAFKA_URL=http://your-kafka-url:port \
             -e SERVE_STATIC=true \
             jchen86/kafka-topic-viewer

Building the Docker image

  1. Run npm build on client directory
$ cd client
$ npm run build
  1. Run docker build on project root directory
$ cd ..
$ docker build -t jchen86/kafka-topic-viewer .

Running from source

  1. Configure the Kafka URL in server/config.js. Default is localhost:9092.

  2. Start the Proxy server

$ npm install && npm run start
  1. Configure the topic names to monitor in client/src/app/config.js

  2. Start the client app

$ cd client
$ npm install && npm run serve

Kafka Topic Viewer should now be live on your server at port 3000.

License

The project is licensed under the MIT license.

kafka-topic-viewer's People

Contributors

jimmygchen avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

onenix icednut

kafka-topic-viewer's Issues

Message count on a topic is limited by the display limit config

The message count next to the topic should display number of total messages received since monitoring started.

Currently it displays correctly up to the display limit, for example: if 10 is the max number of message to display and store on client, it only display count up to 10.

Add topic toggle to side bar

Add ON/OFF topic toggles. By default (configurable), first 3 topics will be toggled ON.

As a user
I want to be able to toggle topics on and off
so that I only see topics that I am interested in.

Support for additional or custom deserialization methods

Currently the viewer only supports message in JSON or plain text format.
This is due to the kafka-proxy-ws library converting the byte arrays using toString() then stringifying all messages using JSON.stringify.

It is common to produce kafka messages using a data serialisation system such as Avro. It would be useful to add support for additional or custom deserialization methods.

Display message title using a provided expression

The title for message currently displays offset of the message.
This could be enhanced to display a label using a provided expression (per topic).

For example, given the following message structure:

{
     firstName: "",
     lastName: "",
     employmentHistory: [{
         role: "",
         company: ""
     }, {
         role: "",
         company: ""
     }]
}

To use the the "role" of first employment item as message label, one could provide the following expression: employmentHistory[0].role

Message Dialog keeps popping up when a new message is received

Steps to reproduce:

  1. Click on a message to open message dialog for the first time
  2. Close the message dialog by press ESC
  3. From now, when a new message is received, the message dialog appears without user clicking on a message.

Expected behaviour: message dialog should only appear when user clicks on a message.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.