Git Product home page Git Product logo

kafka-map's Introduction

kafka map

English | 简体中文

Add wings to programming and install navigation to kafka

Introduction

kafka-map is a kafka visualization tool developed using Java17 and React.

Supported features:

  • Multi-cluster management.
  • Cluster status monitoring (number of partitions, number of replicas, storage size, offset).
  • Topic create, delete, expansion (delete needs to configure delete.topic.enable = true).
  • Broker status monitoring.
  • Consumer group view and delete.
  • Reset offset.
  • Topic data view and search (Support String and json display).
  • Send message to Topic
  • Delay message (supports 18 levels of delayed messages).

Screenshot

Import cluster

添加集群

Clusters

集群管理

Brokers

broker

Topics

主题管理

Consumer Groups

消费组

Consumer Group Subscription

消费组详情

Topic Partition

topic详情——分区

Topic Brokers

topic详情——broker

Topic Consumer Groups

topic详情——消费组

Topic Consumer Groups Reset Offset

topic详情——消费组重置offset

Topic Configs

topic详情——配置信息

Produce Message

消费消息

Consume Message

消费消息

Delay Message

延迟消息

Disclaimer

Developers wishing to use kafka-map within a corporate network are advised to seek approval from their administrators or management before using the tool. By downloading, using, or distributing kafka-map, you agree to the LICENSE terms & conditions. No warranty or liability is provided.

Required

  • Java17 or higher
  • Apache Kafka 1.1.0 or higher

install by docker

end

Param Description
DEFAULT_USERNAME Initial login username
DEFAULT_PASSWORD Initial login password
docker run -d \
    -p 8080:8080 \
    -v /opt/kafka-map/data:/usr/local/kafka-map/data \
    -e DEFAULT_USERNAME=admin \
    -e DEFAULT_PASSWORD=admin \
    --name kafka-map \
    --restart always dushixiang/kafka-map:latest

install by native

download

wget https://github.com/dushixiang/kafka-map/releases/latest/download/kafka-map.tgz

unzip

tar -zxvf kafka-map.tgz -C /usr/local/

Running in the foreground

# kafka-map dir
cd /usr/local/kafka-map
# Modify the configuration according to your needs
vi application.yml
# run
java -jar kafka-map.jar 

Running in System service

cat <<EOF >> /etc/systemd/system/kafka-map.service
[Unit]
Description=kafka map service
After=network.target

[Service]
WorkingDirectory=/usr/local/kafka-map
ExecStart=/usr/bin/java -jar /usr/local/kafka-map/kafka-map.jar
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

Reload system service && set boot auto-start && start service && view status

systemctl daemon-reload
systemctl enable kafka-map
systemctl start kafka-map
systemctl status kafka-map

Usage

Then use the browser to open the server's port 8080 to access.

FAQ

What if I don't want to use port 8080?

Add --server.port=1234 to the startup command to modify the port to 1234.

# example
java -jar kafka-map.jar --server.port=1234

kafka-map's People

Contributors

daodol avatar dushixiang avatar libra146 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

kafka-map's Issues

删除集群超时

我在服务器上通过docker搭建的kafka-map,能够添加集群,但是想delete的时候无法顺利操作
删除错误-censor

一些功能上的建议

往kafka发送数据之后,主题详情页面需要手动刷新更新,希望能够得到响应后自动刷新。

消费组管理现在查看报错?

Error: Request failed with status code 500
at e.exports (createError.js:16:15)
at e.exports (cookies.js:17:12)
at XMLHttpRequest.y (xhr.js:54:7)

多选topic

编辑topic时候是否可以多选?谢谢

image

副本数显示异常

副本数应该显示是1,实际显示的是6

具体见截图,图一为web页面显示内容。图二是通过kafka的命令行展示内容
web页面:
image

命令行显示:
image

另外看了下英文的web页面,副本列对应的英文是ReplicationFactor,那这个地方显示的应该是 1 才对
image

flink消费消息topic不显示消费组

用flink消费kafka消息,查看topic消费组,列表展示为空。进入Kafka容器,用Kafka自带脚本查看消费组是能查看到,并可以查看到消费组消费消息详情。./kafka-consumer-groups.sh --bootstrap-server 192.168.255.151:9092 --list。请问下是什么问题
image
image

程序运行问题

我看着文档运行不起来,需要数据库之类的额外配置吗。?

1.0.0版本启动报错

我是jdk1.8,这是要求更高版本?

Exception in thread "main" java.lang.UnsupportedClassVersionError: cn/typesafe/km/KafkaMapApplication has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(Unknown Source)
at java.security.SecureClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.access$100(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:151)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:46)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:107)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)

连接报错

使用docker运行后,连接报错,信息如下:
server 127.0.0.1:9092 can't connected.
请问是为什么

求助

只需要kafka延时功能,只需要用到哪些文件就可以了?能指点一下不。

权限控制

1、建议支持创建多个用户。每个用户管理不同的kafka或kafka集群。用户可以导入自己的kafka集群,也可管理员分配已有的kafka集群
2、通过界面可修改kafka config的值

shell command

in readme:

docker run -d \
    -p 8080:8080 \
    -v /opt/kafka-map/data:/usr/local/kafka-map/data \
    -e DEFAULT_USERNAME=admin
    -e DEFAULT_PASSWORD=admin
    --name kafka-map \
    --restart always dushixiang/kafka-map:latest

add "" :

docker run -d \
    -p 8080:8080 \
    -v /opt/kafka-map/data:/usr/local/kafka-map/data \
    -e DEFAULT_USERNAME=admin \
    -e DEFAULT_PASSWORD=admin \
    --name kafka-map \
    --restart always dushixiang/kafka-map:latest

建议添加的功能

首先感谢开发!
实际使用中还有以下几个需求感觉比较重要,因此提下建议哈

  • 本项目在部署的时候没有HTTP认证,如果部署到公网上,比较危险。(当然了,谁没事不会闲着蛋疼把这个部署到公网上,但是我为了方便把家里的测试环境直接部署到ipv6公网了(^_^),里面有测试的数据,还是挺危险的。)
  • 能够实现类似kafka-manager的对单个topic的配置进行修改

如何隐藏一些前端功能

前端比如我想隐藏topic的删除和alter 分区的功能,在哪里找。没源码里我找到,能告诉一下么?

context path设置问题

application.yml里设置了context path,但是首页的静态文件依然去根下找,这个要如何配置呢?
image

访问节点超时

刚添加节点时是OK的,过了十几分钟进去查看就超时了;这个时候重新添加一下又OK了,但是过了一会就又会超时。我的kafka集群是用docker部署的和kafkaMap在一个网络里面
微信截图_20220301140501
微信截图_20220301140539

无法查看消费组

在首页点击集群对应的消费组,触发访问接口为/consumerGroups?pageIndex=1&pageSize=10&clusterId=
提示访问kafka broker节点超时,接口返回504
image

添加了两个集群都是一样的报错,分别是阿里云kafka产品和自建的kafka集群。
使用docker启动,查看日志无异常输出

无法访问

image

image
访问不到,是不是我的操作有问题呢?

kakfa减少broken数量后查看topic接口报500

创建kafka时设置的broken数量为8个,删减3后,查看topic,报错,请问下是什么原因。其他服务器通过IP+端口,用Kafka自带命令./kafka-topics.sh,可以查看到topci
image
image

源碼啓動的时候hibernate.dialect报错

源碼啓動的时候hibernate.dialect报错
org.hibernate.HibernateException: Unable to determine Dialect to use [name=SQLite, majorVersion=3]; user must register resolver or explicitly set 'hibernate.dialect'
at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.determineDialect(DialectFactoryImpl.java:109)
at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.buildDialect(DialectFactoryImpl.java:54)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:97)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:101)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
at org.hibernate.id.factory.internal.DefaultIdentifierGeneratorFactory.injectServices(DefaultIdentifierGeneratorFactory.java:152)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.injectDependencies(AbstractServiceRegistryImpl.java:286)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:243)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.(InFlightMetadataCollectorImpl.java:176)
at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:127)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.metadata(EntityManagerFactoryBuilderImpl.java:1224)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:1255)
at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:58)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:365)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:409)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:396)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:341)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1845)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1782)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:602)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:524)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1154)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:908)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:144)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:769)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:761)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:426)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:326)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1313)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1302)
at cn.typesafe.km.KafkaMapApplication.main(KafkaMapApplication.java:24)

2022-05-19 22:40:14.907 ERROR [ main] j.LocalContainerEntityManagerFactoryBean : Failed to initialize JPA EntityManagerFactory: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
2022-05-19 22:40:14.908 WARN [ main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Invocation of init method failed; nested exception is org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
2022-05-19 22:40:14.908 INFO [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2022-05-19 22:40:14.911 INFO [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
2022-05-19 22:40:14.914 INFO [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2022-05-19 22:40:14.926 INFO [ main] ConditionEvaluationReportLoggingListener :

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-05-19 22:40:14.950 ERROR [ main] o.s.boot.SpringApplication : Application run failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Invocation of init method failed; nested exception is org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1786)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:602)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:524)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1154)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:908)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:144)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:769)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:761)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:426)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:326)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1313)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1302)
at cn.typesafe.km.KafkaMapApplication.main(KafkaMapApplication.java:24)
Caused by: org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:275)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
at org.hibernate.id.factory.internal.DefaultIdentifierGeneratorFactory.injectServices(DefaultIdentifierGeneratorFactory.java:152)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.injectDependencies(AbstractServiceRegistryImpl.java:286)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:243)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.(InFlightMetadataCollectorImpl.java:176)
at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:127)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.metadata(EntityManagerFactoryBuilderImpl.java:1224)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:1255)
at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:58)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:365)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:409)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:396)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:341)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1845)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1782)
... 17 common frames omitted
Caused by: org.hibernate.HibernateException: Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set
at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.determineDialect(DialectFactoryImpl.java:100)
at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.buildDialect(DialectFactoryImpl.java:54)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:137)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:101)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263)
... 34 common frames omitted

kafka连接超时

我在服务器上能正常生产消费,在kafka-map上就连接不上,日志里只有第一次启动的日志,不知道到哪里找问题

KAFKA 3.2.3 不支持么?

使用的KAFKA 3.2.3, 添加集群一直提示访问kafka broker节点超时,服务器到kafka的网络是正常的。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.