DISCLAIMER: The goal of this project is to mimic confluent/kafka-rest-proxy to offer a faster interface to topic data for applications like kafka-topics-ui, as kafka-rest-proxy troubles with larger message payloads or a large amount of messages on the topic. If you are looking for a complete rest-proxy for kafka, this is not the tool you are looking for.
- Run kafka-setup:
yarn run kafka:start - Open kafka topics ui:
http://localhost:8000
curl -X GET \
http://localhost:8082/topics \
-H 'cache-control: no-cache' \
-H 'content-type: application/json' \
-d '{
"name": "peter",
"format": "json",
"auto.offset.reset": "earliest"
}'curl -X GET \
http://localhost:8082/topics/peters-topic \
-H 'cache-control: no-cache' \
-H 'content-type: application/json' \
-d '{
"name": "peter",
"format": "json",
"auto.offset.reset": "earliest"
}'curl -X POST \
http://localhost:8082/consumers/peter \
-H 'cache-control: no-cache' \
-H 'content-type: application/json' \
-d '{
"name": "peter",
"format": "json",
"auto.offset.reset": "earliest"
}'curl -X POST \
http://localhost:8082/consumers/peter/instances/peter/subscription \
-H 'cache-control: no-cache' \
-H 'content-type: application/json' \
-d '{
"topics": ["peters-topic", "ks-fourth-4b63a506-4e46-4c78-a0b2-24922a1124db"]
}'curl -X GET \
http://localhost:8082/consumers/peter/instances/peter/records \
-H 'cache-control: no-cache' \
-H 'content-type: application/json'curl -X DELETE \
http://localhost:8082/consumers/peter/instances/peter \
-H 'cache-control: no-cache' \
-H 'content-type: application/json'If you want to customize any Kafka REST parameters, simply add them as environment variables, e.g.:
- To set the http.port parameter set the environment to
KAFKA_REST_HTTP_PORT: 8082. - To turn on debug mode for consumer and producer set
KAFKA_REST_DEBUG: "all". - To set the group.id only for consumer set the environment to
KAFKA_REST_CONSUMER_GROUP_ID: "kafka-rest-consumer-group".
The full list and descriptions of config parameters can be found here.
Check out our docker compose example:
---
version: "2"
services:
zookeeper:
image: wurstmeister/zookeeper:latest
ports:
- 2181:2181
kafka:
image: wurstmeister/kafka:0.10.2.1
ports:
- "9092:9092"
links:
- zookeeper
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 1
KAFKA_PORT: 9092
KAFKA_ADVERTISED_HOST_NAME: "kafka"
KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://kafka:9092"
KAFKA_LISTENERS: "PLAINTEXT://:9092"
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_CREATE_TOPICS: "test:1:1"
KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
kafka-rest:
image: nodefluent/kafka-rest
ports:
- 8082:8082
links:
- kafka
- zookeeper
depends_on:
- kafka
- zookeeper
environment:
KAFKA_REST_DEBUG: "all"
KAFKA_REST_HTTP_PORT: 8082
KAFKA_REST_CONSUMER_METADATA_BROKER_LIST: "kafka:9092"
KAFKA_REST_PRODUCER_METADATA_BROKER_LIST: "kafka:9092"
kafka-topics-ui:
image: landoop/kafka-topics-ui
ports:
- 8000:8000
links:
- kafka-rest
depends_on:
- kafka-rest
environment:
KAFKA_REST_PROXY_URL: "http://kafka-rest:8082"
PROXY: "true"
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent