turulix / memphis Goto Github PK

Docs - Twitter - YouTube

Memphis enables building modern queue-based applications that require large volumes of streamed and enriched data, modern protocols, zero ops, up to x9 faster development, up to x46 fewer costs, and significantly lower dev time for data-oriented developers and data engineers.

When your application requires a message broker or a queue,
Implementing one will require you to -

• Build a dead-letter queue, create observability, and a retry mechanism
• Build a scalable environment
• Create client wrappers
• Tag events to achieve multi-tenancy
• Enforce schemas and handle transformations
• Handle back pressure. Client or queue side
• Configure monitoring and real-time alerts
• Create a cloud-agnostic implementation
• Create config alignment between production to a dev environment
• Spent weeks and months learning the internals through archival documentation, ebooks, and courses
• Onboard your developers
  And the list continues...
  

Or, you can just use Memphis and focus your resources on tasks that matter 😎

Roadmap

Production-ready message broker in under 3 minutes
Easy-to-use UI, CLI, and SDKs
Data-level observability
Dead-Letter Queue with automatic message retransmit
Schemaverse - Embedded schema management for produced data (Protobuf/JSON/GraphQL/Avro)
Storage tiering
SDKs: Node.JS, Go, Python, Typescript, NestJS, REST, .NET, Kotlin
Kubernetes-native
Community driven

• Gastromatic - Synchronizing data using Memphis.dev
• KELA - Real-time cyber threats identification
• Handling millions of discord messages

Helm for Kubernetes☸

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && \
helm install my-memphis memphis/memphis --create-namespace --namespace memphis

Docker🐳 Compose

curl -s https://memphisdev.github.io/memphis-docker/docker-compose.yml -o docker-compose.yml && \
docker compose -f docker-compose.yml -p memphis up

Tutorials
Installation videos

To access Memphis using UI/CLI/SDK from localhost, run the below commands:

  - kubectl port-forward service/memphis 6666:6666 9000:9000 7770:7770 --namespace memphis > /dev/null &

For interacting with the broker via HTTP:

  - kubectl port-forward service/memphis-rest-gateway 4444:4444 --namespace memphis > /dev/null &

Dashboard/CLI: http://localhost:9000
Broker: localhost:6666 (Client Connections)
REST gateway: localhost:4444 (Data + Mgmt)

For Production Environments Please expose the UI, Cluster, and Control-plane via k8s ingress / load balancer / nodeport

Dashboard/CLI: http://localhost:9000
Broker: localhost:6666

• Async task management
• Real-time streaming pipelines
• Data ingestion
• Cloud Messaging
  • Services (microservices, service mesh)
  • Event/Data Streaming (observability, analytics, ML/AI)
• Queuing
• N:N communication patterns
• Ingest Grafana Loki logs at scale

We welcome you to our discord server with your questions, doubts and feedback.

If you see an error message or run into an issue, please create bug report. This effort is valued and it will help all Memphis{dev} users.

If you have an idea, or you think that we're missing a capability that would make development easier and more robust, please Submit feature request.

If an issue❗with similar feature request already exists, don't forget to leave a "+1". If you add some more information such as your thoughts and vision about the feature, your comments will be embraced warmly :)

Memphis.dev is an open-source project.
We are committed to a fully transparent development process and appreciate highly any contributions.
Whether you are helping us fix bugs, proposing new features, improving our documentation or spreading the word - we would love to have you as part of the Memphis.dev community.

Please refer to our Contribution Guidelines and Code of Conduct.

Thanks goes to these wonderful people ❤:

Memphis is open-sourced and operates under the "Memphis Business Source License 1.0" license Built out of Apache 2.0, the main difference between the licenses is: "You may make use of the Licensed Work (i) only as part of your own product or service, provided it is not a message broker or a message queue product or service; and (ii) provided that you do not use, provide, distribute, or make available the Licensed Work as a Service. A “Service” is a commercial offering, product, hosted, or managed service, that allows third parties (other than your own employees and contractors acting on your behalf) to access and/or use the Licensed Work or a substantial set of the features or functionality of the Licensed Work to third parties as a software-as-a-service, platform-as-a-service, infrastructure-as-a-service or other similar services that compete with Licensor products or services." Please check out License to read the full text.