Git Product home page Git Product logo

yaks's Introduction

YakMQ

Stories in Ready

This is intended to be an experimental message bus with a defined consistency model, albeit more inspired by Kafka than RabbitMQ.

I'm eventually aiming for a Causal+ consistency model (see the ChainReaction paper), but it seems wise to start out with a simpler model, following the original Chain replication paper.

Inspiration:

yaks's People

Contributors

cstorey avatar waffle-iron avatar

Watchers

 avatar James Cloos avatar  avatar

yaks's Issues

Sharding?

Need to investigate how this will interact with causality tracking.

Join as tail of queue

At this point, dumping all state and reloading will do. Fancy stuff like Merkle trees comes later.

Configuration file

Assuming things will be at fixed locations in argv works okay for now, but it'd be more convenient to have an actual config.

Async cancellation of subscribers

With a subscription, the client thread ends up blocking in the store code, and hence there's no way for us to detect that a client has disconnected. This means that when there are no more elements for that stream (as often in tests), then that thread will wait infinitely, never getting any data, nor realising that the socket is no more.

Rejoin as non-tail

Chainreaction suggests use of a standard consistent hash ring for discovery and mapping locations, so a new node may join a chain in the middle or as a head node.

Remove truncate op

Rather than invalidating iterators, we can just render the whole point moot for now. This will require randomizing space names in tests, mind.

Allow asynchronous publishers/consumers

Currently, we require that an event is completely acked before returning to the publisher, or require that a connection go into consumer mode entirely. This limits both throughput and flexibility.

Could chains be effectively trees?

Bit of a wild idea, this one. But having trees rooted at the head node could be good when having a discovery system that doesn't give us serializability. We'd need to figure out how chain nodes could re-organise themselves.

Auto-reconnect to peers

If a head / middle node loses the connection to it's down stream peer, we basically halt and catch fire. All further requests fail with a Protocol Error, as the connection is now closed.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.