Git Product home page Git Product logo

rabbitmq-lvc-plugin's Introduction

Last value caching exchange

(NB: Simon keeps a fork of this which is more likely to be up-to-date with RabbitMQ releases, at https://github.com/simonmacmullen/rabbitmq-lvc-plugin)

This is a pretty simple implementation of a last value cache using RabbitMQ's pluggable exchange types feature.

The last value cache is intended to solve problems like the following: say I am using messaging to send notifications of some changing values to clients; now, when a new client connects, it won't know the value until it changes.

The last value exchange acts like a direct exchange (binding keys are compared for equality with routing keys); but, it also keeps track of the last value that was published with each routing key, and when a queue is bound, it automatically enqueues the last value for the binding key.

How to build it

Set up rabbitmq-public-umbrella, as per the instructions at http://www.rabbitmq.com/plugin-development.html:

$ hg clone http://hg.rabbitmq.com/rabbitmq-public-umbrella
$ cd rabbitmq-public-umbrella ; make checkout ; make

Then get the LVC plugin and symlink it into plugins:

$ git clone git://github.com/squaremo/rabbitmq-lvc-plugin.git
$ (cd rabbitmq-lvc-plugin ; make)
$ mkdir -p rabbitmq-server/plugins
$ cd rabbitmq-server/plugins
$ ln -s ../../rabbitmq-lvc-plugin ./

and finally, run the server: $ cd .. $ make run

In the startup banner you should see a line something like

starting rabbit_exchange_type_lvc                 ...done

To use the LVC exchange, with e.g., py-amqp:

import amqplib.client_0_8 as amqp
ch = amqp.Connection().channel()
ch.exchange_declare("lvc", type="x-lvc")
ch.basic_publish(amqp.Message("value"),
                 exchange="lvc", routing_key="rabbit")
ch.queue_declare("q")
ch.queue_bind("q", "lvc", "rabbit")
print ch.basic_get("q").body

Limitations

"Recent value cache"

AMQP is inherently racey. It is quite possible to see different last-values but the same subsequent message stream, from different clients.

This won't matter if you simply want to have a value to show until you get an update. If it does matter, consider e.g. using sequence IDs so you can notice out-of-order messages.

There's also a race in the pluggable exchanges hook, so that clients can "see" the binding before the hook has been run; for the LVC, this means that there's a possiblity that messages will get queued before the last value. For this reason, I'm thinking of tagging the last value messages so that clients can fast-forward to it, or ignore it, if necessary.

Values v. deltas

One question that springs to mind when considering last value caches is "what if I'm sending deltas rather than the whole value?". Thre LVC exchange doesn't address this use case, but you could do it by using two exchanges and posting full values to the LVC (from the originating process -- presumably you'd be using deltas to save on downstream bandwidth).

Direct exchanges only

The semantics of another kind of value-caching exchange (other than fanout) aren't obvious. To choose one option though, say a newly-bound queue was to be given all values that match its binding key -- this would require every supported exchange type to supply a reverse routing match procedure.

rabbitmq-lvc-plugin's People

Contributors

mmorrisontx avatar squaremo avatar tonyg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

rabbitmq-lvc-plugin's Issues

RabbitMQ 2.7.0 - rabbit_basic:delivery - unknown....

RabbitMQ 2.7.0

rabbitmq-server-2.7.0/src/rabbit_basic.erl +95 - delivery/4
rabbit_exchange_type_lvc.erl +76 delivery/5

  • Q, rabbit_basic:delivery(false, false, none, Msg, undefined))
  • Q, rabbit_basic:delivery(false, false, Msg, undefined))

plugin feature request or maybe help ...

It is not issue! It is feature request.

Can we use topic exchange too. Currently plugin uses direct but for example i need to get last values based on certain routing keys like test.*

Is it possible to implement this extra feature:

When a message arrives in queue - check mnesia DB if previous message is same like current message. If "yes" then do not notify and do not flood subscribers with same data. Can you give me some directions .. or code.., .. or advice... ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.