Git Product home page Git Product logo

libzmq-rs's Introduction

⚠️ I wouldn't recommand using ZeroMQ or any ZeroMq bindings library (including libzmq-rs), unless you absolutely have to. If you do, then libzmq-rs might fit your use case since it basically makes ZeroMQ not a complete footgun. However, just because this library hides the unmaintainable mess that is ZeroMQ, doesn't mean the mess does not exist. See this comment for more context.

Apache 2.0 licensed MIT licensed

libzmq-rs

A strict subset of ØMQ with an ergonomic API.

[dependencies]
libzmq = "0.2"

Dead Simple Sample

use libzmq::{prelude::*, *};
use std::convert::TryInto;

// Use a system assigned port.
let addr: TcpAddr = "127.0.0.1:*".try_into()?;

let server = ServerBuilder::new()
    .bind(addr)
    .build()?;

// Retrieve the addr that was assigned.
let bound = server.last_endpoint()?;

let client = ClientBuilder::new()
    .connect(bound)
    .build()?;

// Send a string request.
client.send("tell me something")?;

// Receive the client request.
let msg = server.recv_msg()?;
let id = msg.routing_id().unwrap();

// Reply to the client.
server.route("it takes 224 bits to store a i32 in java", id)?;

// We can reply as much as we want.
server.route("also don't talk to me", id)?;

// Retreive the first reply.
let mut msg = client.recv_msg()?;
// And the second.
client.recv(&mut msg)?;

Installation

This crate builds and generates bindings from source. This means that you do not need to install libzmq. However building from source requires:

General Goals

  • Conform to these API guidelines.
  • Provide an ergonomic API
  • Prevent footguns (which are plentifull in libzmq)
  • Minimize the learning curve
  • Don't sacrifice any performance
  • Extensively document

To do so we will only use a subset of libzmq. If you'd rather have a complete port, check out rust-zmq.

Frequently Asked Questions

See the FAQ.

Acknowledgements

License

This project is licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in libzmq by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

libzmq-rs's People

Contributors

akhilles avatar dmweis avatar jean-airoldie avatar rotty avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

libzmq-rs's Issues

Implement bencharks

We should add some criterion benchmark to test performance per socket type. The benchmarks results should be in throughput per second and should be displayed in a file somewhere.

Inline everything

We should consider inlining the hot path based on benchmarks. Parts to consider:

  • send calls
  • recv calls
  • Msg creation / convertion
  • Poller method calls

Cannot retreive authentication result

When a connection succeeds the socket is no longer in mute state and can queue outgoing messages via send. The problem is if the authentication failed the user will never know it: he has effectively no way of knowing if his credentials were misconfigured. Here is the issue I filled zeromq/libzmq#3505.

Implement PgmAddr parsing

Currently PgmAddr is only a wrapper around String. Using the same interface (if possible), proper parsing (FromStr trait) and appropriate methods should be implemented.

See zmq_pgm for reference.

Support uniquely generated inproc adresses

This would be the inproc equivalent of a system assigned port.

Currently to do unit testing in the API we create a ton of Ctx so that we don't name clash with other unit tests. We could instead add a method to InproAddr that generates a unique inproc address and use the global context. This would also improve documentation as a side effect since users won't be tempted tu created a Ctx unnecessarily from reading the doc.

Horrendous CURVE performance

Performance using CURVE is horrendous. Not sure if this is a recent regression.

When looking at the callgraph i get that most of the overhead is cause by:
69.15% 69.00% ZMQbg/IO/0 writer [.] crypto_onetimeauth
which is a fcn that should only occur when the handshake occurs. So something is wrong.

Add a pull request template

  • Add changes to CHANGELOG.md
  • Add doc & test
  • Commit message format (problem & solution?)
  • Cargo fmt & clippy
  • Etc.

Implement fuzzing

Since we are relying of c / c++ code we should fuzz the socket inputs / outputs. We should probably use something like afl. Then we should file issues with our findings (if any) to the libzmq project.

Feature gate bindgen

We could feature gate the bindgen generation. This would remove the dependency on clang. Only maintainers would need to enable this feature to update the bindings once in a while.

Add timer API

We should implement a timer API similar tokio timer. This would allow use to schedule timeouts and intervals using the libzmq event loop.

Here is the general design I had in mind.

  • Create a Server in a thread (global singleton?). This server will be in charge of keeping track of all the Interval and sending notification to the client.
  • Create a Client thread an connect it to the server.
  • Send a request for scheduling an Interval with a specific id to the server.
  • The sever will send reply messages with the specified id at the specified interval.
  • There can be many Client thread connected to the Server thread at the same time.

Channel feature

We should provide for some channel patterns that can send Box allocated arbitrary types for inter thread communication.

Panic on Trigger::from_bites

Since Poller now supports any type that implements AsRawFd, this mean we could be polling all sorts of things. There are platform specific event flags that are currently not supported by the bitflag which causes a crash at poll.rs:185. We should conditionally define these flags per platform.

Improve heartbeat API

Current heartbeat API as a lot of mutually dependent options. Or options that change meaning when together. We should improve the API.

Implement EpgmAddr parsing

Currently EpgmAddr is only a wrapper around String. Using the same interface (if possible), proper parsing (FromStr trait) and appropriate methods should be implemented.

See zmq_pgm for reference.

Support Authentication Proxying

There are two non-mutually exclusive possibilities:

  • Proxy the authentication at the zmq level by simply forwarding the requests to the remote server
  • Proxy the authentication at the application level by creating a authentication protocol. This would allow implementation of authenticating services in userland.

Socket event logger

Use the event monitoring API from #23 to create a socket event logger. This would fix #19 by emitting error messages when an authentication fails.

Things to consider:

  • Use a global singleton logger vs. create a logger

Send calls without a AuthHandler hang

Since we enable ZAP by default, when no zap handler is running, send calls will hang forever.
There are 2 possible solutions I can think off:

  • Create a spawn a ZAP handler for every context.
  • Check for ZAP related events before sending to detect when a auth attempt failed. We could make bind and connect calls blocking a check whether the ZAP handshake was sucessfull.

Support event monitoring

Support the zmq_socket_monitored_versioned API to retrieve events for sockets.

My inital idea is to create a unique monitor_addr for each socket. Then we would call zmq_monitor_versioned with this addr to create a ZMQ_PUB monitor socket. The we could create a SocketMonitor which would leverage a ZMQ_SUB and subscribe to the appropriate events. It would then unpack the data into a SocketEvent enum and feed it to a user provided closure. If there is no significant overhead, we should enable that by default.

We could use the same API to create a event logger.

This is required to fix #19.

Remove ZMQ_LINGER

ZMQ_LINGER is one of those unexpected features of ZMQ that is not all that useful in production. Real applications usually handshake and disconnect with ZMQ_LINGER=0.

We should consider setting ZMQ_LINGER=0 by default (or set the Ctx with no_linger=true by default). Maybe we should remove the set_linger option entirely.

Implement TcpAddr parsing

Currently TcpAddr is only a wrapper around String. Using the same interface (if possible), proper parsing (FromStr trait) and appropriate methods should be implemented.

Note that TcpAddr is not exactly the same as SocketAddr since it supports the binding of interfaces etc.

See zmq_tcp for reference.

Implement InprocAddr parsing

Currently InprocAddr is only a wrapper around String. Using the same interface (if possible), proper parsing (FromStr trait) and appropriate methods should be implemented.

See zmq_inproc for reference.

Add decent examples

Maybe a 3 level deep broker example to illustrate the routing_stack concept? We should use config files etc.

Remove Unlimited high water mark

The general rational is that unlimited high water mark is basically useless.

First of all its a footgun that might encourage users to uncap their memory usage which will eventually lead to using all physical memory. Secondly, one could emulate this behavior by passing i32::max_value().

Support Gather and Scather sockets

These are the thread safe equivalents of PUSH-PULL.

This are currently not documented in libzmq so we would have to reverse engineer how they work.

Use stricter types instead of i32

Instead of accepting a i32 and then later erroring out because the i32 was negative or zero, we should use stricter types. This would remove the type checking from the method.

Implement proper endpoint parsing

This is a tracking issue around the proper implementation of Endpoint.

Currently Endpoint only parses the transport and stores the address into a string without parsing. This is meant to future-proof our socket methods until proper parsing is developed.

These are the sub-issues to must be closed:

  • Implement TcpAddr #6
  • Implement IpcAddr #7
  • Implement InprocAddr #8
  • Implement PgmAddr #9
  • Implement EpgmAddr #10

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.