andrewvc / em-zeromq Goto Github PK

just some ideas off the top of my head for discussion

1. Replace ffi-rzmq with officially supported zmq
2. Define façades for ZMQ::Context, ZMQ::Socket
3. EM::ZeroMQ::Connection should inherit from EM::Connection - this is less confusing to regular eventmachine users
4. Each class should only be responsible for one thing only, akin to the single responsibility principle

@schmurfy what do you think ?

Do have a look at my hack and see if it makes sense, it would serve to explain my expectations above.

I have one question that is poking my head for long time.
Given the code below, comparing with your usage warning GC, is supposed that gc variable should be collected when the GC run. right ?

require 'eventmachine'
require 'objspace'

EM.run do 
  gc = Object.new
  puts gc.object_id
  ObjectSpace.define_finalizer(gc, proc {|id| puts "Collected #{id}" })
  EM.add_timer(1) {  puts "Starting GC"; GC.start; }
end

The curious part is that the GC will never reclaim gc object, I don't know why ?
Do you know ?
Before run the snippet above, I thought that the gc will be collected without any doubts like the Usage Warning

The snippet below, the _Object.new_ without root will be reclaimed properly.

require 'eventmachine'
require 'objspace'

EM.run do 
  ObjectSpace.define_finalizer(Object.new, proc {|id| puts "Collected #{id}" })
  EM.add_timer(1) {  puts "Starting GC"; GC.start; }
end

Released yesterday :)

I am trying to implement a request-response pattern, but I can't get the response socket to send a message back to the request socket in its handler. I have written some very simple code to test it:

em_req.rb

require 'em-zeromq'

client_id = ARGV[0] ? ARGV[0].to_i : 1
message = ARGV[1] || "Foo"

Thread.abort_on_exception = true

class ReqHandler
  attr_reader :received

  def on_readable(socket, messages)
    messages.each do |m|
      puts "Received message from server: #{m.copy_out_string}"
    end
  end
end

trap('INT') do
  EM.stop
end

ctx = EM::ZeroMQ::Context.new(1)

EM.run do
  conn = ctx.connect(ZMQ::REQ, 'tcp://127.0.0.1:9000', ReqHandler.new, identity: "client#{client_id}")
  conn.socket.send_string(message)
end

em_rep.rb

require 'em-zeromq'

Thread.abort_on_exception = true

class ResponseHandler
  attr_reader :received

  def on_readable(socket, messages)
    message = messages.first.copy_out_string
    puts "Received message from client: #{message}"

    socket.send_msg("re: #{message}")
  end
end

trap('INT') do
  EM.stop
end

ctx = EM::ZeroMQ::Context.new(1)

EM.run do
  socket = ctx.bind(ZMQ::REP, 'tcp://127.0.0.1:9000', ResponseHandler.new)
end

I have written similar code using the push-pull pattern and got that to work, but for request-response all I get is the response code printing "Received message from client1: Foo" but the reply never reaches the request code. I suspect it has to do with writing to the socket in the response code's handler, because the same thing happens when I use a request-router pattern. The only time it works is when I send a message from the server without sending a message from the client first (using push-pull).

Any ideas about what might be causing this? I know andrewvc isn't maintaining this gem anymore, but I thought I would post this issue anyway in the hopes of other developers with similar experiences seeing this.

I am using em-zeromq 0.2.2 on Ruby 1.9.2p290.

https://github.com/andrewvc/em-zeromq/blob/master/lib/em-zeromq/socket.rb#L81

I have a DEALER-ROUTER setup. Inbound messages (from the dealer) being processed by the router usually involve sending messages to one or more dealer peers. When the router calls #send_msg, that method in turn calls #notify_readable which checks if there are new messages on the socket and if so fires #on_readable. This causes the #on_readable calls to stack up (in a LIFO fashion) and the outer ones won't be completed until the inbound message flow ceases and it can wind back down the stack.

Basically, if I have a fairly high rate of incoming messages on the router and those trigger multiple outbound messages, only the first outbound message gets sent until the rate decreases and it has time to complete processing of any pending messages.

Is this a bug? Or is it by design? If so, what's the reasoning behind it and how can the driver be configured to avoid this issue when using the above pattern?

I've set up a req/rep pair (code below) but it's not working as I would expect it. If I run the client without a server then I get a response; I would expect to only get a response if the server sends one!

The interesting thing is that if the server is running the client will print "Got return message", the reactor will shut down and the program will correctly terminate. If the server is not running the client will print "Got return message", and then it will wait (I assume it's waiting for the return message). If I subsequently start the server then the client will properly terminate.

############## Client

require 'em-zeromq'

class Handler
def on_writable(socket)
puts "Got return message"
socket.unbind
EM.stop
end
end

EM.run do
context = EM::ZeroMQ::Context.new(1)

socket = context.connect(ZMQ::REQ, 'tcp://127.0.0.1:2091', Handler.new)
puts "sending ready message"
socket.send_msg("ready?")
end

############## Server

require 'em-zeromq'

class Handler
def on_readable(socket, messages)
socket.send_msg "ready"
end
end

EM.run {
handler = nil
context = EM::ZeroMQ::Context.new(1)

context.bind(ZMQ::REP, 'tcp://127.0.0.1:2091', Handler.new)
}

Hello!

Based on your example: http://pastie.org/private/fbfwo6lzgvhcnkhblkcdzq

  => ........

BUT: http://pastie.org/private/42m4yamynxa0krczhbrgtq (with pull_socket.connect('ipc:///tmp/a') )

  => ._._._._._._._._._._._._

Why?

This is not my only problem, but the main now :)

There are two failing tests in the master branch, is it normal ?

Unsure if this is being maintained, but unless manually enabled the Xsub socket won't process data. e.g.

xsub = zmq.socket(ZMQ::XSUB)
xsub.identity = "#{CLIENT_ID}-sub"
xsub.connect(SERVER_PUB_URL)
xsub.notify_readable = true

NoMethodError: undefined method `errno' for ZMQ:Module
getsockopt_with_raise at /home/marvel/.rvm/gems/jruby-1.6.5/gems/em-zeromq-0.2.2/lib/em-zeromq/rzmq_compat.rb:25

This seems kind of strange to me. the zeromq gem say's to check ZMQ.errno like you are doing, but errno is defined in ZMQ::Util and only included in specific zeromq classes, so it's not accessible as ZMQ.errno from anywhere, you have to specifically include ZMQ::Util within the namespace you want to call it from.

Looks like the quick fix is to just call ZMQ::Util.errno

Hi Andrew,

Could you please update the em-zeromq gem on RubyGems.org?

When running example/simple.rb using em-zeromq 0.2.1 from RubyGems.org an exception is thrown because ffi-rzmq is missing an argument. The problem is gone when using github/master (although the version is still 0.2.1?).

A workaround is bundle install to get the github version. However, when running bundle exec is giving me a segmentation fault. So that doen't really solve the problem.

My configuration:

• em-zeromq (0.2.1)
• ffi-rzmq (0.9.0)
• zmq (2.1.4)
• ffi-rzmq (0.9.0)
• ruby 1.9.2p180 (2011-02-18 revision 30909) [i686-linux]
• rvm 1.6.20 by Wayne E. Seguin ([email protected])

The exception thrown is

```/home/sebastian/.rvm/gems/ruby-1.9.2-p180/gems/ffi-rzmq-0.9.0/lib/ffi-rzmq/socket.rb:387:ingetsockopt': wrong number of arguments (1 for 2) (ArgumentError) from /home/sebastian/.rvm/gems/ruby-1.9.2-p180/gems/em-zeromq-0.2.1/lib/em-zeromq/context.rb:48:in create'
from /home/sebastian/.rvm/gems/ruby-1.9.2-p180/gems/em-zeromq-0.2.1/lib/em-zeromq/context.rb:22:in`bind'
from server.rb:19:in `block in

I see it looks like it's fixed in master. Can we get a new release cut soon? em-zeromq completely breaks the ability to use ffi-rzmq directly, without serious monkey patching, because of how it changes what recv returns.

Chris

This is not really an issue, more an open question :)

I started using zmq on a work project and for that I started straight with zeromq 3.1 which is in beta state right now, I have a local branch where I first tried to keep compatibility with other zmq releases but in the end I just dropped it which cleaned some edges.
(in 3.1 some function names changed in the C api, some constants disappeared, other appeared)

My question is what strategy should we use the gem ?
We can do like ffi-rzmq and support all zeromq versions or we can decide that newer releases support the latest zeromq version (out of beta of course), I am personnaly in favor of the last choice given the fact that old versions of the gem will still remain if someone want to use another zeromq version, we just need to document it in the README.

http://api.zeromq.org/2-1:zmq-socket defines ROUTER to be synonymous with XREP and DEALER to be synonymous with XREQ, however in em-zeromq:

ZMQ::ROUTER == ZMQ::XREQ and ZMQ::DEALER == ZMQ::XREP

If i need to set hwm, i need to do it before bind or connect happens. The Context#bind or Context#connect methods make this impossible at the moment.

http://api.zeromq.org/2-1:zmq-setsockopt

int zmq_setsockopt (void *socket, int option_name, const void *option_value, size_t option_len);

Caution: All options, with the exception of ZMQ_SUBSCRIBE, ZMQ_UNSUBSCRIBE and ZMQ_LINGER, only take effect for subsequent socket bind/connects.