msgpack-rpc / msgpack-rpc-python Goto Github PK
View Code? Open in Web Editor NEWMessagePack RPC implementation for Python
MessagePack RPC implementation for Python
Are there any plans to move towards a more recent version of Tornado?
Airsim uses msgpack-rpc-python (related issue), which is not compatible with Tornado >= v5 (deliberate decision here).
Unfortunately, the jupyter extension of VS Code needs Tornado >= v5. Are there any plans to update tornado to a more recent version? (Or are there not enough maintainers?)
Thx!
Constructor of msgpackrpc.Session
class has an optional argument loop=None
. When loop
is not specified, new Loop will be automatically created in the constructor.
However, as automatically-created Loops are not closed automatically, it uses more file descriptors than expected. This becomes an issue for clients that connects/disconnects frequently.
I think automatically-created Loops should be closed (i.e., session._loop._ioloop.close()
) when its associated Session
is closed.
I have two ideas:
Loop
instance is not specified by user (i.e., automatically created in Session
constructor), close the Loop when session.close()
method is called.close_loop=True
to the Session
constructor argument. Close the Loop only if close_loop
is set to True when session.close()
method is called. This behavior is similar to closefd option of io.FileIO.In addition, it is better to add close()
method to msgpackrpc.Loop
class, to allow users to close the internal tornado IOLoop.
HI,
when ClientSocket
was closed, I think the Session._request_table
were no longer necessary.
future
under current strategy will be wait until timeout, but if Client
set timeout None
, when socket closed, there will be endless wait.
def on_close(self, sock):
# Avoid calling self.on_connect_failed after self.close called.
if self._closed:
return
# if sock in self._sockets:
# self._sockets.remove(sock)
# self._connecting = 0
# else:
# Tornado does not have on_connect_failed event.
self.on_connect_failed(sock)
What do you think?
Tornado 3 and 4 works ok. Tornado 5 server.py fails with:
Traceback (most recent call last):
File "server.py", line 8, in <module>
server.listen(msgpackrpc.Address("localhost", 18800))
File "/usr/local/lib/python2.7/dist-packages/msgpackrpc/server.py", line 24, in listen
listener.listen(self)
File "/usr/local/lib/python2.7/dist-packages/msgpackrpc/transport/tcp.py", line 170, in listen
self._mp_server = MessagePackServer(self, io_loop=self._server._loop._ioloop, encodings=self._encodings)
File "/usr/local/lib/python2.7/dist-packages/msgpackrpc/transport/tcp.py", line 157, in __init__
tcpserver.TCPServer.__init__(self, io_loop=io_loop)
TypeError: __init__() got an unexpected keyword argument 'io_loop'
Unfortunately pip defaults to version 5. Suggest we get pip to install 4.5.3 until version 5 is supported.
How much concurrency support?
When you use block modules, then rpc server meet block?
Hello,
I am having trouble with Microsoft AirSim, which uses msgpckrpc as its communication backend. AirSim does this in a python object:
self.client = msgpackrpc.Client(msgpackrpc.Address(ip, port), timeout = timeout_value, pack_encoding = 'utf-8', unpack_encoding = 'utf-8')
Let's say at some point AirSim freezes and I get a TimeoutError from msgpckrpc (which I do, see microsoft/AirSim-NeurIPS2019-Drone-Racing#104 ). So I want to instantiate a new version of this python object (with another msgpackrpc.Address(ip, port) ).
However, doing this apparently simple thing is actually a nightmare and fails because msgpackrpc/tornado don't get properly reinitialized in the process. How can I properly reinitialize msgpckrpc please ?
Hi. How can I extend the default timeout? In my app the rpc call invokes some process that performs a job that takes several seconds.How can I modify the timeout time so that this errors won't throw?
Thanks.
When the server is heavily in use I occasionally get this error:
'NoneType' object has no attribute 'send_message'
Any ideas?
full stack trace:
File "requests_service/client.py", line 30, in _client_call
return client.call(method, args, kwargs)
File "msgpackrpc/session.py", line 41, in call
return self.send_request(method, args).get()
File "msgpackrpc/session.py", line 51, in send_request
self._transport.send_message([message.REQUEST, msgid, method, args])
AttributeError: 'NoneType' object has no attribute 'send_message'
Currently a non-blocking server would look like this:
import multiprocessing, time, threading
import msgpackrpc
# Why is this core thing buried deep in a random module?
from msgpackrpc.server import AsyncResult
def run_threaded(result, x, y):
time.sleep(1)
result.set_result(x + y)
class SumServer(object):
def sum(self, x, y):
result = AsyncResult()
thread = threading.Thread(target=run_threaded, args=(result, x, y))
thread.start()
return result
def server():
server = msgpackrpc.Server(SumServer())
server.listen(msgpackrpc.Address("localhost", 4000))
server.start()
def client():
client = msgpackrpc.Client(msgpackrpc.Address("localhost", 4000))
future = client.call_async('sum', 1, 2)
print(future.get())
def main():
proc_server = multiprocessing.Process(target=server)
proc_server.start()
time.sleep(1)
try:
client()
finally:
proc_server.terminate()
if __name__ == '__main__':
main()
In my opinion, this is a design problem: AsyncResult
, which is essential to writing asynchronous servers, is contained in a non-top-level module. Is it possible to move it to the top level, for example, by importing from __init__.py
?
My use-case:
If I connect to server, I want to know that there is something wrong immediately.
However when I call my client, I want to wait longer, as my calculation in RPC server logic takes a while.
The best would be to be able to provide timeout
kwarg to call()
method, so that a developer can specify separate timeouts for each call.
Question is, how timeout
parameter in call
method should be set as a default. 2 ideas:
None
. In that case timeout which was used for creating Client
is used for call
as it is done right now - backward compatibility provided.I may try to develop something and PR, but let me know first what do you think plz.
maybe i'm missing something, but when using the client, tornado shouldn't be needed right?
right now msgpackrpc/init.py imports Loop which imports the tornado loop, which is a little annoying.
I can't maintain this project now because I don't have enough time.
Does anyone interested in working on this project?
Hi! Looks like the libs works ok with https://github.com/rpclib/rpclib
I tried https://github.com/0rpc/zerorpc-python which didn't work.
But 'localhost' string must be replaces with the actual IP,otherwise it cannot establish a connection.
I couldn't get client.notify()
to work. It looks like the call is going nowhere. The following code prints nothing:
import multiprocessing, time
import msgpackrpc
class SumServer(object):
def notification(self, s):
print(s)
def server():
server = msgpackrpc.Server(SumServer())
server.listen(msgpackrpc.Address("localhost", 4000))
server.start()
def client():
client = msgpackrpc.Client(msgpackrpc.Address("localhost", 4000))
client.notify('notification', 'boo')
def main():
proc_server = multiprocessing.Process(target=server)
proc_server.start()
time.sleep(0.5)
try:
client()
finally:
proc_server.terminate()
if __name__ == '__main__':
main()
Are there any examples of using .notify()
?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.