thriftpy / gunicorn_thrift Goto Github PK
View Code? Open in Web Editor NEWThrift app and worker for gunicorn!
License: MIT License
Thrift app and worker for gunicorn!
License: MIT License
2016-04-21 15:07:26.090 err gunicorn.biz.shopping.error: [26817] ('10.0.40.20', 36547): ImportError('No module named events',)
Traceback (most recent call last):
File "/srv/ves/biz.shopping/.venv/lib/python2.7/site-packages/gunicorn_thrift/thriftpy_gevent_worker.py", line 75, in handle
File "/srv/ves/biz.shopping/.venv/lib/python2.7/site-packages/thriftpy/contrib/tracking/__init__.py", line 143, in process
File "/srv/ves/biz.shopping/.venv/lib/python2.7/site-packages/thriftpy/contrib/tracking/__init__.py", line 195, in _do_process
File "/srv/ves/biz.shopping/.venv/lib/python2.7/site-packages/thriftpy/thrift.py", line 254, in handle_exception
ImportError: No module named events
Gevent 1.2 supports 3.4+ according to their documentation (3.3+ less officially), so theoretically this library shouldn't have any problems supporting thriftpy_gevent in 3.4+ (3.3+). I propose, then, that it should (and if it does, the README should reflect that)
The version of setproctitle being used is broken on alpine linux. It was fixed dvarrazzo/py-setproctitle#44 and the new 1.1.10 version contains this fix. Can we bump the version?
thriftpy2, 在client退出后,服务端检查还有这个连接的socket在ESTABLISHED状态。这个有什么方法可以主动关闭么?
目前使用了thrift_client_timeout, 但是依然不理想
Throw import error when using high version gunicorn as gunicorn has already remove six from it's package, in gunicorn>=20.0。Remove six dependency。 hoping gunicorn_thrift can catch up with the new version, Thank you
Thrift server 本身不就有多进程模型吧 ,你让gunicorn 兼容thrift的协议是为了使用gunicorn的进程管理模型么? 好奇...
I just run command like
gunicorn_thrift tests.thriftpy_app:app -k thriftpy_sync \
--thrift-protocol-factory \
thriftpy.protocol:TCyBinaryProtocolFactory \
--thrift-transport-factory \
thriftpy.transport:TCyBufferedTransportFactory
,when I run on my computer, works well.
But on a server, some request always be hang on a long time.
i can't figure out which part got problem, thriftpy ? thrift_connector? gunicorn_thrift or even sqlalchemy.
finally, I found that one Process or called one worker only work for one connection. so i run into gunicorn documents find -w out.
so could you please add this parameter to Readme.md. so, someone else will not run in this trouble.
btw, could you please just share how to choose best client timeout and service timeout?
thanks.
When I try to install dependencies for an application that uses gunicorn_thrift
, I get errors from pip complaining about not finding a version that satisfies my given requirement.
Looking on PyPI, the only available distribution for gunicorn_thrift
0.2.1 is a Python 2 wheel.
Can you guys please upload an sdist and a Python 3 wheel?
Thanks!
Hey Eleme.
For a few days I've been trying to use New Relic monitoring on my gunicorn_thrift app. I've had no issues with my traditional gunicorn apps. I can launch my gunicorn_thrift application with no issues, but new relic gets no data.
I can launch it with Gunicorn like so:
NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program [gunicorn commands here]
But if I use:
NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program gunicorn_thrift tests.thriftpy_app:app -k thriftpy_sync \
--thrift-protocol-factory \
thriftpy.protocol:TCyBinaryProtocolFactory \
--thrift-transport-factory \
thriftpy.transport:TCyBufferedTransportFactory
I get no errors, but no communication to new relic.
Any advice for me?
Here is the situation I ran into:
In my project requirements.txt, I use gunicorn==19.1.1 and gunicorn_thrift master branch via git.
Then I run pip install -r requirements.txt, pip freeze, and gunicorn is 19.1.1. After that I try to run gunicorn_thrift, but error occur:
Traceback (most recent call last):
File "/virtualenv/bin/gunicorn_thrift", line 5, in <module>
from pkg_resources import load_entry_point
File "/virtualenv/lib/python2.7/site-packages/pkg_resources.py", line 3011, in <module>
parse_requirements(__requires__), Environment()
File "/virtualenv/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: gunicorn==19.1.0
Maybe a better way to write requirements.txt should be using gunicorn>=19.1.0
, instead of gunicorn==19.1.0
?
Also by doing this, if we got some issues caused by upstream changes, we can fix it in time.
i use this to create a service that provide RPC interface.
but when it was in pressure test,
cpu usage suddenly falled for a moment(about 10 seconds),then lots of requests were time out.
then all of these were back to normal.but after several minutes.it repeated again.
i use gevent model and 6 workers.
can you help me ?
使用了reuse_port=True 起了8个worker进程
但是 netstat -tunlp的时候 发现只有一个进程在监听端口
是不是SO_REUSEPORT没有效果,还是哪有什么问题?
I'm going to take a crack at this, so here's an issue for me to reference later:
TODO:
Hello,
I was trying to bringup gunicorn_thrift with http instead of roc to make it work under heroku load balancers.
I am facing issues when pushed gunicorn_thrift with rpc.
How can I change that to http?
Right now, I am bringing up app with
app = TProcessor(thrift_service,thrift_service_handler)
Hi,
I did a test and found a problem that bothered me.
I start gunicorn_thrift server and set --worker with the number of my cpu cores, it is 8.
Then I did the stress testing to the multiple workers server.
I found the total cpu usage of these 8 worker process is less than 100%, and the stress testing is broken.
I doubt if multiple worker server can use multiple cpu cores. Is it really a multi process with python server? Whether or not related to GIL with python.
Or, it is totally my program problem...
The following is some data of my testing:
THREAD_NUM = 100 // thread number per round
ONE_WORKER_NUM = 1000 // loop number of each thread
LOOP_SLEEP = 0.1 // time interval of each loop
PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
20 0 221136 30468 7380 S 23.9 0.2 0:40.24 gunicorn: worker [PythonServer:processor]
20 0 221196 30456 7380 S 20.9 0.2 0:33.67 gunicorn: worker [PythonServer:processor]
20 0 2626964 18972 3564 S 19.6 0.1 0:03.06 python stress_test.py
20 0 221116 30444 7380 S 17.0 0.2 0:29.29 gunicorn: worker [PythonServer:processor]
20 0 221104 30440 7380 S 14.0 0.2 0:24.07 gunicorn: worker [PythonServer:processor]
20 0 221100 30428 7380 S 11.3 0.2 0:18.64 gunicorn: worker [PythonServer:processor]
20 0 221476 30416 7380 S 7.6 0.2 0:14.29 gunicorn: worker [PythonServer:processor]
20 0 221332 30408 7380 S 6.6 0.2 0:11.93 gunicorn: worker [PythonServer:processor]
20 0 221008 30144 7308 S 5.7 0.2 0:10.69 gunicorn: worker [PythonServer:processor]
from thriftpy2.rpc import make_server
make_server(): this way start app, it can accept several client connection. but when i change to use gunicorn_thrift ,one worker just can accept one client connection.
--worker-connections INT
The maximum number of simultaneous clients. [1000]
i have add the parameter “--worker-connections” explicitly.
cmd:
rpc.rpc_server:app -k thriftpy_sync -b 0.0.0.0:6000 -w 1 --threads 1 --worker-connections 2 -t 5 --thrift-protocol-factory thriftpy2.protocol:TCyBinaryProtocolFactory --thrift-transport-factory thriftpy2.transport:TCyBufferedTransportFactory --log-level debug
I noticed installing from pypi and using a thriftpy_gevent worker is broken in the latest version 0.2.21.
Installing directly from this repo seems to solve the problem:
git+https://github.com/eleme/gunicorn_thrift.git
After further inspection the code looks pretty different on pypi
pypi 0.2.21
thriftpy_gevent_worker.py
line 100
Any chance of updating pypi distribution?
Thanks!
写了一个thrift文件
service PingPong {
string ping()
}
service Hello{
string hello2015()
}
然后使用thriftpy
import thriftpy
from thriftpy.thrift import TProcessor
from thriftpy.thrift import TMultiplexedProcessor
class PingPongDispatcher(object):
def ping(self):
return "pong"
class HelloDispatcher(object):
def hello2015(self):
return "hello2015"
pingpong_thrift = thriftpy.load("pingpong.thrift")
app1 = TProcessor(pingpong_thrift.PingPong, PingPongDispatcher())
app2 = TProcessor(pingpong_thrift.Hello, HelloDispatcher())
processor = TMultiplexedProcessor()
processor.register_processor("pingpong", app1)
processor.register_processor("hello", app2)
然后使用gunicorn_thrift启动
gunicorn_thrift app:processor -k thrift_gevent --thrift-protocol-factory thriftpy.protocol:TBinaryProtocolFactory --thrift-transport-factory thriftpy.transport:TBufferedTransportFactory --error-log -
客户端使用的是java、访问python服务器端时出现错误
[2015-08-29 14:09:49 +0800] [719] [ERROR] 'TCyBufferedTransportFactory' object has no attribute 'getTransport'
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/gunicorn_thrift/gevent_worker.py", line 42, in handle
self.get_thrift_transports_and_protos(result)
File "/Library/Python/2.7/site-packages/gunicorn_thrift/gevent_worker.py", line 25, in get_thrift_transports_and_protos
itrans = self.app.tfactory.getTransport(result)
AttributeError: 'TCyBufferedTransportFactory' object has no attribute 'getTransport'
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/gevent/greenlet.py", line 327, in run
result = self._run(_self.args, *_self.kwargs)
File "/Library/Python/2.7/site-packages/gunicorn_thrift/gevent_worker.py", line 59, in handle
itrans.close()
UnboundLocalError: local variable 'itrans' referenced before assignment
<Greenlet at 0x104e74870: <functools.partial object at 0x104ea4ec0>(<socket at 0x104f24750 fileno=11 sock=127.0.0.1:80, ('127.0.0.1', 49831))> failed with UnboundLocalError
请问这是怎么回事?本人新手学习thriftpy和使用gunicorn_thrift、跪请大神回复
项目使用 gunicorn_thrift + thriftpy2,接口正常请求时间400ms左右,但是持续请求测试时,客户端打印请求时间发现,偶尔会出现十几秒甚至几十秒的请求时间(我timeout设置比较大)。接口本身逻辑简单且服务端(业务代码)打印请求时间正常。线上请求量不算大,QPS最高也就300r/s
不知道你们有没有遇到过类似问题......
when I use gunicorn_thrift -c gunicorn.py pythriftServer:app
to launch my thrifypy2 app, import error occured like follow
(monitor) ➜ pythriftCoco gunicorn_thrift -c gunicorn.py pythriftServer:app
Traceback (most recent call last):
File "/home/cicada/.pyenv/versions/monitor/bin/gunicorn_thrift", line 7, in <module>
from gunicorn_thrift.thriftapp import run
File "/home/cicada/.pyenv/versions/3.6.5/envs/monitor/lib/python3.6/site-packages/gunicorn_thrift/thriftapp.py", line 11, in <module>
from . import config
File "/home/cicada/.pyenv/versions/3.6.5/envs/monitor/lib/python3.6/site-packages/gunicorn_thrift/config.py", line 4, in <module>
from gunicorn import six
ImportError: cannot import name 'six'
maybe gunicorn_thrift version is not matched with gunicorn
Gunicorn added support in 19.4 and gevent did in 1.1.
I tested gunicorn_thrift and the gevent worker with those versions, and it seems to work fine.
Hi. Is there some way to do more grained configuration in gunicorn_thrift just like gunicorn? E.g. worker number or thread number.
eleme/gunicorn looks live have been removed. Could not open https://github.com/eleme/gunicorn/commit/0ac40b460149f757cbbc04288f4764c3e261a645 any more.
cc @wooparadog
and how to config self-define proto_factory or modiefied TBinaryProtocolFactory ?
您好! 请问关于使用thriftpy2 + gunicorn_thrift + gevent worker-class 使用场景来再生产环境里,增强thriftpy server吞吐性能使用场景,我遇到如下几个问题,特来请教大神:
(1) 对于使用thrfitpy2框架,并业务handler里,处理一条请求里有较多rpc访问,我理解上述 gunicorn 多worker进程 + 每个worker使用gevent类型,应该是性能最佳的组合? 当然我没有实践,跟@ethe 确认
(2) 我看worker-class = gevent 只适用于protocol=TBinaryProtocol 和 transport=TBufferedTransport, transport不支持TFramedTransport么?
(3) 我原来使用thriftpy2 make_server接口,采用的TthreadServer, 多线程模式并发处理请求,业务处理handler里,需要在处理并发请求时,加锁,做一些同步处理,这种场景,改为gunicorn_thrift fork多worker进程模式,应当如何保障在多个worker进程处理请求时,共享一个全局的进程锁,多worker同步了?
@ethe 能否在百忙之中,及时帮我们答疑解惑了? 万分感谢! 以及是否方便微信联系了? 我们在较大规模应用thriftpy2 以及相关组件
I'm use the gunicorn_thrift like last issues,some time is run well ,but when it run a while ,I got a error
1741648 [2017-12-06 12:15:39 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7438)
1741649 [2017-12-06 12:15:39 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7445)
1741650 [2017-12-06 12:15:39 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7447)
1741651 [2017-12-06 12:15:39 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7448)
1741652 [2017-12-06 12:15:39 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7449)
1741653 [2017-12-06 12:15:39 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7451)
1741654 [2017-12-06 12:15:39 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7453)
1741655 [2017-12-06 12:15:39 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7456)
1741656 [2017-12-06 12:15:39 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7457)
1741657 [2017-12-06 12:15:39 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7458)
1741658 [2017-12-06 12:15:39 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7459)
1741659 [2017-12-06 12:15:40 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7434)
1741660 [2017-12-06 12:15:40 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7437)
1741661 [2017-12-06 12:15:40 +0000] [5727] [CRITICAL] WORKER TIMEOUT (pid:7443)
1741662 [2017-12-06 12:15:40 +0000] [7506] [INFO] Booting worker with pid: 7506
1741663 [2017-12-06 12:15:40 +0000] [7507] [INFO] Booting worker with pid: 7507
1741664 [2017-12-06 12:15:40 +0000] [7508] [INFO] Booting worker with pid: 7508
1741665 [2017-12-06 12:15:40 +0000] [7509] [INFO] Booting worker with pid: 7509
1741666 [2017-12-06 12:15:40 +0000] [7510] [INFO] Booting worker with pid: 7510
1741667 [2017-12-06 12:15:40 +0000] [7511] [INFO] Booting worker with pid: 7511
1741668 [2017-12-06 12:15:40 +0000] [7512] [INFO] Booting worker with pid: 7512
1741669 [2017-12-06 12:15:40 +0000] [7513] [INFO] Booting worker with pid: 7513
1741670 [2017-12-06 12:15:40 +0000] [7514] [INFO] Booting worker with pid: 7514
1741671 [2017-12-06 12:15:40 +0000] [7515] [INFO] Booting worker with pid: 7515
1741672 [2017-12-06 12:15:40 +0000] [7514] [ERROR] Exception in worker process:
1741673 Traceback (most recent call last):
1741674 File "/usr/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 507, in spawn_worker
1741675 worker.init_process()
1741676 File "/usr/local/lib/python2.7/site-packages/gunicorn_thrift/thriftpy_gevent_worker.py", line 98, in init_process
1741677 return super(GeventThriftPyWorker, self).init_process()
1741678 File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 192, in init_process
1741679 super(GeventWorker, self).init_process()
1741680 File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 118, in init_process
1741681 self.wsgi = self.app.wsgi()
1741682 File "/usr/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
1741683 self.callable = self.load()
1741684 File "/usr/local/lib/python2.7/site-packages/gunicorn_thrift/thriftapp.py", line 39, in load
1741685 self.thrift_app = self.load_thrift_app()
1741686 File "/usr/local/lib/python2.7/site-packages/gunicorn_thrift/thriftapp.py", line 33, in load_thrift_app
1741687 return utils.load_obj(self.app_uri)
1741688 File "/usr/local/lib/python2.7/site-packages/gunicorn_thrift/utils.py", line 16, in load_obj
1741689 mod = importlib.import_module(module)
1741690 File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
1741691 __import__(name)
1741692 File "/usr/local/lib/python2.7/site-packages/gevent-1.1rc2-py2.7-linux-x86_64.egg/gevent/builtins.py", line 58, in __import__
1741693 result = _import(*args, **kwargs)
1741694 File "/home/zhouyi/heybox_script/release/pubg_thriftpy/pubg_account_server.py", line 13, in <module>
1741695 pg_account = psycopg2.connect(database="account", user='postgres', password="", host="10.10.10.54", port='19934')
1741696 File "/usr/local/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
1741697 conn = _connect(dsn, connection_factory=connection_factory, async=async)
1741698 OperationalError: FATAL: sorry, too many clients already
it seems like the WORKER got timeout but not exit so the master booting another worker , finally the worker connect to pg got the ' too many clients' error, anyone know what's the problem ?
I was run thriftpy server as the readme suggest, and i used gunicorn_thrift as follow config
gunicorn_thrift account_server:app -w 60 \
--worker-connections 2000 -k thriftpy_sync --thrift-protocol-factory thriftpy.protocol:TCyBinaryProtocolFactory \
--thrift-transport-factory thriftpy.transport:TCyBufferedTransportFactory --threads 5 -D \
--error-logfile /homex2/zhouyi/log/account_thrift_error.log --access-logfile /homex2/zhouyi/log/account_thrift_access.log\
-b 0.0.0.0:16796 --reload --forwarded-allow-ips 10.10.10.55
the test code was
import time
from multiprocessing import Process
import thriftpy
account_thrift = thriftpy.load("account.thrift", module_name="account_thrift")
from thriftpy.rpc import make_client
def main():
thread_list = []
start_time = time.time()
for i in xrange(0,800):
th = Process(target=process_func)
th.start()
thread_list.append(th)
for i in thread_list:
i.join()
print time.time() - start_time
def process_func():
th = MyThread()
th.process_func()
class MyThread:
def __init__(self):
self.client = make_client(account_thrift.AccountRpc, '127.0.0.1', 16796)
def process_func(self):
for i in xrange(1,10000):
print self.client.rpc_account_sid('account.5e4c769046c54b37a6f441fed028186d')
time.sleep(0.5)
if __name__ == '__main__':
r = main()
when i run the test code i got error with block
and i can't connect to the rpc server with other client too i got error
thriftpy.transport.TTransportException: TTransportException(message="Could not connect to ('127.0.0.1', 16796)", type=1)
, Is there any config error and what't the problem?
I'v installed gunicorn_thrift, but can't find the command gunicorn_thrift?
I would recommend changing the requirement gunicorn version to 19.9.0
. Because older version has issue: benoitc/gunicorn#1822
Later version has six
import problem.
In my situation, I just set client timeout as low as possible described in #45, but i always got a timeout message like Thriftpy/thrift_connector#48 said.
At last, I found that in https://github.com/eleme/gunicorn_thrift/blob/master/gunicorn_thrift/thriftpy_sync_worker.py#L75, it always log exception with full error stack.
But I think we should first identify to exception like below:
if isinstance(e, socket.timeout):
self.log.info("timeout")
else:
self.log.exception(e)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.