Support gevent on Python3 about gunicorn_thrift HOT 4 CLOSED

@dragonpaw was there any additional config that you needed to add to get it working with the thriftpy_gevent worker? i'm still unable to utilize it on 0.2.12, which is the version that corresponds to your PR.

my app would work under the thriftpy_sync worker with the command:

$ gunicorn_thrift index:app \
  --bind 127.0.0.1:9090 \
  --chdir thrift \
  --worker-class thriftpy_sync \
  --thrift-protocol-factory thriftpy.protocol:TCyBinaryProtocolFactory \
  --thrift-transport-factory thriftpy.transport:TCyBufferedTransportFactory

... but when i change the worker-class to thriftpy_gevent, which would be in the AVAILABLE_WORKERS in six.py:

$ gunicorn_thrift index:app \
  --bind 127.0.0.1:9090 \
  --chdir thrift \
  --worker-class thriftpy_gevent \
  --thrift-protocol-factory thriftpy.protocol:TCyBinaryProtocolFactory \
  --thrift-transport-factory thriftpy.transport:TCyBufferedTransportFactory

i get a blank error response of just Error:. the behavior is the same if i use any of the other unsupported workers as well.

environment:

$ python --version
Python 3.5.1
$ pip freeze
gunicorn==19.4.5
gunicorn-thrift==0.2.12
ply==3.8
setproctitle==1.1.8
thriftpy==0.3.7

i also tried manually pip installing gevent into my dependencies as well just for effect, but i'm unsure if that did anything. i still get the same behavior.

i'm still pretty new to python in general, so i apologize if i missed something basic here...

from gunicorn_thrift.

We're running with the following config(s) for what it is worth:

gunicorn_config.py:

# -*- coding: utf-8 -*-

worker_class = "thriftpy_gevent"
thrift_protocol_factory = "thriftpy.protocol:TCyBinaryProtocolFactory"
thrift_transport_factory = "thriftpy.transport:TCyBufferedTransportFactory"

errorlog = "-"
loglevel = "info"

bind = "0.0.0.0:7070"
workers = {{grains.num_cpus}}
threads = {{grains.num_cpus}} * 4
worker_connections = 1000
backlog = 2048

import os
def on_connected(worker, addr):
    #worker.log.info("connected: %s", addr)
    os.environ['REMOTE_IP'] = addr[0]
    os.environ['REMOTE_PORT'] = str(addr[1])

The {{ grains.* }} represents some magic from Saltstack. The last bit is so we can debug where connections come from via environment variables.

For running the app we use runit, but basically the command comes out to exec chpst -uwww-data -n-5 gunicorn_thrift -c /etc/gunicorn_thrift.py awesome_company.{{grains.cluster}}.thrift.server:app where server.py defines an app object which is a TProcessor(FooThriftServiceClass, FooPythonClass)

from gunicorn_thrift.

The main topic aside. @dragonpaw I spotted a potential bug in your on_connected hook.
Gevent is capable of receiving many connections at the same time. You should use contextlocal to save IP/Port information, rather than os.environ.

We are using the following:

def on_connected(worker, addr):
    from ...ctx import g
    if addr:
        g.set_conn_meta('client_addr', addr[0])

Note: g is a threadlocal object that is imported after fork(so after gevent patch).

from gunicorn_thrift.

@dragonpaw thanks for the response! i was able to resolve my issue by using a configuration file. gunicorn seems to provide more informative startup error messages if you pass in a config file for it to read

from gunicorn_thrift.