tornadoweb / tornado Goto Github PK

Like Google webapp : http://code.google.com/intl/fr/appengine/docs/python/tools/webapp/req...
it could be nice to have an "allow_multiple=True" in the get_argument
function. (or a get_all_argument)

Example : currently, if self.request.arguments returns {'key1':
['val3'], 'key2': ['val1', 'val2']}
self.get_argument(key2) will return 'val2'

and with an allow_multiple=True :
self.get_argument(key2, allow_multiple=True) it should return ['val1',
'val2']

I know it's possible to do : self.request.arguments['key2'], but here
we missed all unicode/utf-8 functions.

Another thing, in my example : {'key1': ['val3'], 'key2': ['val1',
'val2']}
For key1, that is not "multiple", self.request.arguments should return
{'key1': 'val3', 'key2': ['val1', 'val2']}.
i.e. : key1 should not be a list but a value.

I think it is the same issue with self.request.files, that always
returns a list even if there is only one file.

more info here : http://groups.google.com/group/python-tornado/browse_frm/thread/33325d1c500dccef/7af8756c4547b6bc

This is because of the same origin policy. See django's patch for more info:

http://code.djangoproject.com/changeset/9554

P.S. Thanks for releasing Tornado!

Since this commit http://github.com/facebook/tornado/commit/aa836cdf3952d52efc68201872b3a0053aca33ad
there is always mistaken css style constructions appears if I've defined css_files(), but haven't defined embedded_css():

<style type="text/css">
.......
</style>

That's because you're filling css_embed with inclusion path.

   if css_files:
        ...
        css_embed = ''.join('<link href="' + escape.xhtml_escape(p) + '" '
                            'type="text/css" rel="stylesheet"/>'
                            for p in paths)
        ...
    if css_embed:
        css_embed = '<style type="text/css">\n' + '\n'.join(css_embed) + \
            '\n</style>'

I think you'll find the same problem with embedded_javascript() and javascript_files()

On line 424: slice.indices(size) call is missing the first arg (i.e. key).

I think it should move to settings

handlers.extend([
(r"/static/(.*)", StaticFileHandler, dict(path=path)),

http://groups.google.com/group/python-tornado/browse_thread/thread/593d41a6c2fb9246

When I try to use the DELETE method with the HTTP client, I get:
Traceback (most recent call last):
File "/Users/dking/src/reddit/rdb/rdbclient.py", line 154, in delete
resp = self.http.fetch(req)
File "/usr/local/lib/python2.6/site-packages/tornado/httpclient.py", line 63, in fetch
File "/usr/local/lib/python2.6/site-packages/tornado/httpclient.py", line 372, in _curl_setup_request
KeyError: 'DELETE'
It looks like this is happening here (http://github.com/facebook/tornado/blob/master/tornado/httpclient.py#...
curl_options = {
"GET": pycurl.HTTPGET,
"POST": pycurl.POST,
"PUT": pycurl.UPLOAD,
"HEAD": pycurl.NOBODY,
}
for o in curl_options.values():
curl.setopt(o, False)
curl.setopt(curl_options[request.method], True)

Is this a quick-fix, or does pycurl just not support the method?

The server does not parse folded HTTP headers. Please see IETF RFC 2616 section 2.2 for the details.

LSB Version: :core-3.0-amd64:core-3.0-noarch:graphics-3.0-amd64:graphics-3.0-noarch
Distributor ID: RedHatEnterpriseAS
Description: Red Hat Enterprise Linux AS release 4 (Nahant Update 3)
Release: 4
Codename: NahantUpdate3

change ioloop.py, change 145 line, add
time.sleep(0.0001)

it works.

Could you add an option for the url_escape method that don't use
quote_plus but only quote.

Something like this :

def url_escape(value, quote_plus=True):
"""Returns a valid URL-encoded version of the given value."""
if quote_plus:
return urllib.quote_plus(utf8(value))
return urllib.quote(utf8(value))

See http://groups.google.com/group/python-tornado/browse_thread/thread/7331cc79de2f09f7

The _signature method of tornado.auth.FacebookMixin uses the application secret when signing the requests. This causes all the authenticated requests to the Facebook API fail since the requests should be signed using the session secret. I guess Facebook has changed their API at some point. http://wiki.developers.facebook.com/index.php/Authorization_and_Authentication_for_Desktop_Applications

'set_cookie' and 'get_cookie' in web.py must quote and unquote

for example

new_cookie[name] = value
must be
new_cookie[name] = urllib.quote(value)

because if '=' in set-cookie, ie and webkit will ignore it

base64.b64encode('yaki')
'eWFraQ=='

http://www.ietf.org/rfc/rfc2109.txt

The two state management headers, Set-Cookie and Cookie, have common
syntactic properties involving attribute-value pairs. The following
grammar uses the notation, and tokens DIGIT (decimal digits) and
token (informally, a sequence of non-special, non-white space
characters) from the HTTP/1.1 specification [RFC 2068] to describe
their syntax.

av-pairs = av-pair *(";" av-pair)
av-pair = attr ["=" value] ; optional value
attr = token
value = word
word = token | quoted-string

Issue: http://groups.google.com/group/python-tornado/browse_thread/thread/4da447e9a79344c2

Solution: http://groups.google.com/group/python-tornado/msg/2c74716c5cedb119

Hi,

I believe that I encounter a bug when try to create an instanace of httpserver.HTTPRequest.

In the constructor of httpserver.HTTPRequest, I think this line
self.host = host or headers.get("Host") or "127.0.0.1"
should probably mean
self.host = host or self.headers.get("Host") or "127.0.0.1"

Otherwise, creating an HTTPReqeust instance without giving headers will die here.

Thanks,
Victor

If I upload a file with a semicolon (;) ["my;file.pdf"], Tornado
crashes

File "tornado\wsgi.py", line 172, in _parse_mime_body
name, name_value = name_part.strip().split("=", 1)
ValueError: need more than 1 value to unpack

Semicolon in filename is most of the time nerver well handled in
python (don't know why it is not fixed... or maybe I don't know how it
works) and we "often' get : "my" instead of "my;file.pdf".

more info here : http://groups.google.com/group/python-tornado/browse_frm/thread/6e6ade3b63667fd0/e9aab86519647dd8

with out subclassing RequestHandler.

Is there a way to hook up middleware into tornado. Process every incoming request and then route it to the handler.

I am trying to handle urls with trailing slashes -> ex: /login/ & /login

(r"/login/?", LoginHandler),

Using a regex solves the problem but have a slight doubt that it kills some performance. With middleware we can do something like url.rstrip('/')

I was doing some evaluation on the httpclient with async calls on Python 2.6.2 and received the following IOError:

ERROR:root:Exception in callback <bound method AsyncHTTPClient._perform of <tornado.httpclient.AsyncHTTPClient object at 0xa17e18c>>
Traceback (most recent call last):
    File "/home/steve/sandbox/tornado/tornado/ioloop.py", line 241, in _run_callback
          callback()
    File "/home/steve/sandbox/tornado/tornado/httpclient.py", line 222, in _perform
        self.io_loop.update_handler(fd, events)
    File "/home/steve/sandbox/tornado/tornado/ioloop.py", line 127, in update_handler
        self._impl.modify(fd, events | self.ERROR)
IOError: [Errno 2] No such file or directory

I added a catch for this in the httpclient where calls the epoll modify function, so that it registers the fd if the modify fails. The change set is here:

http://github.com/stevvooe/tornado/commit/d85baebc02791ac4ddc3ce4a3d70ac5361fae3ba

Bug in line 275 of ioloop.py causes NameError:
self.io_loop = io_loop or ioloop.IOLoop.instance()

The line should be:
self.io_loop = io_loop or IOLoop.instance()

Testing against things behind mod_wsgi is not a real test. There are too many variables in your performance test, this is not a good scientific experiment.

I believe L127->L129 should read:

    if not self.request.supports_http_1_1() and not self.settings.no_keep_alive:
        if self.request.headers.get("Connection") == "Keep-Alive":
            self.set_header("Connection", "Keep-Alive")

and not:

    if not self.request.supports_http_1_1():
        if self.request.headers.get("Connection") == "Keep-Alive":
            self.set_header("Connection", "Keep-Alive")

Not sure if it's by design, but in get_secure_cookie(), a cookie over 31 days is always considered expired. It has no relation to expires_days argument in set_secure_cookie. Effectively it means that providing expires_days > 31 in set_secure_cookie() will have no effect and cookie will last for 31 days.

Httpserver HTTPHeaders uses C-extension instead of native python.
I strongly disagree with, unless there is a huge performance improvements.
I do not understand why the use of non-core functions to the expansion of c?

The prototype for a start_response method is as follows:

def start_response(status,response_headers,exc_info=None):

See here: http://www.python.org/dev/peps/pep-0333/#the-start-response-callable.

The version in Tornado does not take the exc_info parameter meaning it is not possible to run a number of middlewares under it. It's an easy one to fix though :)

You should publish tornado on PyPI, so it's possible to use it as a dependency.

Even if you consider this beta software, it's considered a best practice. People have already started using Tornado, so there's no reason not to make it easier for people to deploy it.

296 s/WEEKDAYS/_weekdays/

Simple request. Just looking for tornado.version = (0, 2) or '0.2' or something.

If the title of the blog contains ", then escape in the template it not enough.

I change it, you can look my blog: http://chenxiaoyu.org/blog/archives/82

but code is not check DBUtils version.

code:

def reconnect(self):
    """Closes the existing database connection and re-opens it."""
    self.close()
    try:
        from DBUtils import PooledDB
        pool_con = PooledDB.PooledDB(creator=MySQLdb, **self._db_args)
        self._db = pool_con.connection()
    except:
        self._db = MySQLdb.connect(**self._db_args)
        self._db.autocommit(True)

See http://groups.google.com/group/python-tornado/browse_thread/thread/7331cc79de2f09f7

I have a huge json string in my template and I am using

{{ json_decode(json_string)["name"] }}

similarly to render other attributes I am calling json_decode everytime. Is there a way I can assign the output of the json_decode operation to a variable?

{% data_dict = json_decode(json_string) %}

I get ParseError: unknown operator: 'data_dict'.
Any work arounds?

iostream.py provides a way to set a callback that should be invoked when a connection is closed. When using _Select this callback won't be invoked until a read or write is performed on the socket.

Select signals remote hang-ups by creating a read event with 0 byte length. This won't be noticed by iostream's _handle_event when a connection is idle since it only subscribes to io_loop.ERROR events.

This works fine with epoll since epoll with emit a HUP event that will get noticed.

Nevermind, got it to work by doing the following:

table = 'logs'
column_id = 'col_id'
column_data = 'col_data'
log_data = 'some data here'
self.db.execute(
"INSERT INTO %s (%s, %s)" % (table, column_id, column_data) +
"VALUES (%s, %s)", 'NULL', log_data
)

On Linux 2.6 with Python 2.6.2, when I run demos/chat/chatdemo.py the server listens on an IPv4 socket and not an IPv6 socket. lsof shows:-

python 4755 chaz 6u IPv4 11426409 TCP *:8888 (LISTEN)

website/templates/documentation.txt - Line 376

Should have an action attribute instead of two method attributes.

Replace "/" by os.sep int the line 1040:
self.root = os.path.abspath(path) + "/"

Is there a way to run system commands asynchronously? For instance, let's say you're given an image as input, and you'd like to run some image conversion or optimization program in the background. Can you do this without blocking?

The user marc in the group encountered this, and the patch was supplied in their post here: http://groups.google.com/group/python-tornado/browse_thread/thread/29ec5a1300c10e2/1c7c8239cad08def?lnk=gst&q=put+method#1c7c8239cad08def

diff --git a/tornado/httpserver.py b/tornado/httpserver.py
index 460f5c8..60957c7 100644
--- a/tornado/httpserver.py
+++ b/tornado/httpserver.py
@@ -201,7 +201,7 @@ class HTTPConnection(object):
def _on_request_body(self, data):
self._request.body = data
content_type = self._request.headers.get("Content-Type", "")

•    if self._request.method == "POST":
  

•    if self._request.method in ("POST", "PUT"):
       if content_type.startswith("application/x-www-form-
  

  urlencoded"):
  arguments = cgi.parse_qs(self._request.body)
  for name, values in arguments.iteritems():

If I am reading this correctly:
The body of the remove_callback method should be
self._callbacks.remove(callback)
not
self._callbacks.pop(callback)

I found some function _utf8 and _unicode repeatedly appear in different modules, that's not convenient maintain.
The following function appear in web.py module is right?
def _utf8(s):
if isinstance(s, unicode):
return s.encode("utf-8")
assert isinstance(s, str)
return s
or the following function appear in httpclient.py module is right.
def _utf8(value):
if value is None:
return value
if isinstance(value, unicode):
return value.encode("utf-8")
assert isinstance(value, str)
return value
I used to use last code. Thanks!

Right now, tornado.escape.json_encode will throw an error if any arguments other than the object to be encoded (such as the ensure_ascii option to prevent Unicode escaping) are passed.

Is it possible to pass these options through as well?

I have a some machines with Solaris 10 and I wanted to try/use tornado on them also - but select() is very limited -, so I created a new little class (based on your _Select class) that uses libevent, so you can have better polling methods on non-linux systems.
It depends on python-libevent.

Here you can find my modification, Solaris/FreeBSD users may find it useful:
http://github.com/dyuri/tornado/blob/master/tornado/ioloop.py

I just spent a bit of time digging around to find out that in tornado.httpclient._curl_setup_request there's this code::

try:
    curl.setopt(pycurl.HEADERFUNCTION,
                functools.partial(_curl_header_callback, headers))
except:
    # Old version of curl; response will not include headers
    pass

The "old version of curl" refers to any version before 7.16.4 (see the ChangeLog here: http://pycurl.sourceforge.net/ChangeLog ).

In the tornado README, there's a suggested Mac OSX command of::

$ sudo easy_install setuptools pycurl==7.16.2.1 simplejson

To resolve Tornado's dependencies. Perhaps it's worth either updating that to 7.16.4 (looking at the change log I can't see any compatibility / libcurl version issues) or perhaps at least flagging up the choice?

If not worth tweaking the README at least perhaps this post might help if you're scratching your head...

Thanks,

James.

A nice to have would be for the request handler to automatically append a / to urls that don't have one, similar to how Django does it.

Currently I'm configuring urls like this

application = tornado.web.Application([
(r"/", MainHandler),
(r"/test/*", views.TestHandler),
])

But being able to drop the regex * would make for simpler code and reduce a chance of a typo by forgetting to include it.

Added support to parse the body into the arguments of Request object on PUT as well as POST

http://github.com/araddon/tornado/commit/f9b62ff57d698cb935be318e776ce509243dc06c

Related to this (allow_multiple) in get_arguments : http://github.com/facebook/tornado/issues#issue/26

It could be nice to have a "nice" self.arguments : same than self.request.arguments but with all unicode jobs done in get_argument + single value not in a list.

i.e :

@Property
def arguments(self)
return dict((k, get_argument(k, allow_multiple=True)) for k in
self.request.arguments)

So if self.request.arguments = {'k1': ['unicode_val1'], 'k2':
['unicode_val21', 'unicode_val22']}
then
self.arguments should returns :
{'k1': u'unicode_val1', 'k2': [u'unicode_val21', u'unicode_val22']}

Reported by Stefan Scholl on the mailing list:

I've just seen that tornado.escape.xhtml_escape (used as "escape" in
the templates) only escapes "&", "<", and ">". It uses escape from
xml.sax.saxutils.

Used in an attribute, ' and " should be escaped, too.

But don't convert ' to ', because this is XML only and can be a
problem with HTML and visitors using Internet Explorer. ' should be
converted to '.

I was running into issues with a JSON Api I was building for cross domain requests coming from Firefox/Safari that were sending an HTTP OPTIONS pre-flight request before sending the http get. See https://developer.mozilla.org/en/HTTP_access_control

My Fix http://github.com/araddon/tornado/commit/d91deae5f22e6d0cd3b86d5b960729ae004bd394

I was able to get Firefox/Safari to work correctly by implementing an "options" method on my handler similar to get/post but actually just returning nothing.

website/templates/documentation.txt

Passage:
Modules can include custom CSS and JavaScript functions by overriding
the embedded_css, embedded_javascript, javascript_file, or
css_file methods:

Change
javascript_file -> javascript_files
css_file -> css_files

:207 s/Read/Write/