Bug fixes and features (thanks to the Brizzly team)

123 views
Skip to first unread message

Bret Taylor

unread,
Jan 11, 2010, 2:47:56 PM1/11/10
to python-...@googlegroups.com
The folks at Brizzly (http://brizzly.com/) use Tornado, and they have contributed a number of patches, which I just integrated into the main branch of Tornado. You can see the full change here: http://github.com/facebook/tornado/commit/7014417608d03e7af2c7aba58ba358d8657a24a2

Among the changes:
  • KQueue support for Mac OS X and BSD
  • Support for named URLs and reverse URLs (more on that below)
  • Add a method to RequestHandler that can be used to detect when the connection has been closed by the client. Useful for long-lived connections so you know when to clean up resources. Just override on_connection_close in your request handler, which will be called when the connection is closed for any reason.
  • Some tweaks to IOLoop to make it easier to run unit tests with asynchronous code
  • Bug fixes to the WSGI code (which is used to run Django on Tornado and used to run Tornado within Google AppEngine)
Named URLs and reverse URLs have been talked about on this list before. They enable you to name a URL on your site so you can easily generate URLs without hard-coding the path in all of your source code and templates. Say articles on your blog have the pattern r"/article/([0-9]+)", and you name that URL "article". Then, reverse_url("article", 12) will return "/article/12". You can use the reverse_url function within templates or self.reverse_url within your request handlers. You name URLs by making a small tweak to your list of request handlers to use tornado.web.url. For example, you might convert your application from this:

    application = tornado.web.Application([
        (r"/", MainHandler),
        (r"/article/([0-9]+)", ArticleHandler),
    ])

to this:

    application = tornado.web.Application([
        tornado.web.url(r"/", MainHandler),
        tornado.web.url(r"/article/([0-9]+)", ArticleHandler, name="article"),
    ])

Notice the name="article" keyword argument in the second URL.

I am in the process of doing the long-overdue update to our documentation to include this and previous changes. Let me know if you encounter any issues with this change, and thanks again to the Brizzly folks for their patches.

Bret

ag3nt

unread,
Jan 11, 2010, 2:50:14 PM1/11/10
to Tornado Web Server
This is great news! Thanks!!!!!!

On Jan 11, 11:47 am, Bret Taylor <btay...@gmail.com> wrote:
> The folks at Brizzly (http://brizzly.com/) use Tornado, and they have
> contributed a number of patches, which I just integrated into the main

> branch of Tornado. You can see the full change here:http://github.com/facebook/tornado/commit/7014417608d03e7af2c7aba58ba...
>
> Among the changes:
>
>    - KQueue support for Mac OS X and BSD
>    - Support for named URLs and reverse URLs (more on that below)
>    - Add a method to RequestHandler that can be used to detect when the


>    connection has been closed by the client. Useful for long-lived connections
>    so you know when to clean up resources. Just override on_connection_close in
>    your request handler, which will be called when the connection is closed for
>    any reason.

>    - Some tweaks to IOLoop to make it easier to run unit tests with
>    asynchronous code
>    - Bug fixes to the WSGI code (which is used to run Django on Tornado and

Matthew Ferguson

unread,
Jan 11, 2010, 2:57:25 PM1/11/10
to python-...@googlegroups.com
Bret,

This is great! Thank you and thanks to Brizzly. Since quite a few features have been added since 0.2 came out, when can we expect 0.3 to become official? What needs to be done before 0.3 is ready?

Andrew Gwozdziewycz

unread,
Jan 11, 2010, 3:04:15 PM1/11/10
to python-...@googlegroups.com
I don't have time right now to look at the diff, but is this any
different than my proposed change with patch?
http://groups.google.com/group/python-tornado/browse_frm/thread/90ef3b4c4923508d?tvc=1&q=reverse_url.
I'm guessing maybe better reversing than my hacked up version?

--
http://www.apgwoz.com

Andrew Gwozdziewycz

unread,
Jan 11, 2010, 3:04:47 PM1/11/10
to python-...@googlegroups.com
(the reverse urls portion--not KQueue support and such, that is)

--
http://www.apgwoz.com

Ben Darnell

unread,
Jan 11, 2010, 3:12:10 PM1/11/10
to python-...@googlegroups.com
On Mon, Jan 11, 2010 at 12:04 PM, Andrew Gwozdziewycz <apg...@gmail.com> wrote:
> I don't have time right now to look at the diff, but is this any
> different than my proposed change with patch?
> http://groups.google.com/group/python-tornado/browse_frm/thread/90ef3b4c4923508d?tvc=1&q=reverse_url.
> I'm guessing maybe better reversing than my hacked up version?

It's based on your change. I think the only thing I changed was to
make Application aware of URLSpecs instead of making URLSpecs pretend
to be tuples. The actual regex reversing code is yours - thanks for
the contribution.

-Ben

Bret Taylor

unread,
Jan 11, 2010, 3:27:28 PM1/11/10
to python-...@googlegroups.com
There are a few bugs I am hoping to fix this week around auto-reloading, pre-forking, and a handful of others. Once those are fixed, I think we should release 0.3.

Bret

Bret Taylor

unread,
Jan 11, 2010, 3:28:22 PM1/11/10
to python-...@googlegroups.com
Andrew, sorry for not calling you out in the original email. As Ben mentioned, it looks like it was based on your change. Thanks so much for taking the time, and I will be sure to properly thank you next time :)

Bret

On Mon, Jan 11, 2010 at 12:04 PM, Andrew Gwozdziewycz <apg...@gmail.com> wrote:

Matthew Ferguson

unread,
Jan 11, 2010, 3:28:17 PM1/11/10
to python-...@googlegroups.com
Awesome!  Would you like the community to contribute to the documentation?

Andrew Gwozdziewycz

unread,
Jan 11, 2010, 3:29:32 PM1/11/10
to python-...@googlegroups.com
On Mon, Jan 11, 2010 at 3:12 PM, Ben Darnell <ben.d...@gmail.com> wrote:
> On Mon, Jan 11, 2010 at 12:04 PM, Andrew Gwozdziewycz <apg...@gmail.com> wrote:
>> I don't have time right now to look at the diff, but is this any
>> different than my proposed change with patch?
>> http://groups.google.com/group/python-tornado/browse_frm/thread/90ef3b4c4923508d?tvc=1&q=reverse_url.
>> I'm guessing maybe better reversing than my hacked up version?
>
> It's based on your change.  I think the only thing I changed was to
> make Application aware of URLSpecs instead of making URLSpecs pretend
> to be tuples.  The actual regex reversing code is yours - thanks for
> the contribution.

Great! I'm glad this change made it in so that I can avoid having a
fork that differs so much. I made URLSpecs look like tuples to for
better compatibility, but I didn't think too much about it.

I guess the biggest problem then is that the regex reversing isn't
very robust, as you may have noticed. It'll work fine for simple
things, but you can't nest parens or anything like that.

One thing I've been considering recently, is to drop the regex
completely and use an extended routes approach. So, instead of /(\d+)/
you might do /:digits/, or in the case of named groups /:digits<num>/
which would then just get converted to the appropriate regex at
instantiation time. But, I'm not sure if this adds anything or not...


--
http://www.apgwoz.com

Andrew Gwozdziewycz

unread,
Jan 11, 2010, 3:30:56 PM1/11/10
to python-...@googlegroups.com
Bret-

No problem! Tornado is fun to play with, so I'm happy to
contribute--even without attribution :). I asked, not because I was
upset that I wasn't attributed, but more to see if my tree was still
compatible.

Thanks,

Andrew

--
http://www.apgwoz.com

Sergey Konozenko

unread,
Jan 11, 2010, 4:53:10 PM1/11/10
to Tornado Web Server
Bret,

Are there any chance you can look at the problem mentioned in this
thread?

http://groups.google.com/group/python-tornado/browse_thread/thread/276059a076593266

It looks like there is some number of users who experience the
problem. Essentially, the problem might be a show-stopper for those
who do experience it. Or it might get reduced to some simple exception-
handling changes. It could be an artifact of a particular platform
configuration. Is there any chance that somebody with enough knowledge
in that specific area can get involved?

Thanks,
Sergey

David P. Novakovic

unread,
Jan 11, 2010, 7:32:29 PM1/11/10
to python-...@googlegroups.com
Hate to sound ungrateful, but I gotta agree with Sergey here... the epoll problem seems pretty serious.

Otherwise, thanks a bunch for the new features and fixes!

I'd love to see how Brizzly are unit testing their code... this is one area I'm feeling particularly vulnerable in..

D

Ben Darnell

unread,
Jan 11, 2010, 7:47:41 PM1/11/10
to python-...@googlegroups.com
The general pattern for our unittests is something like this:

class MyTest(unittest.TestCase):
def testFoo(self):
server = HTTPServer(...)
server.listen(9999)
client = AsyncHTTPClient()
client.fetch("http://localhost:9999/foo", callback=self.handle_response)
IOLoop.instance().start()

def handle_response(self, response):
self.assertEquals(...)
IOLoop.instance().stop()


Most of our tests don't use the full http server/client machinery, but
we do have a few end-to-end tests like this.

-Ben

David P. Novakovic

unread,
Jan 11, 2010, 8:07:45 PM1/11/10
to python-...@googlegroups.com
Ben,

Thanks a bunch for that. We are also running a high load twitter client. The tornado beta is running at http://beta.tweete.net

We may have some interesting notes to share.

Cheers :)

David 

Elias Torres

unread,
Jan 11, 2010, 8:25:43 PM1/11/10
to python-...@googlegroups.com
Ben,

I was having issues running multiple end-to-end tests like that one due to socket left open.

http://groups.google.com/group/python-tornado/browse_thread/thread/867cfb2665ea10a9/6778dd6654c5acda

This is what I did. Did you need to do this? How do you run your tests? python-nose?

http://groups.google.com/group/python-tornado/msg/6778dd6654c5acda

-Elias

Ben Darnell

unread,
Jan 11, 2010, 8:47:13 PM1/11/10
to python-...@googlegroups.com
Hmm. I guess we don't actually have more than one test that starts an
http server in the same module. Most of our tests are on lower-level
components, so it's just a few end-to-end sanity checks that need to
open a port. Most of our tests are actually still using django's test
runner, and the tornado-based tests are currently separate files that
call unittest.main(), and we just run them all with a shell script.
Adding an explicit way to release the HTTPServer's port seems like a
good idea.

-Ben

Bret Taylor

unread,
Jan 12, 2010, 12:48:14 AM1/12/10
to python-...@googlegroups.com
Yes, sorry. I will try to make time to take a look at this issue on Wednesday.

Bret

XiaoXiao Qin

unread,
Jan 12, 2010, 4:19:07 AM1/12/10
to python-...@googlegroups.com
HI, All,

I would like to find the tornado function which can get the client
IP , HTTP_USER_AGENT(browser type) and more information from client
side. So please give me some hints.

Thanks.

Steven

Matthew Ferguson

unread,
Jan 12, 2010, 4:44:25 AM1/12/10
to python-...@googlegroups.com
You can find the request information and subsequently the HTTP headers in self.request (an HTTPRequest object). So....
self.request.protocol
self.request.host
self.request.method
self.request.uri
self.request.version
self.request.remote_ip
self.request.body
self.request.headers (this is a dict with all the HTTP headers)

So, for example, if you want to get the User's browser information, you would use self.request.headers.get('User-Agent', None) (the none is the 'default' if User-Agent is not set, you should always be prepared for a header to not be set by a browser)

Hope that helps a little. If you're really new to Python, Django may be a good place to start, as some of the pieces of Tornado are 'Djangoic' and Django itself is a great intro to Python web programming and has great documentation and a great community. If the GFW doesn't block Django for its use of magic.

XiaoXiao Qin

unread,
Jan 12, 2010, 4:49:58 AM1/12/10
to python-...@googlegroups.com
Thanks, I got, BTW, thanks GFW :)

Steven

Message has been deleted
Message has been deleted

Sergey Konozenko

unread,
Jan 12, 2010, 10:17:41 AM1/12/10
to python-...@googlegroups.com
Господа Солохины,

Would you be so kind and take your business somewhere else? It does not make much sense to spam a developers forum.


2010/1/12 GOOGLE <a.sol...@gmail.com>
http://www.tech-faq.com

2010/1/12 David P. Novakovic <davidno...@gmail.com>



--
Человек, убегающий от ужасов нищеты, должен вовремя оста-
новиться, чтобы по инерции не попасть в кошмар богатства.
в Рядом с богатством нищета вдвойне тяжела.Хвала Аллаху, молитвы и да пребудет Его пророков и посланников
Исламистского повестке дня все, что вы можете думать о
Мы успешно завершили эту программу, в том числе различные функции, представляющие интерес для братьев-мусульман по всему миру
Они могут теперь услышать призыв на время молитвы и знания, особенно в не-мусульманин, который не снимается в ушах.
в Тот, кто хотел и не смог разбогатеть – должен подальше держать-
ся от состоятельных людей, чтобы не травмировать себя.Самые несчастные среди бедных те, у кого жадные богатые
родственники. Из-за них им не только не помогают другие, так
еще от них ждут помощи. Мало того, на них еще и распростра-
няются недоверие, зависть, злость, ненависть…
МЫ ОБЯЗАТЕЛЬНО ВАМ ОТВЕТИМ В БЛИЖАЙШЕЕ ВРЕМЯ .НЕ ПРИСЫЛАЙТЕ СПАММ  ВАЖНЫЕ ПИСЬМА ОТМЕЧАЙТЕ ЗВЕЗДОЧКОЙ.  С УВ   СОЛОХИН АЛЕКСАНДР
СОЛОХИНА ТАТЬЯНА ДРУГИХ В ЭТОМ ПОЧТОВОМ ЯЩИКЕ НЕТ! ЭТО ЭЛЕКТРОННЫЙ ЯЩИК СЕМЬИ СОЛОХИНЫХ!

Reply all
Reply to author
Forward
0 new messages