Tornado websocket throughput capacity

[Debby Mendez]

I have found a number of benchmark reports that cover the request rate that can be handled by Tornado, but haven't seen anything that deals with the total data rate capacity - ie bytes per second that can be read in or written out.  I am interested in both websockets and http as we use both.

I ran some tests using a simple websocket handler (below) as the server.

The client simulates the approximate size and timing of messages that we will have in our actual service:it sends 200kbytes every 10 seconds on each websocket, staggered so not all connections are sending at once.

At about 5 mbits/sec total (each way) the python process CPU time hits 75% on my freebsd 3.2GHz system.  There are 30 websocket connections.  CPU usage grows proportionately with data throughput so I believe this is headed toward a CPU bottleneck.

(The same test on my Windows 2.8GHz shows python at 30% CPU.)

We have another server (written in c++ using kqueue) that can handle much higher throughput on the same hardware.

My question is: am I getting expected throughput, or is something wrong?  I am pretty sure tornado is using kqueue and not select, as "top" shows the server in "kqread" when idle.

The server:

import tornado.ioloop

import tornado.web

import tornado.websocket

import settings

class WSHandler(tornado.websocket.WebSocketHandler):

    def open(self):

        print ("WebSocket opened " + self.get_websocket_scheme())

    def on_message(self, message):

        self.write_message(u"You said: " + message)

    def on_close(self):

        print ("WebSocket closed")

appsettings = { 

    "debug" : True, 

    "static_path" : "./",

    } 

application = tornado.web.Application([

    (r"/ws", WSHandler)

],

    **appsettings)

if __name__ == "__main__":

    application.listen(7780)

    ssloptions = {"certfile": settings.SSL_CERT_FILE, "keyfile": settings.SSL_KEY_FILE } 

    application.listen(7443, ssl_options = ssloptions)

    tornado.ioloop.IOLoop.instance().start()

Thank you for any insight.

[Ronan Amicel]

Is your benchmark using SSL or not? Did you compare both cases (with/without SSL)?

-- 

Ronan Amicel

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Debby Mendez]

Thanks Ronan.  I'm using SSL, which is how we expect to deploy.  I'll try without ssl and report back.  Any reason to suspect that would make an order of magnitude difference?

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

[Ben Darnell]

The most likely problem is that Tornado's websocket masking function is slow.  It's a pure-python loop over all the characters in the message.  There's a pull request to add an optimized C implementation of this function (https://github.com/facebook/tornado/pull/579), but I'm reluctant to add C extensions to Tornado unless I can figure out how to make them optional (I'm also curious about using cython instead of manual C extensions).  Pypy should make a big difference on this function as well.

-Ben

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.

[aliane abdelouahab]

Hi Ben, Tornado is not listed in the compatibility page, 

https://bitbucket.org/pypy/compatibility/wiki/Home

[Debby Mendez]

I separated the unmasking operation into its own method and ran the profiler.  That looks like the bottleneck.  Of the options you suggested:

- C extension

- Pypy

- cython

what are the pros and cons?  Pypy sounds like it might have scalability benefits all around, but is that proven/stable with Tornado?  Do you think I would get the same performance benefit from either the C extension or Pypy? The C extension might be less risky since it's an isolated change, but would be a maintenance hassle when we upgrade Tornado?

[Debby Mendez]

Without ssl CPU usage is slightly lower, but only a few percentage point difference

On Wednesday, July 10, 2013 6:05:15 PM UTC-4, Ronan Amicel wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.

[Ben Darnell]

On Thu, Jul 11, 2013 at 9:51 AM, Debby Mendez <de...@josesmex.com> wrote:

I separated the unmasking operation into its own method and ran the profiler.  That looks like the bottleneck.  Of the options you suggested:

- C extension

- Pypy

- cython

what are the pros and cons?  Pypy sounds like it might have scalability benefits all around, but is that proven/stable with Tornado?  Do you think I would get the same performance benefit from either the C extension or Pypy? The C extension might be less risky since it's an isolated change, but would be a maintenance hassle when we upgrade Tornado?

I like pypy because it's a big speedup across the board, but I've never used it in production. There are still some compatibility issues with third-party modules, and there's still (last time I checked) a bug that can cause problems if you're using it with ssl (https://bugs.pypy.org/issue1238).  

A C extension (whether hand-written or cython) is a safer, more localized change.  The main drawback is that if there is a C extension, you need to have a compiler and the appropriate python headers installed (this was a recurring obstacle for new users before Tornado 2.0, when pycurl was required).  I'd like to make any C extension optional to make it easier for new users to get started (although once you can build extensions there are no problems with upgrading).  Cython is attractive because you can generate the C extension from an annotated python implementation so you don't have to worry as much about them getting out of sync.  

-Ben

[Debby Mendez]

Tried the c extension - big success!  I had to do some fiddling to get it to work with python3 and tornado 2.4.  I think this is the order of magnitude difference I was looking for!  The same test that was running at 75% cpu is now down to 8%.

As for making extensions optional, could there be a parameter to the build process that indicates whether to build extensions, and then the python code can catch any import exception and fall back to a python implementation (as in the pull request you directed me to)?  I'm relatively new to python so I don't really know the build/install conventions.

I may need to come back for more help if I find additional bottlenecks, but for now, thank you so much.

[Ben Darnell]

On Fri, Jul 12, 2013 at 3:29 PM, Debby Mendez <de...@josesmex.com> wrote:

Tried the c extension - big success!  I had to do some fiddling to get it to work with python3 and tornado 2.4.  I think this is the order of magnitude difference I was looking for!  The same test that was running at 75% cpu is now down to 8%.

As for making extensions optional, could there be a parameter to the build process that indicates whether to build extensions, and then the python code can catch any import exception and fall back to a python implementation (as in the pull request you directed me to)?  I'm relatively new to python so I don't really know the build/install conventions.

It's sort of possible, but the tool support isn't great.  It's easy to add extra arguments to "python setup.py install", but if you're using tools like a pip requirements file there are additional constraints.  I think we could do it with a setuptools "extra", although I believe extras are off by default:  you'd have to do "pip install tornado[fast]" to get the version with the extension module; I'd prefer to build it by default and allow "pip install tornado[slow]" for users without compilers available.  

-Ben

[A. Jesse Jiryu Davis]

In PyMongo's setup.py we always try to build our C extensions but, if that fails, catch the error and print a warning:

https://github.com/mongodb/mongo-python-driver/blob/master/setup.py#L121

PyMongo can run without the extensions, slowly. The implementation in setup.py could probably be prettier. I like the idea of tornado[slow] to avoid even trying to build extensions if the user knows the build will fail.

[Naoki INADA]

I've created wsaccel for accelerate websocket performance.

https://github.com/methane/wsaccel

https://pypi.python.org/pypi/wsaccel

2013年7月11日木曜日 2時15分06秒 UTC+9 Debby Mendez: