Async and Batch processing examples?

[Landon]

Hey There,

You make mention of the batch processing capabilities in the library-
but i'm scratching my head at a practical example.

Do you have anything I could look at?

Additionally, I'm not sure what's going on with your tornado async
example. Are you having the async callback from the AsyncHTTPClient
execute the function call? From the documentation, that's what it
seems like is going on, when it's really not necessary (you could use
FD or something in the ioLoop, for example.)

Thanks!

-L

[Josh Marshall]

 Hey Landon --

 

You make mention of the batch processing capabilities in the library-
but i'm scratching my head at a practical example.

Do you have anything I could look at?

So, batch processing is something that's used frequently in the JSON-RPC world -- I know it is supported by XML-RPC extensions, but I don't run into it as often. That being said, both the built-in xmlrpclib and my jsonrpclib client libraries support batch requests through the MultiCall class. It just basically means you "call" multiple functions in the normal way, and then tell it to send / recv the results at the end (in whatever manner the client library supports). A simple example with xmlrpclib is here (scroll down to "MultiCall"):

http://www.doughellmann.com/PyMOTW/xmlrpclib/

You shouldn't have to do anything to enable batch processing on tornadorpc servers -- it should just work.

 

Additionally, I'm not sure what's going on with your tornado async
example.  Are you having the async callback from the AsyncHTTPClient
execute the function call? From the documentation, that's what it
seems like is going on, when it's really not necessary (you could use
FD or something in the ioLoop, for example.)

On the asynchronous side of things, I'm just using Tornado's AsyncHTTPClient as an example asynchronous driver. I designed the asynchronous side of things to be parallel to Tornado -- that is, you put a decorator on the request method to indicate that it shouldn't close at the end of the method, and then it is up to the callbacks to actually send on the data when the async library call returns.

Any asynchronous library designed to work with Tornado SHOULD work in this situation -- so whether you are using Tornado's AsyncHTTPClient, or using the async MongoDB driver, you should be able to put it into the stack. My testing has been pretty limited, however, so I would love any feedback you have when working with it.

Thanks!

Josh Marshall