Websocket lag with JS

[sulter....@gmail.com]

I have this weird issue when using JS webscokets with tornado (tornado being server). I've been battling with this for days, and I have no idea what is causing this, but I suspect it might have something to do with tornado.

The tornado server simply adds all connections to a list, and sends a message to them each x sec (0.01 sec). Each JS client simply changes some html code "onmessage". Short and simple code of both

server: http://pastebin.com/GtCypv7q

client: http://pastebin.com/MyxfTys4

The issue being: when the client connects via "localhost", and is obviously on the server machine, the JS updates as fast as expected (I can see each 0.01sec tick). When the JS connects trough my external IP (even on server machine), I get this weird "lag", like if JS only updated about ~1sec instead of the expected  0.01sec. I did some testing by making JS make a list of all the data, and it looks like the data isn't actually lost, but JS just updates each 1 sec instead of 0.01sec. 

[Serge S. Koval]

There are few problems with your implementation:

1. You're overloading IOLoop by huge number of callbacks - they will be added, but 

2. You're using sleep to pause callback - it will block IOLoop while it is sleeping

How to fix it:

1. Use PeriodicCallback to send messages every 0.01s

2. Don't use long blocking calls - it blocks whole application

Serge.

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Serge S. Koval]

Correction:

* 1. You're overloading IOLoop by huge number of callbacks - they will be added, but won't be executed immediately.

Also, not sure how it'll work in your case, as you're not starting IOLoop.

[sulter....@gmail.com]

Oh sorry, because of all the different version I've been trying out, I actually posted code for something that doesn't work. This is the working server code (JS still the same):

http://pastebin.com/n0P5575P

the only difference is the IO line, so instead of tornado.ioloop.IOLoop.instance().add_callback(main) , I'm using tornado.ioloop.IOLoop.instance().run_sync(main) , which makes it work.

Still the issue described before is there of course. And I would really appreciate if you could at least give me a hint of what might be causing that behavior. 

As of using sleep - I only use this in the test server application (the actual server is event based) for the sake of clarity, and I'm sure that it's not the source of the issue (?). Thank you for pointing it out though. 

[Serge S. Koval]

Why do you use `run_sync` in endless loop?

`run_sync` starts IOLoop, runs your function, waits for pending callbacks to be gone and stops ioloop. This is not right way to do it.

Try this gist and see if you still have the problem: https://gist.github.com/mrjoes/5438041

[sulter....@gmail.com]

Thank you again for your time! I'm afraid I'm not able to use your suggestion, despite really wanting to try. I was not able to run the function PeriodicCallback, getting various errors: ""AttributError: SelectIOLoop object has no attribute 'PeriodicCallback'", and others, depending on what I tried. Here is the code: http://pastebin.com/8hE2Vrwu

[Serge S. Koval]

What do you mean by didn't work? Can you share exception text? What is your tornado version?

http://www.tornadoweb.org/en/stable/ioloop.html#tornado.ioloop.PeriodicCallback

[sulter....@gmail.com]

Like I said, I get: "  AttributError: 'SelectIOLoop' object has no attribute 'PeriodicCallback'   " on the line of  tornado.ioloop.IOLoop.instance().PeriodicCallback(main, 10).start() .

I'm using version 3.0.1 with python 3.3.

[A. Jesse Jiryu Davis]

PeriodicCallback isn't an attribute of the loop, it's its own class. When the docs say "tornado.ioloop.PeriodicCallback" they mean it's in the ioloop module, not on the IOLoop instance:

periodic_callback = PeriodicCallback(my_function, 5)

periodic_callback.start()

[Serge S. Koval]

Somehow overlooked exception text.

Anyway, as Jesse said - it is class in ioloop module and not member of the IOLoop class.

[sulter....@gmail.com]

Thanks for your and Jesse's replay, unfortunately, as I expected, that wasn't the source of my main problem, and the behavior persists. Just for the record, this is the current code:

client: http://pastebin.com/zbx3VmzA

server: http://pastebin.com/kjPR2fwU

Just to explain it again: When I connect as localhost, javascript is all smooth, and you can see the numbers update at the expected speed (each 10ms). When I choose to connect with my external IP, the javascript updates about ~1/sec, acting all "laggy".

[Serge S. Koval]

Only thing that comes to my mind - Nagle delay: http://en.wikipedia.org/wiki/Nagle's_algorithm

Try doing `self.stream.socket.setsockopt(socket.SOL_TCP, socket.TCP_NODELAY, 1)` in `WebSocketHandler.open`. You need to `import socket` for this to work.

[sulter....@gmail.com]

I don't know how I can thank you enough! Finally the problem is solved! And I have a nicely working websocket server. 

[A. Jesse Jiryu Davis]

Interesting, Serge. Is this a Nagle peculiarity, that it's not applied to localhost by default? And do you think it should be disabled in tornado.websocket under all circumstances?

[Serge S. Koval]

It's applied, but delay is calculated from the packet roundtrip time, so bigger latency is - longer it'll take to flush buffers on sender side.

I think behavior should be controlled by the developer: by default system favors throughput, with option to favor latency.

For IM-like applications extra 500ms delay won't play any role, but for games it would.

Serge.

[A. Jesse Jiryu Davis]

Right, makes sense.

[Ben Darnell]

On Tue, Apr 23, 2013 at 2:09 PM, A. Jesse Jiryu Davis <je...@emptysquare.net> wrote:

Interesting, Serge. Is this a Nagle peculiarity, that it's not applied to localhost by default? And do you think it should be disabled in tornado.websocket under all circumstances?

I think the difference between localhost and other interfaces is the use of delayed acks, not Nagle.  The two features interact badly: http://www.stuartcheshire.org/papers/NagleDelayedAck/  Delayed acks are actually the bigger problem in this case, but it's not possible to disable delayed acks in a platform-independent way as it is for Nagle with TCP_NODELAY.  

It wouldn't be appropriate to turn on TCP_NODELAY for all websocket connections since it can be wasteful of bandwidth  (in this example with tiny packets the majority of your bandwidth is going to packet headers).  However, for applications where it does make sense to trade bandwidth for latency there should probably be a better way to turn it on than reaching in to self.stream.socket.setsockopt.  

-Ben