Timers, Polling vs Pushing, Recursion, and Threads

[JasonG]

After searching through this group, I have still not found a complete
explanation on all this stuff in the subject line but it seems to me
that this would be valuable in a lot of applications. I mean, I can
easily get this to "work" (a few different ways), but the goal is not
just to get it to "work", but rather to work well and scale well. I
would really appreciate some experienced (or Google) answers here. If
you are interested in this topic and have a few minutes, please follow
along and correct where I have made false assumptions.

For simplicity, let's narrow this down to one common use-case. Let's
say we need to build a simple chat-room application. Everyone logged
on will see a scrolling view of what eveveryone types.

Let's start by assuming a polling model. In this case, we know we are
going to have to go periodically back to the server to get the next set
of chat messages. So we need a Timer. Now - the first question is -
do we (A) run a Timer every x milliseconds, or do we (B) have the async
callback re-activate the timer?

In (A) we would use the scheduleRepeating(x) method. This guarantees
that unless there is a script error, that polling event will be
executed every x millis. Now, since the Timer is not going to wait for
the asynchronus callback, I am assuming that if x is too small there
could be problems with asynchronus responses tripping over each other
(confirmation anyone?).

In (B) we would just use the schedule(x) method. In this case, the
event only fires once. Whenever the asynchronus response is called
(either onSuccess/onFailure) it would then call the schedule(x) method
again. Now we have fixed the problem of asynch responses tripping over
each other, but now we are "logically" in a sort of "recursive loop"
that we never get out of. My question here is, does GWT (or JS for
that matter) have a concept of a stack that would get us in trouble
here? Or will the callback resources, including reference to the Timer
be freed up as soon as the onSuccess/onFailure method completes? Some
help here please - we don't want our chat room to be a browser memory
hog.

Next, let's assume a pull model. Now we will be doing some thread
blocking on the server side. I noticed a few people mention this
approach but it sounds a little dangerous to me because it doesn't
scale. You will be perpetually tieing up one app-server worker thread
for each user on the app. So the question is, should this be avoided
altogether? Or has anyone considered or implemented a combination of
polling and pushing? I guess there is always going to be a trade-off
between scalability and asynchronous response to server-side events.
Any enlightening thoughts on this?

Thanks!

[bruciadmin]

Okay, on the last point you mentioned a reluctance in keeping a server
thread for a single users usage. In my mind, a thread is only a big
load on the server if:

1.) It is utilizing a large amount of processing power
2.) It is consuming a large amount of memory

Assuming that the notification threads are waiting for someone to hit
them are just sleeping for 99.999% of the time they exist, the
utilisation will be next to nothing. The memory these threads incur
should again be almost nothing, as they shouldn't be creating many new
objects for their usage.

One other thing which might be an issue is a limit on the number of TCP
connections the server can have at any one time - and whether that puts
a load on the intermediate proxies, routers and firewalls - but I would
guess that is not a huge issue.

As for pushing, well even if that was once a viable option, nowadays
you would be crazy to every try initiate a TCP connection back to a
client browser - ever. The number of hurdles to go through on an
intranet application is huge, let alone an internet deployed
application. You'll notice that peer-to-peer applications accept
incoming connections from the push mentality, but in order to get them
working you have to make sure that client firewall applications accept
them and any firewall gateway devices not only allow the incoming
connection but also ensure the route the connection to a given desktop.
In my experience, your average user will not be inclined to make those
changes to their environment to suite your application.

So I'd try keep in the polling mentality as a way to get this done.
You're correct that there aren't any complete guides on this topic -
we're all waiting on someone like yourself with enough interest to so
do some tests and write one. Then you could wrap your solution up in a
Jar file with a GWT module definition and put it online and we'll all
use it: )

[JasonG]

Thanks bruciadmin.

A point of clarification on what I referred to as "push". I was just
referring to the usage of Object wait and notify. I know it's not a
true server to client push - which, as you mentioned, would be crazy
with the way browser-server technology works today. But it isn't
really a poll model either, since it waits for new information to be
available before responding. This results in a behavior that very much
resembles that of a "push" of information.

Thanks again for the response.

[bruciadmin]

If you're going to use the polling model with the timer just repeating
the calls to server you may have, as you mentioned, issues with the
responses tripping over one another. One way to get around that is to
use sequence numbers so that at least the client is able to reconstruct
the order of events properly (you can use datestamps but I understand
they are actually a bit of an issue at the moment - and watch out for
longs). I guess it also depends on what information is being returned
from the server. If the information returned is a complete model
update, you can disregard any previous information which hasn't arrived
yet. If however, the information coming back from server is incremental
and depends on previous information, the client has no other choice but
make special calls if the previous ones didn't arrive. For a chat
application, your asynchronous updates are probably going to
incremental bits of information (parts of a conversation) and will have
to be able to reconstruct them in order. One upside to the client
javascript being single threaded is you don't need to worry about
asynchronous callbacks clobbering shared objects as you would in a
multi threaded environment. Just some food for thought - yet another
small titbit which isn't the complete example you're after: )

[Scott Blum]

Hi Jason,

I think you'll find that in practice, blocking threads on the server
is the way to go. I know for a fact that a bunch of clients
repeatedly polling on a timer is going to drive up your bandwidth
usage, and I believe it will probably be more processing time as well.

There are probably some subtleties to work out, but I know of some
productions apps that have successfully used blocking server threads.
At some point, I'd really like to put together a nice chat room sample
that would show off the technique.

Scott

[Mike Aizatsky]

Jetty 6 supports polling efficiently:
http://docs.codehaus.org/display/JETTY/Continuations

[Scott Blum]

Fantastic link, Mike. Thanks.

Scott

[djsch...@mitre.org]

Would it be possible to configure an XMPP server to somehow work with a
browser? We've had success using it for pushing with a client side app
and while I know it is designed for web based use, I wasn't sure if
there was a practical way to adapt it to do that.

[JasonG]

Mike - Thanks for this link. I read up on this and I have to say the
problem description is right on the button. Anyone who didn't
understand the problem from my original description should read the
Background section of the Continuations page.

But - the proposed solution is not quite what I am looking for. First,
although the author states it is portable, in reality that does not
seem to be the case. It is using one set of logic for Jetty, and a
different set of logic for all other application servers. Second, the
mechanism relies on the use of Runtime Exceptions in order to function.
I know all you Java veterans out there just cringed. For those of you
who are new to Java, it is a best practice to reserve exceptions only
for exceptional cases - not to leverage them as part of the "normal
flow" of a method.

This being said, unless there is a change to the servlet spec to help
solve this problem - this Jetty Continuations "hack" may be the only
true solution to the problem. So - if you are in a bind and need to
both high scalability and quick responses, then give it a shot...
especially if you are using Jetty anyway. But I can't say I would
recommend this for a general purpose library.

I will keep looking for a solid portable way to solve this problem. In
the meantime, it is likely that the best general-purpose solution will
involve a combination of a configurable Timer on the client and a
configurable wait time on the servlet. These two values could be tuned
depending on the expected use of the application (e.g. 50 concurrent
users vs 10,000 concurrent users). It definitely not a perfect
solution, but given the current Servlet spec, it is probably the best
general-purpose solution.

[vetman]

I am thinking about this problem in a different way.. Maybe totally
wrong but lets see what you guys think..

I have not read the article but I will. Polling seems like the wrong
thing to do. I'm sorry. Having the server get hit by these clients
when it has nothing to say (ie no new/changed information) is just not
right. Also the clients don't update as the data gets changed unless
I poll the server a frequently from each client and no one wants to do
that....

Generally when I am faced with this problem, and I am hoping to find a
solution to this in the GWT world, I use a publish/subscribe mechanism
in which all the clients that are interested in a particular even (a
change of data one the server for instance) receive a notification,
probably this notification contains the data, that would be best.

In this mechanism the registration of the "interested party", the
client with the server contains a pointer back to the client. This
relationship between the client and server sounds a lot like original
set up that we have in GWT. The server has a pointer to the asych
method, which the serverImpl sends data to and then client when
receiving it processes it and updates the GUI. So it seems like we have
everything, if not most of what we need.

Lets carry this thinking a little bit farther along.. So when the event
happens all I need to do is call that method defined in the interface
(the same method that was called by the client) and have that method
send its data to the asych interface, like it normally would, instead
of me, the caller. Then wow we would have it..

See we have the routine that we need to trigger, and we know when to
call it, we just have to have it return to the client aysch connection
and not us. So how can we write to that asycn mechanism.. ...

Any thoughts?

[vetman]

Of course if I think about this I think that it is not possible.. web
unlike RMI and such are different.. but maybe someone out there can
prove me wrong.. I would be happy to be wrong on this .. but..