Same, concurrent web requests

[Ariel Raunstien]

Hey folks.

I have a web server serving several clients, and at certain points all the clients poll it with the same request (and for 90% of them the answer is the same, too).

While I do have caching in the application level, which serves consecutive requests very well, we fumble with really concurrent requests.

Is there a OOTB solution for this, or a known pattern I should be aware of?

It's probably worth mentioning that some of these request may take 10s of seconds (which is fine and dandy for our usage scenario, so shut up), so the "statistically it's not interesting" approach fails here.

Ariel Raunstien

+972-522-747408

http://www.QuickUnit.com/blogs/ariel

[Avi Pinto]

10s?? you really hate your users? :)

now seriously,

first request takes 10s to complete and while it is working you have other requests asking for the same resources?

if you have control over your client side you can raise a flag in the server in first request and tell all other requests to come back in 10s with the key that is going to store the result in the cache.

so the user's browser will issue another request to get the data when it is ready.

this is the simplest polling solution which might be sufficient here.

--
You received this message because you are subscribed to the Google Groups "altnetisrael" group.
To post to this group, send email to altnet...@googlegroups.com.
To unsubscribe from this group, send email to altnetisrael...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/altnetisrael?hl=en.

--

My Portfolio http://www.avipinto.com/
Other contact options: http://www.avipinto.com/contact

[Ariel Raunstien]

Wouldn't it be "better" to keep the connection open, and "wait" for the cache to fill with the requested result, and then just return it? The clients are expecting the data "in a little while" any way, so why not just wait?

Ariel Raunstien

+972-522-747408

http://www.QuickUnit.com/blogs/ariel

[Avi Pinto]

this is also a valid solution but a bit more complicated to implement,

look at SignalR  https://github.com/SignalR/SignalR it does just that

as i said, the solution i suggested is the simplest polling solution, since you know all requests will take up to 10s, telling the client to come back in 10s is not that bad.

note that all the other solutions also utilize scripting as the browser has to open a new connection if the previous one timed out

[Ariel Raunstien]

"10s of seconds" != "10s" 

:)

But I get your point.

Is anyone here making use of SignalR? In production? Anyone has a few words about its reliability?

Ariel Raunstien

+972-522-747408

http://www.QuickUnit.com/blogs/ariel

[Oren Eini (Ayende Rahien)]

We are using it, and it is quite nice.

Are you making use of 304 for the 90% of the common requests?

[Avi Pinto]

missed that one..

[Ariel Raunstien]

It's caching for multiple clients using the same data, so not, not really.

I was not using 304 explicitly, but I assumed that the caching provided by ASP.MVC did that. Is that the case?

re: SignalR - what did you consider was the "worst case scenario" with using it? Does it have a vibrant community?

So yes, this message was basically "I'm too lazy to Google shit".

Ariel Raunstien

+972-522-747408

http://www.QuickUnit.com/blogs/ariel

[Oren Eini (Ayende Rahien)]

Ariel,

Support 304 explicitly, start with that.

On the server, create the request with a locking cache.

That should give you more than enough behavior to replicate what you need.

[Ken Egozi]

Signalr is pretty awesome. And if you'll have problems with it, the guy running it work about five doors away from me so I might help prioritizing pull requests. 

To keep your current design in place for now until you move to actual push based updates, you should be keeping requests open using a async http handler with a reset event triggering the  End part of the pending requests when cache entry is set. That will keep the load of them number crunchers services as well as the poor web server. 

Ken Egozi

http://kenegozi.com/blog

[Avishay Lavie]

I agree with the above, your basic flow should be:

1. Request comes in
2. Data is cached? return it.
3. Data is not cached? fetch it (10s of seconds!) and return it.
4. Data is in the process of being fetched? wait for it to complete
(in an async manner so as not to hog the IIS thread) and then return
it from the cache.

For this you need a caching layer that would be aware of when it's
being read-through, and provide an async event when the data is ready.
With C# 5 async/await the last part should be pretty easy to do, but
if not you can always do it with async handlers.

Avish

[Dotan N.]

You have a first request that would populate cache and this request takes 10s of seconds.
You're asking, should next requests be held until the cache-populating request finishes.

If this is a traditional cache, next requests should also populate cache (think about Http cache server or CDN). But the common conception is that its not because first request take crazily long time related to number of incoming requests.

While you can do async with Node.js or SignalR, my first worry is that you can't predict or have an upper bound on the number of following requests or the number of 10s of seconds. It can be 10 or 10000. Unless you can shed more light on the numbers I'd say drop following requests and accept when ready.

In any case assuming this has many requests, this sounds like an offline worker kinda thing to me.

202 accepted, come back later.

[Ariel Raunstien]

Thank you all for the thoughtful answers.

Oren - would you override ASP's caching just to be able to return the 304s? What is the benefit?

Ken, Avish - thanks. MVC's AsyncControllers are really simple to use and this looks like the right approach (http://msdn.microsoft.com/en-us/library/ee728598.aspx)

Dotan - we have a small number of users (the extreme estimate is "dozens") and the time the query takes is between 5 seconds and sometimes 40 seconds. It is a client-side cache invalidation, and the user is not aware of the pending request. Regarding the "offline worker" - if I understand you correctly - you're talking about pre-chewing the data before the first request? We have that in store, and will address to it if it becomes a bigger issue than it is today.

Ariel Raunstien

+972-522-747408

http://www.QuickUnit.com/blogs/ariel

[Oren Eini (Ayende Rahien)]

Ariel,

I would handle this myself, it is very few lines of code, after all.

The benefit that you can rely on http caching, proxies, etc. It is also much faster to return 304 than even a cached reply.

[Dotan N.]

Yes, chewing. For dozens of users (I'm starting to get the feel there's no real issue regarding requests/sec) it might not be that attractive.

--

Dotan, @jondot