Best practice for concurrent loading

[Joseph Andaverde]

I'm curious if there are any established patterns for executing
multiple requests asynchronously before responding with a view.

[Joseph Andaverde]

After playing around for a while I temporarily settled on something
like this. Obviously, it has a lot of improvements that can be made.

app.coffee
----------------
...some stuff
app.get '/', data.load('tweets', 'messages'), (req, res) ->
....etc


data.coffee
------------------
parser = require './utility/atomParser.js'
rest = require 'restler'

fetchMessages = (cb) ->
rest.get('http://groups.google.com/group/kc-nodejs/feed/
atom_v1_0_topics.xml').on('complete', (data) ->
cb(parser(data).items))

fetchTweets = (cb) ->
rest.get('http://search.twitter.com/search.json?q=
%40joeandaverde').on('complete', (data) -> cb(data))

module.exports = {
load: (keys...) ->
that = this
return (req, res, next) ->
finished = []
res.data or= {}
keys.forEach (key) ->
that[key] res.data, (k) ->
finished.push k
next() if finished.length == keys.length

messages: (data, cb) ->
fetchMessages (items) ->
data.messages = items
cb 'messages'
tweets: (data, cb) ->
fetchTweets (result) ->
data.tweets = result
cb 'tweets'

[Dusty Burwell]

That's certainly interesting. Is the point to try to execute the two
requests for data simultaneously? Or just that you need to get two
sets of data and then act on them?

I usually just nest the calls with continuations. However, the Async
package adds some niceties for calling code in parallel or serially
and then calling a continuation. I do find it weird that you're using
a route middleware for loading data, but I can't say that that's
entirely incorrect.

By the way, there's a nicety in CoffeeScript that you can use a fat
arrow (=>) to bind 'this' in a lambda. Instead of:

that = this
return (req, res, next) ->
finished = []
res.data or= {}
keys.forEach (key) ->
that[key] res.data, (k) ->
finished.push k
next() if finished.length == keys.length

you could do:

v
return (req, res, next) =>

finished = []
res.data or= {}
keys.forEach (key) ->

this[key] res.data, (k) ->

finished.push k
next() if finished.length == keys.length

[Joseph Andaverde]

I placed it in the middleware for simplicity. I also just copied how
Andrew fetched pugs on pugbomb.me.
What's a better alternative?

[Dusty Burwell]

Middleware just seems like a weird place for it. I would have imagined
something more along the lines of:

app.get '/', (req, res) ->
fetchTweets (tweets) ->
fetchMessages (messages) ->
.. doWork with tweets and messages ..
res.render ...

- or with the async library -

app.get '/', (req, res) ->
async.parallel [
fetchTweets,
fetchMessages
], (err, results) ->
.. do work ..
res.render ..

[Joseph Andaverde]

Makes sense. What about....

http://railwayjs.com/ <--- maybe placing in a controller?

[ruzz311]

I saw an NPM package for control flow today @ Dailyjs.com
https://github.com/scriby/asyncblock

Maybe helpful for what you were trying to accomplish?