Asynchronous Control Flow

[Alexander Praetorius]

Hi,

I'm reading the list for quite some time now and in regular intervals people ask about it and usually get pointed to the "async" library followed by some other options.

I wonder if there are nice up-to-date articles or tutorials that compare the different paradigms or options of doing so.
Originally coming from a non-programming background, i was used to think in:
=> network planning technique
It's a diagramming language to plan and optimize processes.

Personally, I quite like the examples i sometimes saw that made use of "co" and "generators", but I have a hard time imagining all the different cases and how to compose them.

I always imagined some nice readable way of specifying dependencies of "serial" and "parallel" pars of the process and how they "fork" and "merge back into each other.

Can that be done using "streams" or does that not make sense?

I have some troubles wrapping my head around the topic in general.

[Matt]

On Sat, May 16, 2015 at 7:19 PM, Alexander Praetorius <d...@serapath.de> wrote:

I always imagined some nice readable way of specifying dependencies of "serial" and "parallel" pars of the process and how they "fork" and "merge back into each other.

I'll let someone else answer the other aspects of your question, but for this part, async.auto I find is awesome (it's my most-used part of the library).

You basically specify things that either have no dependencies, or depend on previous data/functions. Here's an example (real world) usage from our app:

        async.auto({

            complete_task: function (cb) {

                if (!req.body.top_performer) return cb();

                notifications.complete_task("company", employee.company_id, "top_performer_badge", cb);

            },

            employee: function (cb) {

                api.model.employees.update(employee, req.body, cb);

            },

            roles: function (cb) {

                api.model.roles.get_by_company_id(employee.company_id, cb);

            },

            ideal_calc: ['roles', function (cb, data) {

                async.each(data.roles, function (role, cb) {

                    iccalc.perform_for_role(role.id, cb);

                }, cb);

            }],

            match_recalc: ['ideal_calc', function (cb) {

                match_recalculator.recalculate_for_company(employee.company_id, cb);

            }],

        }, function (err) {

            if (err) {

                console.error("Error updating employee: ", err);

                return res.send(500);

            }

            return res.send(200);            

        })

As you can see there, ideal_calc depends on 'roles' and match_recalc depends on 'ideal_calc'. The rest can be executed in parallel, and the async library takes care of that for you.

Hope that helps,

Matt.

[Alexander Praetorius]

@Mat thx :-)
I was not aware of "async.auto".
But I still wonder how this can be composed to scale.
The programs I write are executed in different contexts...

• global cli tool
• local cli tool
• script
• npm script
• browserify
• piping into them from the cli
• require(..)ed as a module
• sending UNIX signals to them
• making them listen to all kinds of servers or processes or sending them data

I don't have much experience with using the cluster module or in general orchestrating many node programs, but i guess all these ways might need special treatment.


I can see how with "async.auto" my control flow can be organized, but there are many techniques which will help me to do that.
Where I really have trouble is how to compose that when my program(s) grow(s) combining the ways above.


Once I saw a short tutorial that combined "generators" with "co-routines" and it looked neat, but I dont know how well it works/looks once more complex things should be done.
On the other hand, my guts tell me, that doing all kinds of things in a "streaming" way would be a cool thing. But here too I have problems seeing how exactly that would work out, especially how the syntax would look like. Would that be calling ".pipe(...)" or would that be listening to certain events ".on(...)"?

Sorry, for being so very confused, but I kind of avoided this topic for a long time, because I did not feel the need yet to really approach it, but slowly I see how I need to learn about those kind of things :-)

[Matt]

Looks like you're trying to run before you can walk, to be honest.

Play a bit with async. Play some with Promises (most people recommend Bluebird for that). Learn what works and what fits with your mindset. Nobody can give you an all encompassing perfect answer here.

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CAN5%2BLUu1Yeohr4V97i4TOeif5am9CfriXKU8Ra49zt8sUq_bfQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Michaël Rouges]

It depends on your need, but you can also look to my https://www.npmjs.com/package/queue.io ;)

Michaël Rouges - https://github.com/Lcfvs - @Lcfvs

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CAPJ5V2YGn2jLB3aFi6%3DE6DjTFWy%3DOH%2B%2BDnKKz6E%2B%3Ds2FXDC9ZQ%40mail.gmail.com.