Best way to implement asynchronous loop

[Amit Kumar]

I have a basic question. 

I want to interate through a a JsonArray in a loop and perform some async functions for every element in the loop. I am not sure what is the best way to achieve this given that my module in which this code resides is not a worker module.

Say I have method getAllFoo() which needs to retrieve all 'Foo' records from the database and return a JsonArray. 

getAllFoo(Message<JsonObject> message) {

    // allrecords = send a query to jdbc persistor to fetch all records from a table given a criteri

    final JsonArray results = new JsonArray();

    for (record : allrecords) {

        getMoreDetails(record, new Handler<JsonObject>() { 

            public void handle(final JsonObject detailedRecord) {  

                  results.add(detailedRecord)

            }

        });

    }

    sendOK(message, new JsonObject().putArray(results));

}

getMoreDetails(record) { return fetch-more-details-from-database; }

In the above sample, I want to ensure that sendOK is invoked ONLY after results have been fully populated. Again, I am not sure whats the best way to do it.

[Amit Kumar]

final AtomicInteger counter = new AtomicInteger(0);

increment and check for the last element being handled within the callback and then sendOK. - This is what I have at the moment. But not sure if I like that.. I am sure there is a better way and I am not aware.

[Jordan Halterman]

You should check out CountingCompletionHandler:

https://github.com/eclipse/vert.x/blob/master/vertx-core/src/main/java/org/vertx/java/core/impl/CountingCompletionHandler.java

That implementation is for Vert.x core but the same pattern is applicable to these types of problems. The other option is doing it sequentially with an iterator, but that obviously defeats the purpose of the asynchronous calls.

Jordan Halterman

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Stream Liu]

You could get some answers from mod-Rx,
this is pattern about asynchronous programe, i think vert.x have a
official mod to this pattern.

Well, if you have experience of NodeJS, you should be feel free :)

PS: i wanna paster some urls that mod-rx mod-promise, but i could not
open github for now.

[Jez P]

I agree - this looks like a job for rxJava to me, wrap your allrecords in an observable, use the map function for the processing and subscribe to the resulting observable (and for each next record you do the add), you can add a handler for completion which calls the sendOK function. You will of course need to think about what you do in the event of an error condition, but again you can add an error handler to your subscriber. 

I don't think you need mod-rxvertx for this, just core RxJava.

[petermd]

This is a pretty common Rx pattern. This is the basic structure for executing all detail calls in parallel

db.getAllRecords()

   .flatMap { r -> getMoreDetails(r) }

   .toList

which will return an Observable<List> that will contain all the resolved detail records (or an Error if any call fails)

btw mod-rxvertx doesn't provide any additional abstraction it is just a set of wrappers for the VertX API to turn any async call into an Observable - eg to access the JDBC persistor via the EventBus. not sure why you wouldn't want to use it?

-Peter

[Jez P]

Sorry, wasn't intending to suggest that he wouldn't want to use it, just that for changing the loop he didn't necessarily require it.

You're right though, it gets even nicer using the mod, especially carrying out the initial db call asynchronously too. 

[Amit Kumar]

It looks like there is a suggestion for Rx Java usage. Honestly, I have not experimented with it till now. In fact, my question was also intending to get an answer for "how can my nested async blocks of code" be made more readable and clear. Looks like the design pattern RxJava suggests, will help me there as well.

Thank you all for your help.

[Mikael Karon]

You can also take a look at https://github.com/englishtown/vertx-mod-when/ and https://github.com/englishtown/when.java/ for an alternative to rx.

/ Mike

[Frank Reiter]

On Wed, Feb 19, 2014 at 11:13 PM, Mikael Karon <mikael...@gmail.com> wrote:

You can also take a look at https://github.com/englishtown/vertx-mod-when/ and https://github.com/englishtown/when.java/ for an alternative to rx.

It looks like this makes for some very readable code.  Do you know whether this is a layer on top of vert'x's very careful thread management model or whether it spawns its own thread, potentially reducing the efficiency of  vert.x?

Frank.

[Mikael Karon]

No extra threads at all AFAIK. I think Adrian (author of when.java) is on the list so he'll tell me if I'm wrong ;)

--

[Adrian Gonzalez]

Correct, no extra threads.  

I'm in the process of porting when.js 2.8.0 to when.java.  From the outside it looks pretty much the same, but the internal implementation feels a lot cleaner to me.  Plus it adds monitor/reporting support.

Let me know if you have any questions about when.java and promises.

[Frank Reiter]

On Feb 22, 2014 10:29 PM, "Adrian Gonzalez" <adrianlui...@gmail.com> wrote:
>
> Correct, no extra threads.  

That's what I was hoping to hear.

> I'm in the process of porting when.js 2.8.0 to when.java.  From the outside it looks pretty much the same, but the internal implementation feels a lot cleaner to me.  Plus it adds monitor/reporting support.

Sounds great.   Thanks for making this available!

Frank.