Trying to understand Mojo::UserAgent and Blocking parallel requests on the Mojolicious

[marcos rebelo]

Hi all

This is coming from reading the documentation, I didn't do any code.

I need to request multiple REST call from inside my GET handler on a Mojolicious server. 

I see 2 solutions:

1) Use the Mojolicious IO::Loop to do  the call

Looking to the documentation seemed to me that I can't do it with Mojo::UserAgent since the Mojo::IOLoop->is_running shall be true.

2) Create a Mojo::IOLoop and pass it to the Mojo::UserAgent. Like that I may block the call until all the request have finished. 

Can this be a solution?

Best Regards

Marcos Rebelo

--
Marcos Rebelo
http://www.oleber.com/
Webmaster of http://perl5notebook.oleber.com

[sri]

http://mojolicio.us/perldoc/Mojolicious/Guides/Cookbook#Backend_web_services

--
sebastian

[marcos rebelo]

Thanks,

Now I understood the delay feature. 

Really thanks.

May I suggest to add a link to the example from the page http://mojolicio.us/perldoc/Mojo/UserAgent the content was not clear.

Best Regards

Marcos Rebelo

On Thu, May 24, 2012 at 1:32 PM, sri <kra...@googlemail.com> wrote:

http://mojolicio.us/perldoc/Mojolicious/Guides/Cookbook#Backend_web_services

--
sebastian

--
You received this message because you are subscribed to the Google Groups "Mojolicious" group.
To post to this group, send email to mojol...@googlegroups.com.
To unsubscribe from this group, send email to mojolicious...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mojolicious?hl=en.

[Benjamin Hitz]

Not sure I quite get the 2nd example - but maybe it's just a toy?

You run 2 asynchronous (long running, presumably) searches.  

One of them finishes first (say python) and sends it's result to $self->render.

Then the second one finishes (say perl), but calls the same render method with the same json => {results => \@results}...    But whatever is listening will just overwrite the python search result with the perl one?

I am probably just not visualizing the the client correctly.

Ben

--

Ben Hitz 

Senior Scientific Programmer ** Saccharomyces Genome Database ** GO Consortium

Stanford University ** hi...@stanford.edu

[marcos rebelo]

Hi Benjamin

I'm playing around with a personal project that may serve as base to an Activity Stream. 

https://github.com/oleber/ActivityStream

In one query I will have multiple activities, and some of them will have the data near others will have the data hidden in some obscure system. 

For the Ones that have the data in some Obscure System I need to collect the data fast. The simpler solution is to access the REST interface in parallel for loading the data for that Activities.

As a First implementation I have a second 'Event Loop' that block the process (maybe is the best solution) that depends on HTTP::Async, and I would like to remove this restriction.

https://github.com/oleber/ActivityStream/blob/master/lib/ActivityStream/AsyncUserAgent.pm

The one done purely in Mojolicious allow me to do the tests better.

Lets see what the future brings.

Best Regards
Marcos

[Sebastian Riedel]

> One of them finishes first (say python) and sends it's result to $self->render.
> Then the second one finishes (say perl), but calls the same render method with the same json => {results => \@results}... But whatever is listening will just overwrite the python search result with the perl one?

That does not happen, the whole point of Mojo::IOLoop->delay is to synchronize the two.

--
Sebastian Riedel
http://twitter.com/kraih
http://mojolicio.us

[Benjamin Hitz]

I get it now. I looked at Mojo::IOLoop::Delay source.

$delay->begin increments a counter and $delay->end decrements. $delay->finish only fires when the counter goes back to zero or something.
I am not very used to event programming so I get easily confused.

Is there a race condition here if the first $delay->begin(callback())) finishes before the second $delay->begin executes? If not -- HOW DOES IT KNOW that you are going to run a 2nd (or further) ->begin(). I can see it working as long as there is one active "job"...

Probably I am just naive.

Ben

[Sebastian Riedel]

> Is there a race condition here if the first $delay->begin(callback())) finishes before the second $delay->begin executes? If not -- HOW DOES IT KNOW that you are going to run a 2nd (or further) ->begin(). I can see it working as long as there is one active "job"...

That can't happen, the event loop is blocked during that time.

[Benjamin Hitz]

Huh. I think I probably don't understand event loops. It's not my fault! I was raised in the wild by FORTRAN!

Are you interested in explaining the code in detail? Maybe over IRC? I am just curious; I probably won't even use it. I suppose if I spend enough time with it I would figure it out myself. Not sure I care *THAT* much.

Thanks,
Ben

[Sebastian Riedel]

> Huh. I think I probably don't understand event loops. It's not my fault! I was raised in the wild by FORTRAN!
>
> Are you interested in explaining the code in detail? Maybe over IRC? I am just curious; I probably won't even use it. I suppose if I spend enough time with it I would figure it out myself. Not sure I care *THAT* much.

*hides*

[Benjamin Hitz]

hahaha.. Cheers! It's really quite elegant code. I'll leave you alone and read http://en.wikipedia.org/wiki/Event_loop

Ben

[sri]

> I'll leave you alone and readhttp://en.wikipedia.org/wiki/Event_loop

If i knew how to explain event loops i would have written a guide by
now, but you're welcome to ask on IRC anyway. ;)

--
sebastian

[marcos rebelo]

The idea was grait, but I'm getting problems doing my tests. I'm willing to use Mojolicious as a Mock Server for my classes using the Asynchronous calls. To explain my problem, I did some sample code like:

use Mojo::Base -strict;

use Test::Most;

use Test::Mojo;

use Mojolicious;

my $t = Test::Mojo->new( Mojolicious->new );

$t->app->routes->get('/my_test')->to( 'cb' => sub { shift->render_json( [1] ) } );

my @calls;

my $delay = Mojo::IOLoop->delay( sub { push( @calls, 'finalize' ); Mojo::IOLoop->stop; } );

$delay->begin;

$t->ua->get( '/my_test', sub { push( @calls, 'GET /my_test' ); $delay->end; } );

# many more get calls prepared like before

Mojo::IOLoop->start;

cmp_deeply( \@calls, [ 'GET /my_test', 'finalize' ] );

done_testing;

In this code I see one thing that I don't like. The Mojo::IOLoop->start, but worst then that the Mojo::IOLoop->stop. 

Is there a simple way of removing Mojo::IOLoop->stop and substitute Mojo::IOLoop->start by something like:

while ( Mojo::IOLoop->as_something_to_do ) {

    Mojo::IOLoop->one_tick

}

Best Regards

Marcos Rebelo

--
You received this message because you are subscribed to the Google Groups "Mojolicious" group.
To post to this group, send email to mojol...@googlegroups.com.
To unsubscribe from this group, send email to mojolicious...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mojolicious?hl=en.

--

[Sebastian Riedel]

> Is there a simple way of removing Mojo::IOLoop->stop and substitute Mojo::IOLoop->start by something like:
>
> while ( Mojo::IOLoop->as_something_to_do ) {
> Mojo::IOLoop->one_tick
> }

The whole use case looks very wrong to me, but i will answer anyway since i might just be missing something.

perl -MMojo::IOLoop -E 'Mojo::IOLoop->timer(3 => sub {}); Mojo::IOLoop->start'

Mojo::IOLoop->start will automatically stop when there is nothing to do anymore.