Initializing expensive resources in parallel?

[William Pietri]

Hi! I am trying to decide if Guice would help a problem.

I'm investigating a problem with startup times in a server app. At
launch, the app creates a root object with a few dozen services, with
inter-service dependencies injected manually. A number of these
services are very expensive (minutes to create, much RAM used).
Mapping out the dependencies, it looks like starting things up is
happily parallelizable, but sorting out the relationships and stages
manually is a bit of a pain.

That pain is the sort of thing that Guice was created to solve.
However, in the archives, I see mention that Guice single-threads the
creation of Singletons, which sounds like it would prevent me from
parallelizing service startups.

Is there some way I can use Guice to automatically bind together my
many expensive services in a way that takes advantage of the idle
cores on my servers?

Thanks,

William

[limpb...@gmail.com]

On Nov 18, 4:20 pm, William Pietri <will...@scissor.com> wrote:
> Is there some way I can use Guice to automatically bind together my
> many expensive services in a way that takes advantage of the idle
> cores on my servers?

My personal preference is to use a Service with a Multibinder.
http://code.google.com/p/guava-libraries/source/browse/trunk/src/com/google/common/base/Service.java

Service is a part of the forthcoming Guava project. It's early days,
and we're still trying to shake out problems with the API. I would
like to see it eventually grow into the lifecycle story for Guice. But
the we Guice committers don't all agree on this, and we're not ready
to standardize on the lifecycle API.

[Tim Peierls]

I've sketched an approach here that might work for you: http://pastie.org/707072

The idea is that ConcurrentSingletonScope, when the injector is
created, eagerly starts computing in background the instances it will
eventually provide. The background threads are in a cached thread
pool, which creates new threads when needed. This approach probably
will deadlock if there are circular dependencies -- but creating
circular dependencies is a bad idea to begin with.

The main routine in the code demonstrates, somewhat artificially, that
the provision happens in parallel. In practice, you would not use
explicit Provider instances in this way.

If this were to be adapted for production use, you'd want to add a
scope annotation and bind it to ConcurrentSingletonScope, so you'd be
able to say something like:

@ConcurrentSingleton
public class MyExpensiveStartupService {
@Inject public MyExpensiveStartupService(
Provider<MyExpensiveDep1> myExpDep1Provider,
Provider<MyExpensiveDep2> myExpDep2Provider) {
// Copy parameters to final fields.
}
@Inject public void start() {
// Do something expensive.
expensiveStuff();

// Now make use of my expensive-to-compute dependencies.
// They might already be available, but if not, we block until
they are.
MyExpensiveDep1 myExpDep1 = myExpDep1Provider.get();
MyExpensiveDep2 myExpDep2 = myExpDep2Provider.get();
makeUseOf(myExpDep1, myExpDep2);
}
}

--tim

[Tim Peierls]

Hmm, I'm not sure if the Guice internals are safe to access from other
threads during calls to scope(Key, Provider) -- maybe a Guiceru can
tell us. If not, this approach would have to be tweaked to ensure that
the submitted tasks all wait on a latch that gets opened as soon as it
is safe to proceed.

--tim

[Tim Peierls]

Actually, it's worse than that: Even with a latch, the Guice internals
are so aggressively single-threaded that this approach doesn't work at
all -- the trivial demonstration only works because of the uninjected
custom provider. What a shame.

Sorry if I got any hopes up. :-(

--tim

[William Pietri]

Tim Peierls wrote:
> Actually, it's worse than that: Even with a latch, the Guice internals
> are so aggressively single-threaded that this approach doesn't work at
> all -- the trivial demonstration only works because of the uninjected
> custom provider. What a shame.
>
> Sorry if I got any hopes up. :-(
>
> --tim

Thanks, Tim. You did get my hopes up, but that's ok; this year's
disappointments are next year's shiny new technologies.

Do you have an opinion on Jesse's proposed solution, the one involving
Guava? I don't yet know enough about Guice's internals to know whether
your discovery means that his approach would also founder on the same
rocks you uncovered.

Thanks,

William

[Dhanji R. Prasanna]

If you like you can try the experimental lifecycle module which takes an executor and calls a bunch of services in parallel. See this test for an example:

http://code.google.com/p/google-guice/source/browse/trunk/lifecycle/test/com/google/inject/lifecycle/ParallelizedBroadcastTest.java

Dhanji.

Thanks,

William

--

You received this message because you are subscribed to the Google Groups "google-guice" group.
To post to this group, send email to google...@googlegroups.com.
To unsubscribe from this group, send email to google-guice...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-guice?hl=.

[Tim Peierls]

On Nov 20, 11:24 am, William Pietri <will...@scissor.com> wrote:
> Do you have an opinion on Jesse's proposed solution, the one involving
> Guava? I don't yet know enough about Guice's internals to know whether
> your discovery means that his approach would also founder on the same
> rocks you uncovered.

I'm just trying to get my head around it now. I guess the idea is that
you would move all initialization activity into a start() method and
use the lifecycle stuff to call all the start() methods in parallel. I
guess it's a start -- no pun intended -- but in its current state, I
don't see how you would deal with interdependencies between
Startables, and it seemed like that was the main issue you were hoping
Guice could quasi-magically address: Service A needs both Service B
and Service C started before it can itself start.

What you'd like to be able to say is something like this:

class ServiceA implements Service {
private final ServiceB b;
private final ServiceC c;
@Inject ServiceA(ServiceB b, ServiceC c) { this.b = b; this.c =
c; }
public void start() throws InterruptedException {
waitForStart(b, c); // where waitForStart is a blocking
utility method provided the lifecycle machinery
expensiveInitialization(b, c);
}
...
}

Maybe something like this is already there and I just missed it?

During JSR 330 discussions, when I was nattering on about documenting
concurrency properties of injection, Bob Lee led me to believe that a
parallel injector would be really, really hard to do. So I don't think
we'll ever be able to do something like this:

class ServiceA implements Service {
@Inject ServiceA(Provider<ServiceB> bp, Provider<ServiceC> c) {
expensiveInitialization(bp.get(), cp.get());
}
...
}

and have the ServiceB and ServiceC constructions happen in parallel.

--tim

[Dhanji R. Prasanna]

On Tue, Nov 24, 2009 at 2:28 PM, Tim Peierls <tpei...@gmail.com> wrote:

On Nov 20, 11:24 am, William Pietri <will...@scissor.com> wrote:
> Do you have an opinion on Jesse's proposed solution, the one involving
> Guava? I don't yet know enough about Guice's internals to know whether
> your discovery means that his approach would also founder on the same
> rocks you uncovered.

I'm just trying to get my head around it now. I guess the idea is that
you would move all initialization activity into a start() method and
use the lifecycle stuff to call all the start() methods in parallel. I
guess it's a start -- no pun intended -- but in its current state, I
don't see how you would deal with interdependencies between
Startables, and it seemed like that was the main issue you were hoping
Guice could quasi-magically address: Service A needs both Service B
and Service C started before it can itself start.

In my lifecycle experiement, if you had only one thread running, then the dependencies are naturally expressed by the order of module installation. IMO that is an elegant and simple solution.

For parallelization, the idea is you would have a Startable that is injected with all the deps and sequentially call methods on them. You can have as many startables as you want parallel init streams.

Dhanji.

[William Pietri]

Hi, folks. Thanks for your comments!

Dhanji R. Prasanna wrote:
> If you like you can try the experimental lifecycle module which takes
> an executor and calls a bunch of services in parallel.

Tim Peierls wrote:
> [...] I

> don't see how you would deal with interdependencies between
> Startables, and it seemed like that was the main issue you were hoping
> Guice could quasi-magically address: Service A needs both Service B
> and Service C started before it can itself start.
>

Yes, that's the manual work I was hoping to avoid. Once you get past
some number of services, even though all the dependencies make sense
individually, it's a lot of head-scratching to put it all together properly.

From what you and Dhanji say, it looks my use case is either out of
scope or bleeding edge for Guice, so I'll pass for now. However, I still
may try using Guice after parallelizing the high-cost services internally.

> What you'd like to be able to say is something like this:
>
> class ServiceA implements Service {
> private final ServiceB b;
> private final ServiceC c;
> @Inject ServiceA(ServiceB b, ServiceC c) { this.b = b; this.c =
> c; }
> public void start() throws InterruptedException {
> waitForStart(b, c); // where waitForStart is a blocking
> utility method provided the lifecycle machinery
> expensiveInitialization(b, c);
> }
> ...
> }
>
> Maybe something like this is already there and I just missed it?
>

That would be nice. Actually, in my case, the services currently do all
their initialization at construction time, so I don't even need the
ability to wait, but I could see that others would.

> During JSR 330 discussions, when I was nattering on about documenting
> concurrency properties of injection, Bob Lee led me to believe that a
> parallel injector would be really, really hard to do. So I don't think
> we'll ever be able to do something like this:
>
> class ServiceA implements Service {
> @Inject ServiceA(Provider<ServiceB> bp, Provider<ServiceC> c) {
> expensiveInitialization(bp.get(), cp.get());
> }
> ...
> }
>
> and have the ServiceB and ServiceC constructions happen in parallel.
>

Hmmm... That's unfortunate, as that would be the ideal scenario for me.
If all of my dependent services are going to have duplicate code where
they wait for their dependencies to heat up, I'd love to pull that
duplication out somewhere, and the dependency injection framework seems
like the ideal place for that.

William

[Tim Peierls]

William,

It bothered me to be so close to a solution and not get it, so I tried
again, and this time I figured out how to do it.

See these snippets for details:

http://pastie.org/716802
http://pastie.org/716803
http://pastie.org/716809

The last of these illustrates how you might use it. This approach
works when the expensive startup code is in the constructor or in an
injected method. I wouldn't try it with circular dependencies.

--tim

[Tim Peierls]

I decided this was worth blogging about:

http://tembrel.blogspot.com/2009/11/concurrently-initialized-singletons-in.html

--tim

On Nov 26, 11:36 pm, Tim Peierls <tpeie...@gmail.com> wrote:
> William,
>
> It bothered me to be so close to a solution and not get it, so I tried
> again, and this time I figured out how to do it.
>
> See these snippets for details:
>

> http://pastie.org/716802http://pastie.org/716803http://pastie.org/716809