Just-in-time Providers for Guice

[Pascal-Louis]

Hi all,

We've had a long standing grudge against Guice (which we love in all
other respects!): the inability to have user-defined just-in-time
bindings.

Over the course of the long weekend, I've implemented this feature and
would love some feedback. Check it out on
http://eng.kaching.com/2010/05/just-in-time-providers-for-guice.html.

I would specifically appreciate feedback from the Guice team on the
process to integrate this patch into trunk. On a more tactical note, I
was unsure about which injection points to use, see the ??? comment in
the method
http://github.com/pascallouisperez/guice-jit-providers/blob/master/src/com/google/inject/internal/InjectorImpl.java#LID830.

Best,
PL

[Stuart McCulloch]

On 1 June 2010 11:12, Pascal-Louis <pascallo...@gmail.com> wrote:

Hi all,

We've had a long standing grudge against Guice (which we love in all
other respects!): the inability to have user-defined just-in-time
bindings.

Over the course of the long weekend, I've implemented this feature and
would love some feedback. Check it out on
http://eng.kaching.com/2010/05/just-in-time-providers-for-guice.html.

I would specifically appreciate feedback from the Guice team on the
process to integrate this patch into trunk.

[caveat: just my personal opinion as a fellow patcher...]

people usually open issues for feature requests / patches:

   http://code.google.com/p/google-guice/issues/list

as its much easier to track and review than email threads

I'd also suggest cleaning up the patch so it just contains
the absolute necessary changes for the new feature - the
current patch includes a lot of non-functional / formatting
changes which makes it hard to read and digest

On a more tactical note, I
was unsure about which injection points to use, see the ??? comment in
the method
http://github.com/pascallouisperez/guice-jit-providers/blob/master/src/com/google/inject/internal/InjectorImpl.java#LID830.

for ProviderInstanceBindingImpl it should be the injection
points of the provider instance - in the patch this is just a
wrapper around "jitProvider" so it has no injection points,
which means you could simply use:

   ImmutableSet.<InjectionPoint>of()

this should also remove the need to change InjectionPoint
to handle null superclasses - the InjectionPoint SPI should
only be used on classes (not interfaces) so it always hits
the Object check first - when you used the request key it
might be an interface, which I guess is why you added the
null check to InjectionPoint

btw, are you arranging for the "jitProviders" to be injected
so that they can also contain @Inject dependencies?

Best,
PL

--
You received this message because you are subscribed to the Google Groups "google-guice-dev" group.
To post to this group, send email to google-g...@googlegroups.com.
To unsubscribe from this group, send email to google-guice-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-guice-dev?hl=en.

--
Cheers, Stuart

[Christian Edward Gruber]

Can you talk through the use-cases for user-defined just-in-time
bindings? As far as I can see, using providers to obtain just-in-time
binding behaviour is highly dangerous. i can imagine some leegitimate
use-cases for it, but most of the time when people talk about such
things (often injecting a Provider as a workaround) they really are
trying to workaround Guice's lack of lifecycle support.

One of my deepest concerns with JIT bindings is the wonderful way in
which it moves your wiring errors JIT as well, which makes debugging
more difficult, and can result in learning about such errors in
production AFTER deployment, depending on the particular layout and
timings of your injections. So, powerful, but dangerous and there
might be more suitable ways to accomplish what you want.

What would you need just-in-time bindings for?

Christian.

> Best,
> PL
>
> --
> You received this message because you are subscribed to the Google
> Groups "google-guice-dev" group.

> To post to this group, send email to google-guice-
> d...@googlegroups.com.

[Christian Goudreau]

Here's the issue :

http://code.google.com/p/google-web-toolkit/issues/detail?id=4423

I received that before this message :D

Christian

On Tue, Jun 1, 2010 at 7:49 AM, Christian Edward Gruber <cgr...@google.com> wrote:

Can you talk through the use-cases for user-defined just-in-time bindings?  As far as I can see, using providers to obtain just-in-time binding behaviour is highly dangerous.  i can imagine some leegitimate use-cases for it, but most of the time when people talk about such things (often injecting a Provider as a workaround) they really are trying to workaround Guice's lack of lifecycle support.

One of my deepest concerns with JIT bindings is the wonderful way in which it moves your wiring errors JIT as well, which makes debugging more difficult, and can result in learning about such errors in production AFTER deployment, depending on the particular layout and timings of your injections.  So, powerful, but dangerous and there might be more suitable ways to accomplish what you want.

What would you need just-in-time bindings for?

Christian.

On May 31, 2010, at 11:12 PM, Pascal-Louis wrote:

Hi all,

We've had a long standing grudge against Guice (which we love in all
other respects!): the inability to have user-defined just-in-time
bindings.

Over the course of the long weekend, I've implemented this feature and
would love some feedback. Check it out on
http://eng.kaching.com/2010/05/just-in-time-providers-for-guice.html.

I would specifically appreciate feedback from the Guice team on the
process to integrate this patch into trunk. On a more tactical note, I
was unsure about which injection points to use, see the ??? comment in
the method
http://github.com/pascallouisperez/guice-jit-providers/blob/master/src/com/google/inject/internal/InjectorImpl.java#LID830.

Best,
PL

--
You received this message because you are subscribed to the Google Groups "google-guice-dev" group.

To post to this group, send email to google-g...@googlegroups.com.

To unsubscribe from this group, send email to google-guice-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-guice-dev?hl=en.

--
You received this message because you are subscribed to the Google Groups "google-guice-dev" group.

To post to this group, send email to google-g...@googlegroups.com.

[Pascal-Louis]

Chris,

We have a number of use cases in our system. Let me describe two
below.

1. We use the JsonMarshaller (http://code.google.com/p/
jsonmarshaller/) for convert POJOs to and from JSON. This library
provides you with marshallers.

EntityMarshaller<T>

which lets you marshall instances of T. You create such marshaller
using

TwoLattes.createEntityMarshaller(T.class);

We want to be able to tell to Guice "for literals matching
Marshaller<X> provide an instance using createEntityMarshaller(X)".

2. Proxy objects. We have objects such as Proxy<T extends
InterfaceFooBar> which provide memoizing capabilities to cache
expensive operations. We need to get these proxies whenever we want to
do meta-level operations on T. Having jit providers makes it trivial.

To your point about jit providers producing more run-time errors, I
beg to differ. In our system, we've had a number of cases where we
forgot to list a specific type for which wanted a marshaller... using
jit providers ensures we have implementations for all types. In this
sense, it provides additional safety and makes Guice even more
friendly.

In general, jit providers simply fill the void for user defined jit
capabilities. Guice has a number of those already @ProvidedBy,
@ImplementedBy, no arg constructor, etc.

Also, I do not see how lifecycle has anything to do with jit
providers. These two concerns seem orthogonal to me.

PL

> >http://github.com/pascallouisperez/guice-jit-providers/blob/master/sr...

[Pascal-Louis]

Stuart,

Thanks for the pointers and explanation on injection points. Will take
care of that.

PL

On May 31, 10:27 pm, Stuart McCulloch <mccu...@gmail.com> wrote:

> On 1 June 2010 11:12, Pascal-Louis <pascallouispe...@gmail.com> wrote:
>
> > Hi all,
>
> > We've had a long standing grudge against Guice (which we love in all
> > other respects!): the inability to have user-defined just-in-time
> > bindings.
>
> > Over the course of the long weekend, I've implemented this feature and
> > would love some feedback. Check it out on
> >http://eng.kaching.com/2010/05/just-in-time-providers-for-guice.html.
>
> > I would specifically appreciate feedback from the Guice team on the
> > process to integrate this patch into trunk.
>
> [caveat: just my personal opinion as a fellow patcher...]
>
> people usually open issues for feature requests / patches:
>
>    http://code.google.com/p/google-guice/issues/list
>
> as its much easier to track and review than email threads
>
> I'd also suggest cleaning up the patch so it just contains
> the absolute necessary changes for the new feature - the
> current patch includes a lot of non-functional / formatting
> changes which makes it hard to read and digest
>
> On a more tactical note, I
>
> > was unsure about which injection points to use, see the ??? comment in
> > the method
>

> >http://github.com/pascallouisperez/guice-jit-providers/blob/master/sr...
> > .

>
> for ProviderInstanceBindingImpl it should be the injection
> points of the provider instance - in the patch this is just a
> wrapper around "jitProvider" so it has no injection points,
> which means you could simply use:
>
>    ImmutableSet.<InjectionPoint>of()
>
> this should also remove the need to change InjectionPoint
> to handle null superclasses - the InjectionPoint SPI should
> only be used on classes (not interfaces) so it always hits
> the Object check first - when you used the request key it
> might be an interface, which I guess is why you added the
> null check to InjectionPoint
>
> btw, are you arranging for the "jitProviders" to be injected
> so that they can also contain @Inject dependencies?
>
> Best,
>
> > PL
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "google-guice-dev" group.
> > To post to this group, send email to google-g...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > google-guice-d...@googlegroups.com<google-guice-dev%2Bunsu...@googlegroups.com>
> > .

[limpb...@gmail.com]

PL,

On May 31, 8:12 pm, Pascal-Louis <pascallouispe...@gmail.com> wrote:
> Over the course of the long weekend, I've implemented this feature and
> would love some feedback.

Thanks for the patch. This has been a long-standing feature request -
originally reported in February 2007:

http://code.google.com/p/google-guice/issues/detail?id=49

We decided not to offer this functionality because it makes it much
more difficult to figure out where a binding comes from. Currently
it's possible to use simple tools like grep to find a binding. As soon
as we permit autobinders, you really need a runtime to find a biding's
source.

In addition, you can usually accomplish the same thing statically
without much problem. To satisfy your Proxy example, you'd write
something like this:

bind(new TypeLiteral<Proxy<Foo>>() {})
.toProvider(new ProxyProvider(new TypeLiteral<Proxy<Foo>>()
{});
bind(new TypeLiteral<Proxy<Bar>>() {})
.toProvider(new ProxyProvider(new TypeLiteral<Proxy<Bar>>()
{});

Yes, you will need one bind() statement for each distinct type you
ultimately inject. Yes, this is boilerplate. But it's a small amount
of obvious boilerplate that saves a lot of indirection.

One last note...

Restricting Guice's ability to add indirection is a usability
decision, not a technical one. I recently worked on a 20-developer
Guice project. In any group of 20 developers, you'll have a few people
like you and I: developers who are comfortable with reflection and
indirection, and who dislike boilerplate. And you'll have developers
who prefer structural simplicity and avoid frameworks where they
aren't necessary.

In the course of the project, a few rockstar developers created a few
fancy frameworks that nobody else could maintain. Independently these
were all fine, but overall the frameworks made it quite difficult to
predict what a body of code would do.

It's for this reason that Guice tries to avoid magic. Injections
require @Inject not because the framework needs them (see
PicoContainer) but because they serve as documentation. Just-in-time
bindings were a mistake because they can mask a missing binding.

Guice is a framework intended to scale well as the number of
developers on the project grows. It is for this reason that we must
sacrifice features that would work great for smaller projects.

Cheers,
Jesse

[Pascal-Louis]

Jesse,

From a team size perspective, I happen to be the VP of Engineering &
CTO of kaChing.com; We are a large team and our code base is huge. We
also happen to continuously deploy our code (see
http://eng.kaching.com/2010/05/applied-lean-startup-ideas-continuous.html)
and so proving, automatically, that our software is correct is a must.

What we ended up doing prior to have JIT providers, and which I've
seen done in other projects, is something along the lines of

ProxyModule(Class... classes) { this.classes = classes; }
void configure() {
for (Class c : classes) bind(TypeLiterals.get(Proxy.class,
c)).to(...);
}

where TypeLiterals is
http://code.google.com/p/kawala/source/browse/trunk/src/com/kaching/platform/guice/TypeLiterals.java.
As such, I don't buy the "grepability" argument of Guice. This is
simply not a fact.

In our experience, JIT providers reduce the complexity of injection by
making things much more natural. "Mashaller<Foo>? Sure, I can get
this!" Versus the "Did I bind it for this specific entry point? What
about this other entry point?". And yes, without them, you typically
find these problems at the very last moment when the software is
already in production.

PL

[Pascal-Louis]

Jesse,

I believe there is a disconnect in the way we approach jit providers.
We both agree that it reduces risk of missing bindings making Guice in
the wild safer and reduces boilerplate. However, where I implicitly
trust API users not to hang themselves, you want to avoid any
possibility for that.

The only use cases JIT providers are built for are supporting type
schemas as opposed to type literals. Informally defined, a type schema
is "a type literal possibly containing a wildcard (?)". So for
instance, taking the Proxy example of before, what we need to express
is

bindJustInTime(new TypeLiteral<Proxy<?>>() {}).toProvider(...);

Given that simplification, the JitProvider interface could be
simplified to

interface JitProvider<T> {
T get(Key<T> key);
}

which makes it quasi-impossible to mis-use it. It would be quite
simple to do a type schema instantiation check, i.e. check that a type
literal is a valid instance of a type schema, in order to implement
this simplification.

Would that address your concerns?

PL

On Jun 1, 9:29 pm, Pascal-Louis <pascallouispe...@gmail.com> wrote:
> Jesse,
>
> From a team size perspective, I happen to be the VP of Engineering &
> CTO of kaChing.com; We are a large team and our code base is huge. We

> also happen to continuously deploy our code (seehttp://eng.kaching.com/2010/05/applied-lean-startup-ideas-continuous....)

> and so proving, automatically, that our software is correct is a must.
>
> What we ended up doing prior to have JIT providers, and which I've
> seen done in other projects, is something along the lines of
>
>   ProxyModule(Class... classes) { this.classes = classes; }
>   void configure() {
>     for (Class c : classes) bind(TypeLiterals.get(Proxy.class,
> c)).to(...);
>   }
>

> where TypeLiterals ishttp://code.google.com/p/kawala/source/browse/trunk/src/com/kaching/p....

[limpb...@gmail.com]

On Jun 2, 7:41 am, Pascal-Louis <pascallouispe...@gmail.com> wrote:
> Would that address your concerns?

It would, but it still doesn't quite pay for its complexity.
Autobinders would let you replace something structurally simple but
verbose (a dozen or so repetitive bind() statements) with something
structurally complex but compact (some reflection to implement
JitProvider).

[Pascal-Louis]

Jesse,

In practice, the dozen or so bindings are more like 50 or 100; and
very fragile. Add one use of a proxy, and now you need to add this
binding to all the entry points. (You might wonder why we don't have a
common module with all of those. Many are singleton scoped and so we
do not want to create them in systems that do not need them.)

Since this has bitten us a lot in the past, we wanted to spend the
time to replace this with a generic and robust solution.

I'll do the improvements and report back. I'll also address Stuart's
comments about whitespace edits and such to have a clean patch.

PL

[Stuart McCulloch]

On 3 June 2010 09:16, Pascal-Louis <pascallo...@gmail.com> wrote:

Jesse,

In practice, the dozen or so bindings are more like 50 or 100; and
very fragile. Add one use of a proxy, and now you need to add this
binding to all the entry points. (You might wonder why we don't have a
common module with all of those. Many are singleton scoped and so we
do not want to create them in systems that do not need them.)

FWIW there are alternative approaches that work with Guice today:

1)  Use a factory to retrieve your marshalling objects...

//================================================
  interface EntityMarshallerFactory
  {
    <T> EntityMarshaller<T> getMarshaller( Class<T> entityType );
  }
//================================================

You could then @Inject an instance of EntityMarshallerFactory into
your code - maybe not as elegant as injecting the marshaller directly
but it works.

2)  Inject a concrete parameterized class, not an interface...

//================================================
  class JitProvider<T>
    implements Provider<T>
  {
    @Inject
    TypeLiteral<T> type;

    public T get()
    {
      try
      {
        return (T) type.getRawType().newInstance();
      }
      catch ( IllegalAccessException e )
      {
        throw new RuntimeException( e );
      }
      catch ( InstantiationException e )
      {
        throw new RuntimeException( e );
      }
    }
  }

  class JitProviderExample<T>
  {
    @Inject
    JitProvider<T> jitProvider;
  }

  public class Test
  {
    @Inject
    void testJitProvider( JitProviderExample<String> example )
    {
        System.out.println( example.jitProvider.get().getClass() );
    }
  }
//================================================

This relies on the existing limited JIT capability that's in Guice today.
Downsides are that you can't easily swap implementations, it doesn't
play well with child injectors, plus the other JIT related issues Jesse
mentioned (can lead to too much magic).

3)  Write a module that uses the Guice SPI to scan your application
     bindings/types for mentions of EntityMarshaller<...> so it can bind
     any missing bindings explicitly before the injector is created.

I'm using this technique at the moment with good results, also means
that you can use the Guice Grapher to visualize the whole application
because there aren't any JIT-related rabbit holes that the Grapher can't
follow down.

Downside is that this doesn't work with types that crop up at runtime,
ie. that aren't in the original application bindings/types used to create
the injector.

HTH

Since this has bitten us a lot in the past, we wanted to spend the
time to replace this with a generic and robust solution.

I'll do the improvements and report back. I'll also address Stuart's
comments about whitespace edits and such to have a clean patch.

PL

On Jun 2, 5:35 pm, "je...@swank.ca" <limpbiz...@gmail.com> wrote:
> On Jun 2, 7:41 am, Pascal-Louis <pascallouispe...@gmail.com> wrote:
>
> > Would that address your concerns?
>
> It would, but it still doesn't quite pay for its complexity.
> Autobinders would let you replace something structurally simple but
> verbose (a dozen or so repetitive bind() statements) with something
> structurally complex but compact (some reflection to implement
> JitProvider).

--
You received this message because you are subscribed to the Google Groups "google-guice-dev" group.
To post to this group, send email to google-g...@googlegroups.com.

To unsubscribe from this group, send email to google-guice-d...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-guice-dev?hl=en.

--
Cheers, Stuart

[Pascal-Louis]

Jesse,

This improvement has implemented, see for the main commit

http://github.com/pascallouisperez/guice-jit-providers/commit/2cae6f9db9b2b9d0ce1ec76cf9e50ffb83095701

Take a look at the updated test

http://github.com/pascallouisperez/guice-jit-providers/blob/master/test/com/google/inject/JitProvidersTest.java

just-in-time bindings are now much more natural

bindJit(new TypeLiteral<Factory<?>>() {})
.toJitProvider(FactoryJitProvider.class)
.in(Singleton.class);

which would match Factory<Integer>, Factory<List<String>> and such.
Variance is also supported: you could restricting the matching to

bindJit(new TypeLiteral<Factory<? extends Serializable>>() {})
.toJitProvider(FactoryJitProvider.class)
.in(Singleton.class);

and only Serializable types could instantiate the wildcard. This logic
is implemented in MoreTypes

http://github.com/pascallouisperez/guice-jit-providers/blob/master/src/com/google/inject/internal/MoreTypes.java#L588

As you will see, there are some TODOs that I need to finish but I
wanted to highlight the working feature to get feedback.

Should I finish, clean up my client, and send you a patch?

Best,

PL

On Jun 2, 5:35 pm, "je...@swank.ca" <limpbiz...@gmail.com> wrote:

[limpb...@gmail.com]

PL,

We still disagree about whether this feature is beneficial to Guice
projects. I continue to believe that the additional flexibility is too
dynamic and hurts the long term maintainability of a project.

The biggest day-to-day problem Guice developers face is management and
predictability of bindings. I'd love to see contributions that solve
these problems - making it as easy to navigate to the binding of a
type as it is to navigate to the type itself. Only once such problems
are solved will I be interested in more dynamic binding resolution
strategies.

Regards,
Jese