[PSR-11] Review of the delegate lookup feature

[David Négrier]

Ok folks!

I've been delaying this for a while to give some room for the rest of the questions about PSR-11, but now is the time to dwelve into the details of the delegate lookup feature :)

I know some of you have doubts about this part of the PSR, so the first thing might be to decide whether we want to keep it or not. If we decide to keep it, I'm sure there are plenty of things we can do to improve the wording :)

(post is quite long and contains images. If it does not display well, I cross-posted here)

What is it?

The delegate dependency lookup feature (let's call it DDL) is a design pattern.

It is the only way we have found to compose containers.

What problem does it solve?

Let's admit you want to compose/chain containers (more on why you might want to do this later).

To do this, you would typically use a "CompositeContainer" (a container without any entry whose role is to ask each "child" container in turn "do you contain the entry I'm looking for?")

The CompositeContainer is not enough. Let's admit container 1 contains your controller. You fetch your controller. Your controller has a dependency on your ORM's entity manager. If the entity manager is part of container 2, container 1 will be unable to fetch it (because internally, it will perform a call "$this->get()" to fetch the entityManager dependency. Similarly, the entityManager should be able to fetch a dependency (for instance the dbConnection) inside another container...

The delegate lookup feature simply states that containers should not fetch their dependencies locally, but instead, they should fetch their dependencies from the top-most container (in this example, this is the CompositeContainer).

In our example, this would go like this:

- The framework asks for the "controller" entry to the CompositeContainer 

- The CompositeContainer asks Container 1: "do you have "Controller". Container 1 answers yes

- The CompositeContainer asks Container 1: "give me "Controller".

- Container 1 needs to fetch the "EntityManager" dependency, so Container 1 calls $compositeContainer->get('EntityManager'); (here! container 1 is "delegating" the dependency lookup to the composite container)

- The CompositeContainer asks Container 1: "do you have "EntityManager"? => response is no.

- The CompositeContainer asks Container 2: "do you have "EntityManager"? => response is yes. The CompositeContainer asks Container 2: "give me "EntityManager".

- ... and do on

Do we want this?

To be honest, PSR-11 is already useful without the DDL feature. The ContainerInterface is enough to allow users to swap container implementations.

Being able to compose containers is a nice feature, but it's optional. PSR-11 can be useful without DDL.

What are valid use cases for this?

First of all, I do not expect major full-stack frameworks to use this. It is obvious from the example above that there is a performance hit when using a composite container (you have to ask each container in turn whether it contains the instance you are looking for or not).

Frameworks very focused on performance (like Symfony full-stack) should stay away from CompositeContainers (more on performance below).

Yet, here are a few use cases where composite containers are valuable:

1- Migration!

Let's admit you started a small app with Slim3 and Pimple. Your app is getting bigger. You now have more than 200 services declared in Pimple and you want to migrate away to something more powerful (maybe you want to benefit from Autowiring or maybe you need lazy services for performance issues...)

The DDL feature allows you to put Pimple side-by-side with your new container of choice and migrate entries slowly, one by one. This is really cool because it makes a daunting task a lot easier.

2- I don't care about performance, give me features!

Running containers side-by-side is a great away to enhance a container with the features of another container. For instance, you could enhance your existing container with a sidekick containers dedicated to creating aliases or serving a "lazy" version of your services, etc...

Not everybody cares for container performance. For instance, if you are doing async PHP (with ReactPHP or another lib), container performance is not a concern at all (since services are created once at the beginning of the script and are reused across requests).

By the way, what is the real performance impact of using a CompositeContainer and DDL?

I've been doing some tests using a modified version of Symfony.

You can read my old article about the performance impact on DDL here. Spoiler alert: impact is quite low, I was not able to measure it.

How do we implement this?

DDL support is quite easy to add in any container out there. From my experience with container-interop, I've never seen a case where it took more than a few lines of code to add support.

Typical implementation goes like this:

1- You modify the constructor to accept an additional parameter: the root container. This parameter is optional. If not passed, the root container is the container itself.

So your container constructor now looks like this:

public function __construct($param1, $param2, ..., ContainerInterface $rootContainer = null) {

    ...

    $this->rootContainer = $rootContainer ?: $this;

}

In your container code, when you perform a call to get on a dependency, instead of calling $this->get, you call $this->rootContainer->get.

Aaaaaand you're done :)

Important: As you can see, if you are not using a composite container, the impact on performance of a container is null (it is almost the same code executed). So a container can add support for DDL without impacting its average performance.

Is it used somewhere?

The ContainerInterface has been mostly designed by looking at what common methods were supported by containers out there.

On the contrary, the dependency delegate lookup design pattern has been "invented" by container-interop and does not stem from previous work from one container or another. This is of course to be expected, because without the ContainerInterface, it is not possible to envision composing containers.

The delegate dependency lookup feature is already supported by a number of containers out there (see: https://github.com/container-interop/container-interop#projects-implementing-the-delegate-lookup-feature )

I don't know if it is wildly used or not, but I know at least one place where this was useful to me: the laravel <=> container-interop/service-provider bridge. I used it here to add support for container-interop's service providers into Laravel. Rather than forking Laravel container to add support for service providers in it, I decided to add another container (Simplex) that already had support for service providers next to Laravel's one.

Summary:

Is it essential to PSR-11? No

Is it useful? Yes, in some specific cases (I do not expect it to be wildly used)

Is it easy to implement? Dead easy

Does it have an impact on performance? No if only one container is used (business as usual), yes if many containers are used (but this is to be expected)

Should it be part of the PSR?

My personal opinion is yes. It does not hurt, it is optional, it is the only way we could put containers side-by-side and let them share entries. I believe advertising this design pattern in the PSR will make it more wildly adopted. This, in turn, will increase framework interoperability.

Note: there are a ton of things I would still like to discuss, like "can a composite container contain entries?" or "can I nest composite containers?". Also, @crell asked a lot of questions in his initial review and I'm only surfacing a few answers but I'm afraid that will get too tricky quickly, so I'm going to stop there for the time being. :)

But here is the real question: do you think DDL is useful, and do you think it should be part of PSR-11?

Best regards,

David.

[Alessandro Pellizzari]

On 17/11/2016 11:29, David Négrier wrote:

> I know some of you have doubts about this part of the PSR, so the first
> thing might be to decide whether we want to keep it or not. If we decide
> to keep it, I'm sure there are plenty of things we can do to improve the
> wording :)

I see the advantages of using delegate containers (I use them daily),
but honestly I don't think they should go in this PSR.

PSR-11 is focused on getting things from a container, and delegates are
completely transparent from this POV.

As long as a Container implements get() it can become a delegate of a
(possibly) more powerful Container that implements delegation.

I think it would be better in a "Container creation and configuration" PSR.

Bye.

[Alessandro Lai]

I agree with Alessandro.

Since this seems a pretty controversial point, I would drop it, in favor of continuing the discussion into a new separate PSR, or maybe the "add stuff in the container" PSR.

PSR-11 is just a stepping stone for everything in this area, so I would prefer to drop it and push it to approval.

[David Négrier]

Hi both Alexandros,

Thanks for your feedback. I can understand that you think delegate lookup should not go in this PSR.

However, I don't think I agree with 2 statements:

You say:

> PSR-11 is focused on getting things from a container, and delegates are
> completely transparent from this POV.
>
> As long as a Container implements get() it can become a delegate of a
> (possibly) more powerful Container that implements delegation.

My whole point is that in order to have containers working side by side, having a 'composite container' delegating the lookup of entries to sub-containers is not enough.
You also have to make sure that for any "child container" can retrieve dependencies from other "child containers". This is only possible by promoting the DDL design pattern. (see the picture in my post above and the explanation below for more details).

> I would drop it, in favor of continuing the discussion into a new separate PSR, or maybe the "add stuff in the container" PSR

In my opinion, this has nothing to do in the "add stuff in the container" PSR. Maybe we can put it in another PSR, but definitely not the "add stuff in the container" PSR, this is a completely different issue in my opinion.

--
David.

[GeeH]

Thanks David for such a comprehensive roundup of what's being proposed. I personally found the time and effort you put into explaining the problem a great help.

I am against adding this functionality for reasons already stated - I believe it's out of the scope of this PSR and would be an implementation detail rather than an actual part of the standard. With that said, I am only slightly against it, we do something similar in Zend\ServiceManager where we allow people to travel up to the composite container (although it's not really a composite container as it can contain services) so I'm very interested in possibly using the pattern you explained in place of this. 

I'm not convinced it should be in the spec, but would happily concede if others were all in favour.

G

[Larry Garfield]

Thanks, David. My main concern is that as presented the delegate
feature is not really a feature, more a "by the way, here's this cool
thing you can do." The composite container isn't defined at all, and as
you note the question of what happens when the delegate container has
its own services is also completely undefined. Yet that is a valid use
case, and in fact the main one that I'd see being useful: Allowing a
library to provide its own mini-container for its services, which then
simplifies adding a large and more complex library (Doctrine, Guzzle,
etc.) to an application without the need for a custom bridge
module/bundle/whatever.

It may be that case is better handled through the forthcoming container
registration spec. If that's the case, then I see insufficient reason
to include DDL here. If it is kept, it needs to be considerably
overhauled to clarify those many edge cases, and suggest, well, what if
anything would be more than a "by the way, this quirk would be
possible."

One or the other: Remove, or revamp to be more clear and useful.

--
Larry Garfield
la...@garfieldtech.com

> > <https://www.thecodingmachine.com/psr-11-an-in-depth-view-at-the-delegate-lookup-feature/>
> > )
> >
> > *What is it?*
> >
> > The *delegate dependency lookup feature* (let's call it DDL) is a *design
> > pattern.*

> > It is the only way we have found to compose containers.
> >

> > *What problem does it solve?*

> >
> > Let's admit you want to compose/chain containers (more on why you might
> > want to do this later).
> > To do this, you would typically use a "CompositeContainer

> > <https://github.com/jeremeamia/acclimate-container/blob/master/src/CompositeContainer.php>"

> > (a container without any entry whose role is to ask each "child" container
> > in turn "do you contain the entry I'm looking for?")
> >
> > [image: composite.png]
> >
> > The CompositeContainer is not enough. Let's admit container 1 contains
> > your controller. You fetch your controller. Your controller has a
> > dependency on your ORM's entity manager. If the entity manager is part of
> > container 2, container 1 will be unable to fetch it (because internally, it
> > will perform a call "$this->get()" to fetch the entityManager dependency.
> > Similarly, the entityManager should be able to fetch a dependency (for
> > instance the dbConnection) inside another container...
> >
> > The delegate lookup feature simply states that containers should not fetch
> > their dependencies locally, but instead, they should fetch their
> > dependencies from the top-most container (in this example, this is the
> > CompositeContainer).
> >
> > In our example, this would go like this:
> >
> > - The framework asks for the "controller" entry to the CompositeContainer
> > - The CompositeContainer asks Container 1: "do you have "Controller".
> > Container 1 answers yes
> > - The CompositeContainer asks Container 1: "give me "Controller".
> > - Container 1 needs to fetch the "EntityManager" dependency, so Container

> > 1 calls *$compositeContainer->get('EntityManager')*; (here! container 1

> > is "delegating" the dependency lookup to the composite container)
> > - The CompositeContainer asks Container 1: "do you have "EntityManager"?
> > => response is no.
> > - The CompositeContainer asks Container 2: "do you have "EntityManager"?
> > => response is yes. The CompositeContainer asks Container 2: "give me
> > "EntityManager".
> > - ... and do on
> >
> >

> > *Do we want this?*

> >
> > To be honest, PSR-11 is already useful without the DDL feature. The
> > ContainerInterface is enough to allow users to swap container
> > implementations.
> >
> > Being able to compose containers is a nice feature, but it's optional.
> > PSR-11 can be useful without DDL.
> >

> > *What are valid use cases for this?*

> > *By the way, what is the real performance impact of using a
> > CompositeContainer and DDL?*

> >
> > I've been doing some tests using a modified version of Symfony.
> > You can read my old article about the performance impact on DDL here

> > <https://www.thecodingmachine.com/psr-11-performance-impact-of-the-delegate-lookup-feature/>.

> > Spoiler alert: impact is quite low, I was not able to measure it.
> >
> >

> > *How do we implement this?*

> >
> > DDL support is quite easy to add in any container out there. From my
> > experience with container-interop, I've never seen a case where it took
> > more than a few lines of code to add support.
> >
> > Typical implementation goes like this:
> >
> > 1- You modify the constructor to accept an additional parameter: the root
> > container. This parameter is optional. If not passed, the root container is
> > the container itself.
> >
> > So your container constructor now looks like this:
> >
> > public function __construct($param1, $param2, ..., ContainerInterface
> > $rootContainer = null) {
> > ...
> > $this->rootContainer = $rootContainer ?: $this;
> > }
> >
> > In your container code, when you perform a call to get on a dependency,
> > instead of calling $this->get, you call $this->rootContainer->get.
> >
> > Aaaaaand you're done :)
> >

> > *Important*: As you can see, if you are not using a composite container,

> > the impact on performance of a container is null (it is almost the same
> > code executed). So a container can add support for DDL without impacting
> > its average performance.
> >

> > *Is it used somewhere?*

> >
> > The ContainerInterface has been mostly designed by looking at what common
> > methods were supported by containers out there.
> > On the contrary, the dependency delegate lookup design pattern has been
> > "invented" by container-interop and does not stem from previous work from
> > one container or another. This is of course to be expected, because without
> > the ContainerInterface, it is not possible to envision composing containers.
> >
> > The delegate dependency lookup feature is already supported by a number of
> > containers out there (see:
> > https://github.com/container-interop/container-interop#projects-implementing-the-delegate-lookup-feature
> > )
> >
> > I don't know if it is wildly used or not, but I know at least one place
> > where this was useful to me: the laravel <=>
> > container-interop/service-provider bridge

> > <https://github.com/thecodingmachine/laravel-universal-service-provider/>.

> > I used it here to add support for container-interop's service providers

> > <https://github.com/container-interop/service-provider> into Laravel.

> > Rather than forking Laravel container to add support for service providers
> > in it, I decided to add another container (Simplex) that already had
> > support for service providers next to Laravel's one.
> >
> >

> > *Summary:*

> >
> > Is it essential to PSR-11? No
> > Is it useful? Yes, in some specific cases (I do not expect it to be wildly
> > used)
> > Is it easy to implement? Dead easy
> > Does it have an impact on performance? No if only one container is used
> > (business as usual), yes if many containers are used (but this is to be
> > expected)
> >
> > Should it be part of the PSR?
> >
> > My personal opinion is yes. It does not hurt, it is optional, it is the
> > only way we could put containers side-by-side and let them share entries. I
> > believe advertising this design pattern in the PSR will make it more wildly
> > adopted. This, in turn, will increase framework interoperability.
> >
> > Note: there are a ton of things I would still like to discuss, like "can a
> > composite container contain entries?" or "can I nest composite
> > containers?". Also, @crell asked a lot of questions in his initial review
> > and I'm only surfacing a few answers but I'm afraid that will get too
> > tricky quickly, so I'm going to stop there for the time being. :)
> >
> > But here is the real question: do you think DDL is useful, and do you
> > think it should be part of PSR-11?
> >
> > Best regards,
> > David.
> >
>

> --
> You received this message because you are subscribed to the Google Groups
> "PHP Framework Interoperability Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to php-fig+u...@googlegroups.com.
> To post to this group, send email to php...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/php-fig/273f0df4-ee6b-4712-b550-4609fb515ec6%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Alessandro Pellizzari]

On Thu, 17 Nov 2016 06:02:48 -0800 (PST)
David Négrier <david....@gmail.com> wrote:

> My whole point is that in order to have containers working side by
> side, having a 'composite container' delegating the lookup of entries
> to sub-containers is not enough.
> You also have to make sure that for any "child container" can
> retrieve dependencies from other "child containers". This is only
> possible by promoting the DDL design pattern. (see the picture in my
> post above and the explanation below for more details).

I disagree. :)

I think it's an implementation detail. The final user can decide to use
a delegating container as a front-facing one, and a non-delegating
container "hidden inside". The usage will still be transparent.

If a component provides a non-delegating container for its internal
use, the outer delegating container doesn't need to know it. It just
calls get() on the non-delegating one. The use doesn't care either, as
he just calls get() on the delegating one.

I can't think of a case in which it would be mandatory for all
container to be delegating-aware.

Bye.

[Matthieu Napoli]

I'm fine with both option (include or don't include it in PSR-11).

As you said David, it's a design pattern. We don't *need* to standardize it to use it. Containers that want to do stuff like that can, it's not required for decoupling frameworks from containers.

If it's really needed it could be a separate PSR, now that most of the work is done it could be easy to write, but probably it won't even be necessary.

> My main concern is that as presented the delegate feature is not really a feature, more a "by the way, here's this cool thing you can do."

Agreed.

[David Négrier]

Le jeudi 17 novembre 2016 18:58:24 UTC+1, Larry Garfield a écrit :

My main concern is that as presented the delegate
feature is not really a feature, more a "by the way, here's this cool
thing you can do."

It's more something like: "by the way, if you implement DDL, your users can do cool things", but I get your point.

 

The composite container isn't defined at all, and as
you note the question of what happens when the delegate container has
its own services is also completely undefined.

The wording certainly needs some polishing.

Regarding what happens if a composite container contains its own services, I would look at it this way: such a container is both a "pure composite container" and a "real container".

See the picture below:

In the case of a composite container with entries, I would expect this composite container to have methods like: "prependChildContainer" or "appendChildContainer". Everything else is exactly the same as described:

- Container 1, 2 and 4 should implement DDL. They should refer to the root container (in this case the "composite container with entries").

For the sake of crazyness, we could also envision a truly complex case (I don't know what this could be used for in real-life). See picture below.

Here, composite containers are nested and one composite container is containing entries. It still works :) Every container containing entries (including the composite one) must point back to the top-most container when looking for dependencies. It's as simple as that.

 

 Yet that is a valid use
case, and in fact the main one that I'd see being useful: Allowing a
library to provide its own mini-container for its services, which then
simplifies adding a large and more complex library (Doctrine, Guzzle,
etc.) to an application without the need for a custom bridge
module/bundle/whatever.

It may be that case is better handled through the forthcoming container
registration spec.  If that's the case, then I see insufficient reason
to include DDL here.  

Agree. This use case would definitely be better handled through the forthcoming container registration PSR. Although you can indeed use DDL and composite containers to provide "one container per lib", it's not very efficient and also, there is no way you can easily "extend" a service from one lib into another (hence the need for another PSR). I'd not recommend this use case. The only use cases I envision are "migration" and "adding additional features" (as stated in my post above).

Yet, I still believe this is a useful (and unknown) design pattern and that including it in the PSR will increase interoperability between containers (even if it is seldomly used)

That being said, so far, we've been few to be truly vocal about this feature (lately, that's mostly me and XedinUnknown). I'd still like to get a bit more feedback before putting a nail in the coffin of DDL :s

@Larry, that's really great you could have a PHP-FIG IRL meeting at PHPWorld. Did you speak about the delegate lookup feature? Was there a general concensus there? Also, could you tell us who was present at this meeting?

Best regards,

David.

[Chuck Burgess]

I would leave it out of this spec, and perhaps apply the invested effort into a DDL spec and/or reference implementation atop this spec.  The need for this spec to go out into the world seems greater to me than the realized benefits of forcing implementations of this spec to include the DDL feature.

CRB
about.me/ashnazg

--

You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+unsubscribe@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/CABAasbcp_kj4_n4ifoGh2ijnnL1%3D%3D8o6sg3eYj_RpjqOc6xj_g%40mail.gmail.com.

[David Négrier]

As I already explained, I'm a bit reluctant to remove the delegate lookup feature because I really wanted to raise awareness of this design pattern (otherwise, composing containers is almost useless). That being said, I guess I need to listen to feedback, and the feedback we received in this thread is mostly that it's not essential to PSR-11.

I wanted to work on a v2 of the delegate lookup feature with a better wording, but I probably won't have time before we switch to FIG 3.0. And I'd definitely prefer PSR-11 to pass the vote with FIG 2.0 (it seems pointless to create a whole working group on PSR-11 as it is already finished :) )

So here is the PR that removes the delegate lookup feature from PSR-11: https://github.com/php-fig/fig-standards/pull/854

Reviews are welcome. That way, we can gauge if the feature is needed after PSR-11 adoption and come back to it in another PSR if needed.

++
David

CRB
about.me/ashnazg

To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+u...@googlegroups.com.

[Alessandro Lai]

Kudos to you for stepping down on your proposal and to have listened to feedback!

The fact that this PR hilights how much text is removed from the PSR underlines how much the delegate lookup needs a separate PSR. I think it will be better on the long run, also because you will be able to leverage PSR-11 itself, as a published and approved PSR.