Service providers PSR: seeking members for a working group

[David Négrier]

Hey list!

PSR-11 has been accepted so we have standardized how to fetch entries from containers.

The next logical step is to find a common way to put things into a container. More generally, the goal we are seeking is to make it possible for a package author to provide/modify entries in any container (rather than having to write a module/bundle for each framework as it's the case today as I described in this talk: https://thecodingmachine.github.io/forumphp2016talk/index_en.html#60).

Let's be clear that the goal is NOT to define a single way of configuring a container. Each container has its own strategy (configuration files, autowiring, PHP code...) that makes it worthwhile. What we are trying here is to find an additional shared way of putting things into a container specifically for package authors. Because it's almost impossible for package authors to write a bridge for every framework out there).

Also, since FIG 3.0 has passed, it's time that we officially create a working group.

As a reminder, we (the container-interop participants) explored several strategies:

- unified file format

- common interfaces for container definitions

- common interfaces for dumping PHP code representing container definitions

- service providers

I have summarized this in those 2 blog posts: https://www.thecodingmachine.com/psr-11-an-overview-of-interoperable-php-modules/ and https://www.thecodingmachine.com/psr-11-get-ready-for-universal-service-providers/ .

The conclusion we reached was that standardized service providers are the way to go. Among many criteria, they are easy to write, and if done right, can be properly optimized by compiled containers.

We started working on it at https://github.com/container-interop/service-provider/

Work is well advanced: we have prototype integrations available with the major frameworks out there.

There is still a lot of work to be done:

- Recently, Rasmus Schultz came up with an alternative proposal. The idea is that rather than trying to provide factories to the containers (what service providers do), a module could provide its own container and publish to the "main container" the list of entries it contains. Rasmus detailed the idea here: https://github.com/container-interop/service-provider/issues/40 . This idea looks a bit like the "delegate lookup feature" that was removed from PSR-11 in the philosophy (several containers running side-by-side), but is different in the implementation. I'd be interested to gather feedback from the community on Rasmus' proposal (I suspect it might be complementary to container-interop/service-provider rather than opposite)

- With the current proposal, we can already add services and extend existing services. We need to know if we want to be able to do more (like adding/extending services conditionally based on the container's configuration). More generally, we need to discuss the exact scope of the PSR (limited to simple use cases? extended?)

- Finally, there will be a huge amount of work waiting for us in the nitty-gritty (exceptions handling, etc...)

I'm hereby calling interested members to step up to create a working group, and for member projects to give as much feedback as possible. PSR-11 has made a big leap forward in the very last days because "big players" joined the party a bit late :). For this PSR, it would be great to have some feedback from you guys early on in the process.

++

David.

Twitter: @david_negrier

Github: @moufmouf

[Paul Dragoonis]

I already worked a lot on these problems with PPI. If you want me to join the WG then i will help out. 

--
You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+unsubscribe@googlegroups.com.
To post to this group, send email to php...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/1573992a-355d-42a8-bd1f-c53ebaa43fd1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matthew Weier O'Phinney]

Count me in!

--

[Paul Dragoonis]

I already worked a lot on these problems with PPI. If you want me to join the WG then i will help out

On 28 Mar 2017 11:01 am, "David Négrier" <david....@gmail.com> wrote:

--

[Xedin Unknown]

Also available to help out.

Maybe, an update from the last time I posted could be useful.

The implementation I had written for the purposes of demonstrating my take on lookup delegation is now dhii/di. It achieves that by every child container being able to reference its parent, and the top-most container obtained this way is always passed to the service factories. That container then delegates lookup to its child containers.

This package is right now being used for 2 major WordPress plugin projects. I approached it in the way Rasmus describes, i.e. I "fake" a top-most WordPress container in the plugin, and attach another composite container to it, and then a container per plugin/addon of the same "family". Of course I use the service provider pattern everywhere, but for one of the projects I use a convention, whereby every "module" can contain a `services.php` file. This file MUST return either an instance of `ServiceProviderInterface`, or an array of service factories, which internally gets wrapped in a generic immutable service provider. All these providers are then given to the container responsible for the plugin.

What I mean to say here is that there's no need for the approaches to be mutually exclusive. I think they are both beneficial, and in my projects they co-exist.

[Miguel Muscat]

Having contributed to dhii/di, I'm available to help out as well :)

[Rasmus Schultz]

I will try to follow up on the alternative provider approach I suggested soon - per my last comment on that issue, another push is needed to evaluate this in practice.

As for the standardized provider format you're proposing, I remain of the opinion that this will be of little practical use in real life.

Bootstrapping everything against a single container is, in my opinion, unrealistic - it works for a low number of providers from vendors who carefully coordinate those providers, but at scale, I am fairly confident this will fail hard.

It's kind of like if everyone was sharing a single namespace, and in many ways it's actually worse. You can't possibly know what other vendors are bootstrapping in their providers. Is it safe for me to register a PSR-16 CacheInterface implementation, for example? If two providers do that, you will get terrible unexpected results when one of these randomly "win" depending on the order in which the providers were added.

I work on a fairly large project, and we're starting to see some of these issues surface - different teams can't know about everything that gets bootstrapped by every provider from every other team. It's not safe.

My hypothesis is that isolated modules, that selectively export only domain-specific services etc. are likely much safer, and I think that using proprietaty providers within a given domain is perfectly fine.

I think that the introduction of a standard provider format will effectively mean that almost no one will be able to make use of proprietary container configuration features - the standard format will compete and be at odds with proprietary formats. If successful, it will largely erase the differences between containers. (If unsuccessful, well, the PSR won't have much value.)

So I'm proposing an alternative, not a supplement. Not that I will be putting my energy into derailing your current proposal, just that I don't believe it solves the problem we're facing in my organization - what we're doing now is very much what you're proposing, bootstrapping everything against a single container, and it's already looking non-viable for four teams trying to coordinate providers. I'm afraid this will be much, much worse with hundreds of vendors shipping providers without any coordination.

Anyhow, I'm hoping to do another iteration on the alternative modular approach, probably in the next two weeks.

[David Négrier]

Hey all!

First of all, thanks for these early feedbacks!

@Rasmus: we are definitely not in a hurry and we can wait for your tests to see how your proposal go. Your comment is very interesting because you are pointing issues that would appear on massive projects. And indeed, when Matthieu Napoli designed the first version of container-interop/service-provider, I know for sure that this type of use case was clearly out of scope.

This is why I believe the very first thing we need to do is to discuss the scope of this PSR.

So far, the scope that Matthieu and I want to support is limited:

Ultimately, we believe that container configuration is the responsibility of the application developer.

Service providers (or application modules/bundles, etc...) are here to provide sensible default services. If the application developer has specific needs, then it's up to him configure the container and he probably should bypass the service providers altogether.

For instance, a DB connection library like DBAL might provide a service provider to configure one DB connection (because it's a sensible default). If your app has 2 or 3 DB connections, and you want one provider to connect to one DB and another provider to connect to another DB, then we are out of the scope. It doesn't mean that you can't do it with the current proposal (I'm exploring the possibility to use "mappers" to rename/alias services in order to circumvent these limitations), but we are comfortable saying the scope of service-providers can be limited.

The container is here to help the developer "wire" the services together.

If the developer has to deal with wiring the service providers together (I connect this DB service provider to this Cache Service provider, etc...), I feel we are simply moving the problem we are trying to solve one level up the chain (wiring service providers instead of wiring services), but we are not actually solving the problem. Especially if you consider it is likely that most service providers will contain 1 or 2 services at most, the amount of wiring to do will pretty much be the same.

I'm pretty much ok sharing a single namespace for all services (all frameworks are already doing this)

Even with such a limited scope, there is place for discussion. For instance: can a service provider act differently based on the services/entries already in the container? (can a service provider provide a "CacheInterface" entry if and only if there is no entry for "CacheInterface" already defined?) etc...

I'd be intereted in gathering comments from framework authors out there. Are you confortable with this scope? Do you think instead that we should cover a wider scope (as described by Rasmus?)

++

David.

[Xedin Unknown]

Not sure why "bootstrapping everything into one container" keeps coming up again and again, as I seem to have described a solution which involves multiple containers. Of course, for this to work some sort of lookup delegation has to be in place.

With regard to "sharing the same namespace", that's true. Especially true in an environment like a CMS, where you expect many plugins in one system to provide services while being ignorant of other plugins' services. For this reason, we prefix the services which are scoped to specifically the plugin, or the plugin's extension. Internally, we have a method often called `_p()` which prefixes a service ID with something that is configured on the service provider. So, when we want to scope something to the particular plugin, or we want to retrieved something scoped to the plugin from its service provider, we pass the key through that method first; otherwise, if we want something that is globally scoped, e.g. WP's DB connection, we don't use that method.

Maybe by optimizing and formalizing this approach it could be possible to solve the issue of namespaces.

As for cases where one team doesn't know what another team is doing - maybe they are different teams in the same organisation, or independent extension developers - I don't feel that this is a problem. Teams that work in the same organisation simply need to coordinate their effort if they are working on adjacent/overlapping areas, such as decide who has authority to develop the database-related services in a period of time. In any case, generally, when you consume some API, you need to know about that API. You don't go invoking/declaring random functions and instantiating/declaring random classes willy-nilly; instead, you research what the available API is. Same thing here: if you are consuming services provided by somebody, you research what they provide. Otherwise, you run into a conflict, which has to be resolved, and that's OK. It's the same thing with functions and classes. I don't believe that the mechanism of conflict resolution is in the scope of a service provider standard.

On Tuesday, March 28, 2017 at 12:01:48 PM UTC+2, David Négrier wrote:

[David Négrier]

Hey list!

I'm coming back from Symfony Live where I've had the chance to have a pretty long and interesting discussion with Nicolas Grekas and Christophe Coevoet (aka @stof) about service providers.

In particular, we spoke about the idea presented by Rasmus. I was under the impression that this idea would be directly rejected by Symfony because it involves more than one container (I already said that in a previous thread). It turns out I was plainly wrong. Concerns raised by Fabien about the multiple containers where linked to the fact that the delegate dependency lookup feature was looking through all containers for all entries (incurring a performance penalty), whereas with the strategy presented by Rasmus, the "main" container would directly know which container should be contacted for a given service (so much faster).

Nicolas pushed the idea of Rasmus a bit further. One of the issue he expressed with the current service-provider interface was a fear of conflicts or missing dependencies: when using a service-provider, the application cannot know what services this service-provider requires and it cannot know the type of the provided services (in fact it somehow could by looking at the return type of the factories in PHP 7).

So his idea is a 2 folded interface.

The first interface would present the "exported" services of a container to the outside world

interface ServiceProviderInterface (?)

{

    /**

     * Returns the list of exported identifiers as key of the array.

     * The value is the FQCN of the service (or the scalar type)

     * Note: how we treat arrays of services is pretty much unclear at this point. Maybe "FQCN[]"?

     *

     * @return array<identifier, type>

     */

    public function listIdentifiers();

}

At this point, it looks pretty much like what Rasmus proposed.

The second interface he envisions declares the list of identifiers that are needed from the container.

interface ServiceConsumerInterface (?)

{

    /**

     * Returns the list of identifiers that this container will need to build services.

     * Signature of the array is the listIdentifiers method.

     * If a type starts with a "?", it is optionnal, so the service-provider can optionnally use it if available.

     * 

     *

     * @return array<identifier, type>

     */

    public function requiredIdentifiers();

}

I hope I'm not betraying the idea of Nicolas, but I think is reasoning is that if we can know for sure what services are provided (with the type) and what services are required (with the type too), the framework or the main container could probably do a lot statically to help avoid wiring errors (or maybe the framework could autowire the containers? OMG! Inception! :) )

Note: A PR a bit similar to the "ServiceConsumerInterface" made its way into Symfony recently (https://github.com/symfony/symfony/pull/21708). They are using it to whitelist the list of services that can be fetched by a container when used as a service locator.

Also, so far, we did not clarify how no way to manage tags or services extensions (for instance, how can a Twig_Extension registers itself in the Twig_Environment) A number of ideas poped up while we discussed. The idea is still young and I guess it needs some work before we can assess if it is worthwhile.

So here we are.

Personnally, I feel pretty confortable with container-interop/service-provider. I understand it well, I understand how naming conflicts can be resolved with the idea of namespacing and mapping exposed by @XedinUnknown. Well, I'm happy with it. I've been working with the concept for over a year, so it feels familiar.

Now, there is this whole new concept. It has some appealing features:

- ability to choose whatever container format you prefer for your package

- ability to introspect a container (this feature alone is very interesting)

- possibility to do some kind of static analysis

- it might also be easier to implement for consuming containers

and it has a big number of holes that needs to be filled (how to extend / tag services...). I'm not as confident with this new proposed approach but this is ok, this is still young.

So let's try to prototype something. Let's see where it goes.

I'd rather test all possible leads we have a choose the best concept rather than start a PSR and realize the path we chose was flawed.

@Rasmus: if you are ok, I can give you a hand on this prototype. I'm on holiday for the coming week (so don't expect any answer from me) but we can move it to container-interop and see how it goes when I come back. We still need to find a suitable name for this concept... ConsumableContainers? (since these are containers that can be consumed by other containers?) Any other idea?

++

David.

[Alessandro Lai]

I like a lot this new proposal! It seems to address all the previous concerns, and has a great approach performance-wise.

I would only advise against the prefixing required service names with a question mark: if a service is required but optional, returning it in the requiredIdentifiers() method seems counter-intuitive (if not plainly wrong) to me! 

[Stephan Hochdörfer]

I'd love to help to drive this forward!

[David Négrier]

Hey guys,

Thanks a lot for all the feedback.

Before we form a working group, it might be a good idea to have a clear idea of where we are headed. We are currently exploring 2 alternative options.

The first one is : https://github.com/container-interop/service-provider/issues/

The alternative one, proposed by Rasmus is here: https://github.com/container-interop/service-provider/issues/40

I'd love to have some feedback of you above those 2 proposals.

Maybe the first step would be to "watch" https://github.com/container-interop/service-provider and then send us some feedback in the related issues?

Best regards,

David.