PSR-11 update: ContainerInterface, cross-framework modules

[Matthieu Napoli]

Hi everyone,

This is an email to let you know of the research work that has been done under the PSR-11/container-interop scope.

Summary of previous discussions:

• container-interop has a ContainerInterface that allows frameworks to decouple from containers. It is implemented by a lot of containers, and used by some frameworks like e.g. Zend Expressive, Slim, … PSR-11 is about standardizing that into a PSR.

• ContainerInterface was also offered as a solution for framework-agnostic modules (aka bundles, plugins, …) by letting each module provide its container, and by chaining them all together. This raised a lot of discussions, and several other approaches to reach the same goal were mentioned

We can identify two different needs from that:

• decouple frameworks from containers, to let users choose their own container (e.g. Zend Expressive, Slim, …)

• cross-framework modules/bundles, so that package authors don't have to write an extra package per framework out there (a good illustration: https://github.com/thephpleague?utf8=%E2%9C%93&query=glide)

We believe ContainerInterface solves the first problem.

Lately, we have been working on the second problem through discussions on this mailing list and Gitter and through experiments.

Approaches for cross-framework modules

We have identified the following main approaches:

• 1. each module can provide its own container, all containers can be chained to build a main "composite" container

This approach was the first considered. It has limitations and cannot cover all features you would expect in a module system, e.g. extend previously defined entries, etc. It is also a bit complex to explain and use.

• 2. standard PHP objects/interfaces representing “dumpable” container definitions.

These are objects that can cast themselves to a string of PHP code. Those objects can be consumed by compilers to generate a PSR-11 compliant container that could itself work side-by-side with another container of an application. We worked on an interface for such objects here: https://github.com/moufmouf/compiler-interop. This experiment is a mitigated success. It does work, but somehow, it requires the notion of “compiling” a container which most of the frameworks out there do not deal with. It forces existing frameworks to work side-by-side with a compiled container, which feels weird to some people.

• 3. standard PHP objects/interfaces describing container definitions.

This was the third approach we considered. We worked on interfaces for such objects: https://github.com/container-interop/definition-interop This experiment is a mitigated success: it works, we managed to integrate it into several containers and we wrote several experimental modules with it. However it is complex: to understand, to integrate into containers, to use to write modules.

• 4. standard container configuration format (e.g. XML, …)

This approach was then considered, as it is similar to the previous one but simpler to use for module authors. It is also easier to understand as it is similar to Symfony configuration (in YAML or XML files), or Spring in Java.

This work has not be formalized yet because of the amount of work needed. This approach would also suffer from a few of the limitations identified in the first approach, plus others related to the fact that you cannot use PHP code in XML or YAML. It would also require the inclusion in the standard of many specific features: the standard must define many different ways for how objects can be created and dependencies injected. That makes the standard complex to define, and would force all containers (even simple ones) to support all the features.

It is however the only approach that allows to perform static analysis of the provided services. For instance, a tool like Packagist could be written to scan the configuration files and allow searching services inside packages. This, however, is not a primary goal we are seeking.

• 5. standard service providers

This is the approach we have been experimenting with lately and it has turned out to be simpler on many level:

• the standard is much simpler, which means it is easier to explain and understand

• it is easier to use as it relies on plain old PHP code

• it is easier to implement support in containers

This is the approach we think has the most potential.

Standard service providers

In order to work and try the approach #5 (standard service providers) we have created a repository: https://github.com/container-interop/service-provider

This repository contains the following interface:

interface ServiceProvider

{

   public static function getServices() : array;

}

This is close to what service providers are in, for example, Pimple, Laravel, etc. Except instead of relying on methods on the container like "register()" or "bind()", the service providers only expose a list of factories.

Please read the "Usage" documentation as it will be easier than copy-pasting the instructions here: https://github.com/container-interop/service-provider#usage

On the consuming side, integrating such service providers into containers was fairly easy. We have also implemented some example modules to illustrate how this can be used: https://github.com/container-interop/service-provider#compatible-projects

What's next?

We now need more framework, container and module authors to take a look at all that. We do not take our conclusions as absolute truth but rather we expose them as "lessons learned". The discussion is open on all topics/approaches.

We need to move forward on:

• validate which approach we will choose - on that we recommend the "service providers"

• once that is done, we need to make it perfect, leading to a PSR

And is the "cross-framework modules" part of PSR-11 or not? It's interesting to note that ContainerInterface is also a requirement for cross-framework modules to decouple factories from container implementations (for example : https://github.com/container-interop/service-provider#usage). That means that if those are 2 separate PSRs, one will depend on the other.

--

Sent on behalf of David, Bernhard and myself

[Márk Sági-Kazár]

Hi guys,

Nice summary. I admit I haven't followed this topic lately, but this is definitely something which worths the efforts, so let me say thank you for all your work.

As I remember the harsh, and the less harsh discussions, one of PSR-11's main point was not configuring the container, but retrieving data from it. Someone even said IIRC, that we are getting to a point where (container) configuration is the only thing that actually makes frameworks different. While I don't 100% agree with that, I think it is quite a good approach to separate configuration (and compilation?) from "reading" the container.

For me, service providers are also part of the configuration. Actually I am a little bit surprised by the service provider approach you linked above. I would expect service providers provide services (aka. configure) to the container and not return actual instances. That seems to me an inappropriate responsibility. Can you explain why you chose this way? Or point to somewhere about justification, because I can't really find it in the usage section.

Last, but not least: I really like the idea of cross-framework packages. I think I saw a post from David about using definition interop and Puli. The idea is great and I am looking forward to using it in practice. It could probably be some sort of replacement for HTTPlug's discovery layer (we already use puli for discovery). However, I see a danger of overstandardization (tm) here: while I like the idea of a generic package integration, it won't be possible in every case (at least not in the near future): as mentioned earlier, configuration for example is very specific to each framework.

These are my thoughts. Keep up the good work, I am really looking forward to use this thing.

Best regards,
Mark

[Matthieu Napoli]

Hi Mark,

I would expect service providers provide services (aka. configure) to the container and not return actual instances. That seems to me an inappropriate responsibility. Can you explain why you chose this way?

Service providers like in Pimple are classes that take a container and configure it. The problem with that is that it requires the container to expose methods for configuration. That's a very impossible requirement in our case because all containers have a different API for configuration (e.g. in would never be compatible with Symfony's container).

Our approach is simpler: our service providers are actually factories of container entries. They do not require configuration methods on containers, so they can be made compatible with all/most of them. Each container entry is, in the end, just a callable to invoke (the method in the service provider).

 

However, I see a danger of overstandardization (tm) here: while I like the idea of a generic package integration, it won't be possible in every case (at least not in the near future): as mentioned earlier, configuration for example is very specific to each framework.

If we manage to replace 90% of framework-specific modules/bundles with cross-framework modules we think it's a big win. Some modules/bundles will not be able to be made generic because they are too specific. That's fine. Trying to cover 100% is unrealistic, and not necessary.

Also keep in mind we want to solve the problem only for modules. These service providers are not meant to be used by frameworks themselves or end users: they can depend on the container of their choice and use 100% of its features.

 

These are my thoughts. Keep up the good work, I am really looking forward to use this thing.

 

Thank you :) 

[David Négrier]

Hey list,

To complete Mathieu's post, I've written a blog article trying to explain the different solutions we have tried regarding cross-framework modules, and the strength and weaknesses of each solution:

http://www.thecodingmachine.com/psr-11-an-overview-of-interoperable-php-modules/

The article is a bit long so I will not repost it here.

Do not hesitate to have a look at it and give us some feedback. We are really interested in gathering as much feedback as possible. If you disagree with what I wrote, please let us know! If you agree, let us know too! This is important for the future.

We need to move forward and validate which approach we will choose - and we need to be sure we will pick the right one!

++

David

Twitter: @david_negrier

Github: @moufmouf

[Xedin Unknown]

Some points I brought up earlier in defense of definition-interop over service-provider here.

[Jan Tvrdík]

First of all – thank you for the research. Now to the questions!

How does service providers deal with services names collisions and package scopes? For example https://github.com/thecodingmachine/dbal-universal-module uses fairly common names (with high-collision chance) for input configuration and prefixes provided services with its namespace. This leads me to the following questions

1. What if I want to use multiple different DBALs in my application (possibly connected to multiple databases)?
2. What if I want to use the same DBAL multiple times because I need multiple Doctrine\DBAL\Connection instances?
3. What if I want to share part of the configuration among the DBALs?

One way (lets call it method A) to solve this may be to change ServiceProvider from static class to normal object and therefore allow to configure it though its constructor. The usage (assuming Simplex container) may look like

$simplex->register(new TheCodingMachine\DbalServiceProvider(
    'docrine.dbal.1st' // prefix for provided/expected values/services
));

$simplex->register(new TheCodingMachine\DbalServiceProvider(
    'docrine.dbal.2nd' // prefix for provided/expected values/services
));

Method A puts burden on providers of service providers, but seems quite flexible – in extreme case it would be possible to do sth like

$simplex->register(new TheCodingMachine\DbalServiceProvider(
    $inputMapping1 = [
        'dbal.host'                => 'doctrine.dbal.common.host',
        'dbal.user'                => 'doctrine.dbal.common.user',
        'dbal.password'            => 'doctrine.dbal.common.password',
        'dbal.dbname'              => 'doctrine.dbal.1st.dbname',
        'Doctrine\DBAL\Driver'     => 'doctrine.dbal.1st.driver',
    ],
    $outputMapping1 = [
        'Doctrine\DBAL\Connection' => 'doctrine.dbal.1st.connection',
    ],
));


$simplex->register(new TheCodingMachine\DbalServiceProvider(
    $inputMapping2 = [
        'dbal.host'                => 'doctrine.dbal.common.host',
        'dbal.user'                => 'doctrine.dbal.common.user',
        'dbal.password'            => 'doctrine.dbal.common.password',
        'dbal.dbname'              => 'doctrine.dbal.2nd.dbname',
        'Doctrine\DBAL\Driver'     => 'doctrine.dbal.2nd.driver',
    ],
    $outputMapping2 = [
        'Doctrine\DBAL\Connection' => 'doctrine.dbal.2nd.connection',
    ],
));

Another way (lets call this method B) to solve this would be improve the implementation of the DI container itself (Simplex in this case) to allow

$simplex->register('docrine.dbal.1st', TheCodingMachine\DbalServiceProvider::class);
$simplex->register('docrine.dbal.2nd', TheCodingMachine\DbalServiceProvider::class);

This is trivial to implement in case of output mapping (provided services / values) but a bit harder for input mapping (it would require another instance of ContainerInterface which would serve as a wrapper for the original simplex instance). Complex remapping would be likely possible as well and look likely as

$simplex->register('docrine.dbal.1st', TheCodingMachine\DbalServiceProvider::class, $inputMapping1, $outputMapping1);
$simplex->register('docrine.dbal.2nd', TheCodingMachine\DbalServiceProvider::class, $inputMapping2, $outputMapping2);

Method B puts burden on consumers of service providers which seems to me now as better option.

Regards,

Jan Tvrdík

On Sunday, March 20, 2016 at 7:01:20 PM UTC+1, Matthieu Napoli wrote:

[Matthieu Napoli]

Hi Jan, thanks for your response.

 

You raise good points but I'm not sure if it's a problem with the standard. We already have modules today (Symfony bundles, ZF modules, Silex/Slim service providers, etc.). If this is not an issue today then I don't think it will be an issue in our case.

 

The problems you mentioned in more details:

 

- what if a module uses common generic names? E.g. "database", "logger", etc.

 

If it's an issue I believe the module will not be used. Just like if a package today had a "Logger" class in the root namespace it wouldn't be used widely because it's not OK and users won't like it.

 

- what if a module uses common project names? E.g. "twig", "monolog", etc.

 

I believe it's fine if it's the official Monolog/Twig module for example. Or more generally if it's meant to configure Twig or Monolog, then OK.

If it's an unrelated package that defines a "monolog" key just because it uses monolog internally then that's not fine. But I think it's up to users to choose not to use it, or create a bug report/pull request to have it changed.

 

- what if I want to use the same service provider multiple times? E.g. I need multiple DBAL instances.

 

Is it possible with existing module systems? Yes, so I don't think it's an issue, or a responsibility of the standard to solve that for us. You have multiple choices here:

    - configure your DBAL yourself if the service providers you were using doesn't support multiple DBAL instances

    - use a different service provider that does support configuring multiple instances

    - prefix an existing service provider?

The last suggestion is something you mentioned, and I don't think it requires modifications in order to work. You can extend (or decorate) a service provider (as they are today) to prefix them:

 

class MyDbalProvider extendsDbalProvider

{

    public static function getServices() {

        $map = parent::getServices();

        $map = /* prefix keys */;

        return $map;

    }

}

 

As you said, it could also be supported by the DI container, which I think is a fine idea.

 

Keep also in mind that entry names could be class names (our draft voluntarily doesn't enforce any rules on entry names), so prefixing blindly might lead to weird entry names. But if you (the end user) prefix entries then you probably know what you are doing.

 

But in any case I think it's solvable by users or by creating specialized modules. The standard is very open thanks to its simplicity, and I think we should preserve that when possible.

 

By the way what do you mean by "output mapping"? I'm not sure I understood the difference with "input mapping"?

 

Matthieu

--

You received this message because you are subscribed to a topic in the Google Groups "PHP Framework Interoperability Group" group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/php-fig/xsY8bRG5K0M/unsubscribe.

To unsubscribe from this group and all its topics, send an email to php-fig+u...@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/d17bd1ed-1cf1-4025-8195-d45318c703ea%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

 

[Jan Tvrdík]

Hi Matthieu,

our draft voluntarily doesn't enforce any rules on entry names

Which I think is a fine decision (although I think that in the end we may end-up restricting it anyway to avoid e.g. control characters) but there should still be some guidelines/vision/best practices on how to name the entries because using random names simply does not work and you can not built reliable mental models around it. We can start by assuming simple convention <vendor>.<package>.<key> for values and <className> for services. Therefore in case of the services provider for Doctrine DBAL the parameters names would be e.g. doctrine.dbal.host instead of dbal.host.

Overall I believe that the naming convention plays a significant role in deciding whether Service providers are a viable solution.

You can extend (or decorate) a service provider (as they are today) to prefix them

Yes, but currently you would either need to rely on public static properties (ugly) or create new class for each prefix (ugly, not reusable). That's one of the reasons why I would strongly advice to drop the static keyword. Now even the smallest change requires creating new class and using inheritance. By making it possible to create service providers which can be at least partially configured, the whole system becomes a lot more powerful. For example you would be able to create MultiConnectionDbalProvider which would accept number of required connections in constructor.

By the way what do you mean by "output mapping"?

Output mapping = mapping of keys from `ServiceProvider::getServices` to how that will became available in final DI container. The most common mapping will likely be prefixing, e.g. the service provider may provide `connection` service but in would actually be registered as `doctrine.dbal.connection` (see last section on using prefixes by default).

Input mapping = mapping of arguments passed to `$container->get`

The standard is very open thanks to its simplicity, and I think we should preserve that when possible.

Agreed.

Interface Handling

Now let me describe another possible problem as mental exercise. Let P and Q both be some service providers providing some implementations of LoggerInterface. Let X and Y both be some service providers depending on some LoggerInterface. Assuming the aforementioned naming convention both X and Y will call `$container->get(LoggerInterface::class)`. Lets also assume that both P and Q will provide service named LoggerInterface.

The question is – what if I want to provide different implementation of `LoggerInterface` to X and Y? 

The problem has an obvious solution if both input and output mappers (method B in my previous post) would be available as end-user is in (almost) full control of composing the objects graph.

Without the mappers we would first of all need to create two new service providers P* and Q* (to rename keys from getServices) which would provide services named LoggerInterfaceP and LoggerInterfaceQ. Then we would need to create new service providers X* and Y*. Unfortunately we would currently need to copy and paste whole methods from X and Y just so we can replace string LoggerInterface to LoggerInterfaceP / LoggerInterfaceQ.

If service providers were not static, method A from my previous post would work as well.

Autowiring

Assuming the aforementioned naming convention, some form of naive (e.g. class inheritance unaware) autowiring among modules does work by design. If I had just P and X service providers in my application, the logger implementation by P would by automatically used by X. In PHP 7 would could easily analyze types of output services from method return types. However there is currently not a simple way to get types of input services. One simple way to address this shortcoming would be to add an optional method getInputServicesTypes(). Such method would also be useful to build service dependency tree.

Note that whether we need to explicitly support autowiring does depend on used naming convention for services.

Using user-controlled prefix (essentially forcing namespace) by default

Lets try another mental exercise. Lets assume naming convention <prefix>.<key> for both values and services. The prefix is a string chosen by the end-user for each service provider (see method B in my prev. post) or value stored in DEFAULT_PREFIX constant when user does not explicitly specify different prefix. Unless end-user needs to use the same service provider multiple times, we may assume default convention <vendor>.<package>. Lets modify ServiceProvider interface to pass the chosen prefix string as the first argument to both getServices and factory methods. This would allow writing code such

const DEFAULT_PREFIX = 'my.dbal';

public function getServices(string $prefix)
{
    return ["$prefix.connection" => 'createConnection'];
}

public function createConnection(string $prefix, ContainerInterface $c)
{
    // PREFER LOGGER SPECIFIC FOR THIS SERVICE PROVIDER, FALLBACK TO GLOBAL
    $logger = $c->has("$prefix.logger") ? $c->get("$prefix.logger") : $c->get('psr.logger');
    return new MyDbal\Connection($logger);
}

This kills the current by design sort-of autowiring but I kind of like it. It handles better the use-case where you need non-default implementation of some interface in some service provider.

Regards,

Jan Tvrdík

...

[Paul Jones]

> On Apr 7, 2016, at 09:13, Jan Tvrdík <jan.t...@gmail.com> wrote:
>
> We can start by assuming simple convention <vendor>.<package>.<key> for values and <className> for services. Therefore in case of the services provider for Doctrine DBAL the parameters names would be e.g. doctrine.dbal.host instead of dbal.host.

For what it's worth, in Aura we settled on "vendor/package:key", where "vendor/package" is the same as the Composer package, and the key is unique within that namespace.

> Overall I believe that the naming convention plays a significant role in deciding whether Service providers are a viable solution.

I tend to agree.


--

Paul M. Jones
http://paul-m-jones.com

[Brad Ito]

Thanks for publicizing this and the blog post.

I'd made my own DI package a while ago for work at a prior company: https://github.com/phlogisticfugu/squirt

strongly based on the not-well-known DI container from Guzzle.

one thing from these which may be useful in an interop discussion is the choice of configuration format.  If there is a standardization around a single configuration format, I humbly propose that the format not be XML/JSON/etc, but be the most natural and performant one for PHP: PHP files.

One can have a file "return" an associative array with constant values with all of the configuration and then load it in another script

$config = include('config.php');

and the advantage here is that this is a highly optimized within PHP itself, using caching and such, because PHP is made to handle PHP files.  One could also do simple string concatenation and arithmetic expressions within the config file.  You can also add comments directly into the configuration file.  And every PHP developer will instantly know how to read the configuration file.

[Brad Ito]

Looking at the use-cases being supported for different approaches, I'd also suggest folks add:

- can read configuration from environment variables

This is of particular concern for app secrets, such as passwords and API keys

for more, see http://12factor.net/config

any of the PHP-based approaches can use the getenv() function for this.

[David Négrier]

Hey Brad,

Regarding your comments about configuration, I like to assume that configuration is directly part of the container and that configuration values are dependencies.

Somehow, I had this feeling it was the right thing to do, and then, I discovered this blog post from Paul that makes it perfectly clear:

http://paul-m-jones.com/archives/6203 "Configuration Values Are Dependencies, Too"

So an idea could be to delegate the handling of the configuration to the container.

Think about this very PSR-11 compliant container:

class ConfigurationContainer implements ContainerInterface {

    public function get($id) {

        return getenv($id);

    }

    public function has($id) {

        return getenv($id) !== false;

    }

}

Such a container could by "appended" to the application container (using a composite container or something) and suddenly, all environment variables could be accessible from the container directly.

Also, I think each framework should keep its own way of managing the configuration (Symfony uses YAML, ZF uses PHP arrays,  Mouf uses defines....) As long as we can make those configuration values available in the container, we are good!

If you are interested, this is what I did when writing the Laravel <=> Service provider adapter here: https://github.com/thecodingmachine/laravel-universal-service-provider/blob/1.0/src/LaravelContainerAdapter.php and also when writing the Symfony <=> Service provider adapter here: https://github.com/thecodingmachine/service-provider-bridge-bundle/blob/1.0/src/SymfonyContainerAdapter.php

If you have a look at the code, you will see the "ContainerAdapter" (that transforms Laravel and Symfony container into a PSR-11 compliant container) checks both the Laravel/Symfony configuration and also container.

++

David.

--

You received this message because you are subscribed to a topic in the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/php-fig/xsY8bRG5K0M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to php-fig+u...@googlegroups.com.
To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/c0adda03-fc8c-427d-aa25-66c5fd06d8e0%40googlegroups.com.

[Matthieu Napoli]

Hi Brad,

To add to David's answer: in the approach we are favoring right now (https://github.com/container-interop/service-provider) you can call getenv() to retrieve environment value.

Since service providers are vanilla PHP code you can do anything you want, so there is no need for a specific support in the standard.

Matthieu