Composer, Installer, Config, and System

[Paul M. Jones]

Hi all,

tl;dr: Should we install packages to the default Composer location of "vendor/Vendor/Package" instead of "package/Vendor.Package", or should we have a custom Composer installer that installs to "package/Vendor.Package"?

* * *

While I have a long history with PEAR and the PEAR ecosystem, I have come to the conclusion of late that its time has passed. Its standards-body activities are being overtaken by the PHP-FIG, and its package distribution mechanism is being overtaken by Composer.

With that in mind, I'd like to use Composer as the package distribution mechanism for Aura. Specifically, this means having a composer.json file in the "system" repository, and use that to install/update packages from all vendors (including vendors not using Aura style packages).

Currently, Aura has a "package" directory where Aura-style packages go. To install a package, you copy or clone a Vendor.Package into that directory; the directory is flat, with all packages side-by-side. The system config mechanism scans that directory, looks for each instance of "Vendor.Package/config/default.php", and uses those files for default configurations. That lets the system bootstrap the configuration for each individual package.

Composer, on the other hand, installs to a default directory location called "vendor". The installed packages are in a 2-directory hierarchy organized by vendor name and package name ("vendor/Vendor/Package").

Composer allows for project-specific installers. If you look at <https://github.com/composer/installers> you will see that different frameworks can specify where to put their installed packages. Indeed, it lets them get installed to various different places in the framework structure. This means a custom Aura installer could put things in the "package" directory using the current conventions.

However, because Aura packages are designed to be self-contained, I wonder if we should change the Aura system to use the default Composer mechanism, and have packages get dropped into the default "vendor" directory (and of course get rid of the "package" directory).

That entails only one problem I can think of. We would need to modify the system config mechanism to look for config files in a different, less-efficient way. It would have to scan all "vendor/*" second-level subdirectories, then look for a "config/aura/default.php" file. Note that this means changing the library packages to use that extra "aura" subdirectory; this is because not all packages in the "vendor" directory are guaranteed to be Aura-style packages, so we need to be able to identify Aura-specific configs. (Of course, we can modify the config caching command to look in this new location when building the cached configs, so the decreased efficiency disappears at that point.)

The benefit to dropping "package/Vendor.Package" in favor of "vendor/Vendor/Package", then, is that we use the default Composer behavior to our advantage. That should make it easier on other projects who want to use Aura libraries as well, since they can assume the default Composer behavior.

Comments? Concerns? Criticism?


-- pmj

[jon.e...@gmail.com]

Hi All,
I really prefer to use Composer over PEAR at this point so +1 on that one.
My personal preference would be to remain consistent with the default Composer behavior with respect to package layout. I don't think the impact to the system would be noticeable, and making it easier for other projects to work with Aura would be of great benefit.

I have seen a lot of work going into Aura over the last few months and it's so nice to see. Well done to all those who have taken the time (unlike me) to move the yard sticks forward.

Jon

[Hari K T]

Hi all, 

tl;dr: Should we install packages to the default Composer location of "vendor/Vendor/Package" instead of "package/Vendor.Package", or should we have a custom Composer installer that installs to "package/Vendor.Package"?

* * *

Hey Paul , 

I prefer to go with Vendor.Package format and place in package format itself . I will try to explain why in this case below .

 

While I have a long history with PEAR and the PEAR ecosystem, I have come to the conclusion of late that its time has passed.  Its standards-body activities are being overtaken by the PHP-FIG, and its package distribution mechanism is being overtaken by Composer.

Yes true . Pear is now sleeping and downloading a 25 MB phar is like :P .

 

With that in mind, I'd like to use Composer as the package distribution mechanism for Aura. Specifically, this means having a composer.json file in the "system" repository, and use that to install/update packages from all vendors (including vendors not using Aura style packages).

Currently, Aura has a "package" directory where Aura-style packages go. To install a package, you copy or clone a Vendor.Package into that directory; the directory is flat, with all packages side-by-side.  The system config mechanism scans that directory, looks for each instance of "Vendor.Package/config/default.php", and uses those files for default configurations. That lets the system bootstrap the configuration for each individual package.

Composer, on the other hand, installs to a default directory location called "vendor".  The installed packages are in a 2-directory hierarchy organized by vendor name and package name ("vendor/Vendor/Package").

I would like to correct you here .

So the default format of composer installation for Aura/Router will be like 

vendor/aura/router/Aura.Router/src/Aura/Router

that means we have another vendor directory . Notice the small aura/router .

 

Composer allows for project-specific installers.  If you look at <https://github.com/composer/installers> you will see that different frameworks can specify where to put their installed packages. Indeed, it lets them get installed to various different places in the framework structure.  This means a custom Aura installer could put things in the "package" directory using the current conventions.

Yes I have been lately playing it yesterday . So if anyone is interested have a look into the PR

https://github.com/composer/installers/pull/24 

http://packagist.org/packages/hari/aura-system

You can have a look into the composer.json which will be used for system 

https://github.com/harikt/aura-system/blob/master/composer.json

 

However, because Aura packages are designed to be self-contained, I wonder if we should change the Aura system to use the default Composer mechanism, and have packages get dropped into the default "vendor" directory (and of course get rid of the "package" directory).

I don't have any issues if there is one more directory as I mentioned earlier :) . But if you still think its ok, I am with that also, but I don't love the change in directory structure for  if we have blogged some where and we don't remember where , then its editing that place also.

 

That entails only one problem I can think of. We would need to modify the system config mechanism to look for config files in a different, less-efficient way.  It would have to scan all "vendor/*" second-level subdirectories, then look for a "config/aura/default.php" file. Note that this means changing the library packages to use that extra "aura" subdirectory; this is because not all packages in the "vendor" directory are guaranteed to be Aura-style packages, so we need to be able to identify Aura-specific configs.  (Of course, we can modify the config caching command to look in this new location when building the cached configs, so the decreased efficiency disappears at that point.)

In that case you don't need to modify config/aura , if composer installs to vendor/aura/pacakage/Aura.Package/config 

The benefit to dropping "package/Vendor.Package" in favor of "vendor/Vendor/Package", then, is that we use the default Composer behavior to our advantage.  That should make it easier on other projects who want to use Aura libraries as well, since they can assume the default Composer behavior.

I don't feel we need to change anything in the composer.json package . Its default type is library .

As in the PR we are talking you can see the extra.installer-path which helps to download it in package folder and the other 3rd party to include or vendor .

According to me the 3rd party libraries are the only one which we don't have access to edit the composer.json which will be downloaded to vendor folder of composer format .

That means changing the name of include makes sense than for package .

Please also get a name by default for the  comment over here 

https://github.com/composer/installers/pull/24#issuecomment-7410032

I feel Paul can have a look into that . I will do the needful according to the comments .

Thanks

[Paul M. Jones]

On Jul 31, 2012, at 8:17 PM, Hari K T wrote:

>> Composer, on the other hand, installs to a default directory location called "vendor". The installed packages are in a 2-directory hierarchy organized by vendor name and package name ("vendor/Vendor/Package").
>>
> I would like to correct you here .
>
> So the default format of composer installation for Aura/Router will be like
>
> vendor/aura/router/Aura.Router/src/Aura/Router
>
> that means we have another vendor directory . Notice the small aura/router .

Not quite. Experimenting with it today, I see both "vendor/Aura/Autoload" and "vendor/Aura/uri" (note the upper case vendor and lower case package on the last one).

>> However, because Aura packages are designed to be self-contained, I wonder if we should change the Aura system to use the default Composer mechanism, and have packages get dropped into the default "vendor" directory (and of course get rid of the "package" directory).
>>
> I don't have any issues if there is one more directory as I mentioned earlier :) . But if you still think its ok, I am with that also, but I don't love the change in directory structure for if we have blogged some where and we don't remember where , then its editing that place also.

Yeah, I get that. However, we have been alpha (and "Google beta") up until this point, so I'm not overly-concerned about changes.

>> That entails only one problem I can think of. We would need to modify the system config mechanism to look for config files in a different, less-efficient way. It would have to scan all "vendor/*" second-level subdirectories, then look for a "config/aura/default.php" file. Note that this means changing the library packages to use that extra "aura" subdirectory; this is because not all packages in the "vendor" directory are guaranteed to be Aura-style packages, so we need to be able to identify Aura-specific configs. (Of course, we can modify the config caching command to look in this new location when building the cached configs, so the decreased efficiency disappears at that point.)
>>
> In that case you don't need to modify config/aura , if composer installs to vendor/aura/pacakage/Aura.Package/config

Except the point I'm making is that it might be useful to go with the default Composer behavior, rather than add an Aura-specific behavior.

> As in the PR we are talking you can see the extra.installer-path which helps to download it in package folder and the other 3rd party to include or vendor .
>
> According to me the 3rd party libraries are the only one which we don't have access to edit the composer.json which will be downloaded to vendor folder of composer format .

Right. However, it seems like it would be best to have everything in one place (vendor) rather than possibly three (vendor, package, include).

-- pmj

[Hari K T]

I am ok with it , if every one is ok with that :) . So we don't introduce any special things to learn and do .

Anyway, remember we still have can download Aura specific package to package folder and 3rd party to include as in the PR to installers we are discussing .

Thanks

Hari K T
M: +91-9388758821 | W: http://harikt.com/blog

[Hari K T]

I would like to correct some of the things earlier I said .

tl;dr: Should we install packages to the default Composer location of "vendor/Vendor/Package" instead of "package/Vendor.Package", or should we have a custom Composer installer that installs to "package/Vendor.Package"?

* * *

Hey Paul , 

I prefer to go with Vendor.Package format and place in package format itself . I will try to explain why in this case below .

 

While I have a long history with PEAR and the PEAR ecosystem, I have come to the conclusion of late that its time has passed.  Its standards-body activities are being overtaken by the PHP-FIG, and its package distribution mechanism is being overtaken by Composer.

Yes true . Pear is now sleeping and downloading a 25 MB phar is like :P .

 

With that in mind, I'd like to use Composer as the package distribution mechanism for Aura. Specifically, this means having a composer.json file in the "system" repository, and use that to install/update packages from all vendors (including vendors not using Aura style packages).

Currently, Aura has a "package" directory where Aura-style packages go. To install a package, you copy or clone a Vendor.Package into that directory; the directory is flat, with all packages side-by-side.  The system config mechanism scans that directory, looks for each instance of "Vendor.Package/config/default.php", and uses those files for default configurations. That lets the system bootstrap the configuration for each individual package.

Composer, on the other hand, installs to a default directory location called "vendor".  The installed packages are in a 2-directory hierarchy organized by vendor name and package name ("vendor/Vendor/Package").

I would like to correct you here .

So the default format of composer installation for Aura/Router will be like 

vendor/aura/router/Aura.Router/src/Aura/Router

So its not vendor/aura/router/Aura.Router/src/Aura/Router

and is vendor/Aura/Router/src/Aura/Router .

Which means the Aura.Package folder is removed :( . I don't  really like this . Either I need the name there itself as in my repo name . Its easy to find in a google search the repo .

and the current new Aura packages as 

vendor/aura/uri/src/Aura/Uri

Yes small letters . And yes this itself was @pmjones mentioning .

 

that means we have another vendor directory . Notice the small aura/router .

 

Composer allows for project-specific installers.  If you look at <https://github.com/composer/installers> you will see that different frameworks can specify where to put their installed packages. Indeed, it lets them get installed to various different places in the framework structure.  This means a custom Aura installer could put things in the "package" directory using the current conventions.

Yes I have been lately playing it yesterday . So if anyone is interested have a look into the PR

https://github.com/composer/installers/pull/24 

http://packagist.org/packages/hari/aura-system

You can have a look into the composer.json which will be used for system 

https://github.com/harikt/aura-system/blob/master/composer.json

But this composer.json has no effect :P . It still downloads to vendor folder . I am wondering whether we need to have it merged to installers in composer repo . Not sure . A big Thanks to @shama for trying to resolve this. Still struggling here on this .

 

 

However, because Aura packages are designed to be self-contained, I wonder if we should change the Aura system to use the default Composer mechanism, and have packages get dropped into the default "vendor" directory (and of course get rid of the "package" directory).

I don't have any issues if there is one more directory as I mentioned earlier :) . But if you still think its ok, I am with that also, but I don't love the change in directory structure for  if we have blogged some where and we don't remember where , then its editing that place also.

 

That entails only one problem I can think of. We would need to modify the system config mechanism to look for config files in a different, less-efficient way.  It would have to scan all "vendor/*" second-level subdirectories, then look for a "config/aura/default.php" file. Note that this means changing the library packages to use that extra "aura" subdirectory; this is because not all packages in the "vendor" directory are guaranteed to be Aura-style packages, so we need to be able to identify Aura-specific configs.  (Of course, we can modify the config caching command to look in this new location when building the cached configs, so the decreased efficiency disappears at that point.)

In that case you don't need to modify config/aura , if composer installs to vendor/aura/pacakage/Aura.Package/config 

Yes we need aura folder , for else we only scan Aura or aura in vendor directory . I am still thinking and it seems odd to keep in vendor directory . I feel there will be a huge amount of time if its not cached .

Think we have 10 to 20 3rd party vendors . This will make us search in 20 + and aura related packages . And we also want to check whether its aura specific one also . This is like an overload than plug and play as we have now. Just keep the Vendor.Package in package folder .

So I disagree completely with this move ..

I wish we could keep all aura related packages in package folder itself  . And all composer downloading 3 party and which are not supported by composer also to go to vendor directory .

So the only change will be rename include to vendor  and keep the package folder as it is. 

If composer cannot download and install Aura and related libraries which are aura specific I feel we should write a script which will do the functionality .   

So the only thing I would like to point is 

1 ) Keep Aura.Package and Vendor.Package of  aura related ones as in the package folder as of now itself . We will change or do the needful on composer installers , and if it will not work , we can relay on something else .

2 ) change name from include to vendor . So composer can install the 3rd party vendors there .

This makes everything work as we are having now .

Thanks 

[Hari K T]

Hi Guys, 

I would like to update on the composer part .

I am not interested basically to move all Aura.Packages or aura related packages outside package folder .

So first of all I am not going the default way for composer .

So what am I with ?

1 ) So the good news is we can install all Aura.Packages to package folder 

2 ) We will rename the include folder to vendor 

3 ) All aura related other vendor packages can be installed via the "type": "aura-package"

4 ) All other vendors other than aura and not aura related will move to vendor folder .

A sample composer.json of how your aura package will be is 

{

    "name": "hari/contact",

    "minimum-stability": "dev",

    "type": "aura-package",

    "description": "This is a sample contact page",

    "keywords": [

        "aura",

        "sample"

    ],

    "homepage": "http://harikt.com",

    "time": "2012-07-30",

    "license": "BSD",

    "authors": [

        {

            "name": "Hari.Contact Contributors",

            "homepage": "https://github.com/harikt/Hari.Contact/contributors"

        },

        {

            "name": "Hari KT",

            "email": "ktha...@gmail.com",

            "homepage": "http://harikt.com"

        }

    ],

    "require": {

        "php": ">=5.4.0",

        "composer/installers": "*"

    }

}

The PR for that is over https://github.com/auraphp/system/pull/19

This is made happened via the discussion with @shama https://github.com/composer/installers/pull/24 and finally  he coming with supporting a https://github.com/shama/aura-system-installer 

We will move the https://github.com/shama/aura-system-installer installer to @auraphp itself is @pmjones and you guys feels the idea is good .

Basically you don't need the https://github.com/shama/aura-system-installer other than installing via composer .

This way we don't make any changes and we can still plug and play functionality .

There is no hard part for you to know .

How you can do is 

1 ) Download tar or zip from https://github.com/auraphp/system/tarball/master https://github.com/auraphp/system/zipball/master

2 ) Unzip and cd to directory

3 ) if you don't have composer download it  and 

wget http://getcomposer.org/composer.phar

And now run php composer.phar update

What you will see is something like 

$ php composer.phar update

Updating dependencies
  - Installing shama/aura-system-installer (dev-master)
    Cloning d2b4b27d41042ef569a7fe6c0c9dfe54e99dc4a7

  - Installing aura/web (1.0.0-beta3)
    Downloading: 100%

  - Installing aura/view (1.0.0-beta3)
    Downloading: 100%

  - Installing aura/sql (1.0.0-beta3)
    Downloading: 100%

  - Installing aura/signal (1.0.0-beta3)
    Downloading: 100%

  - Installing aura/router (1.0.0-beta3)
    Downloading: 100%

  - Installing aura/marshal (1.0.0-beta3)
    Downloading: 100%

  - Installing aura/http (1.0.0-beta3)
    Downloading: 100%

  - Installing aura/di (1.0.0-beta3)
    Downloading: 100%

  - Installing aura/cli (1.0.0-beta3)
    Downloading: 100%

  - Installing aura/autoload (1.0.0-beta3)
    Downloading: 100%

  - Installing aura/framework (1.0.0-beta3)
    Downloading: 100%

  - Installing aura/uri (1.0.0-beta3)
    Downloading: 100%

Writing lock file
Generating autoload files

You can edit the composer.json to download other vendors also .

The composer.json of system will be

{

    "name": "hari/aura-system",

    "minimum-stability": "dev",

    "require": {

        "shama/aura-system-installer": "*",

        "aura/autoload": "1.0.0-beta3",

        "aura/cli": "1.0.0-beta3",

        "aura/di": "1.0.0-beta3",

        "aura/http": "1.0.0-beta3",

        "aura/framework": "1.0.0-beta3",

        "aura/marshal": "1.0.0-beta3",

        "aura/router": "1.0.0-beta3",

        "aura/signal": "1.0.0-beta3",

        "aura/sql": "1.0.0-beta3",

        "aura/uri": "1.0.0-beta3",

        "aura/view": "1.0.0-beta3",

        "aura/web": "1.0.0-beta3"

    }

}

That's it for now .

Any concerns / critics shoot it :) .

[Paul M. Jones]

Hi all,

After trying both options (using the "vendor" directory per the standard Composer motif, and the Aura-specific "package" directory) we've settled on using the "package" directory after all.

You will notice that the Aura system package now has a composer.json file and composer.phar; you can issue "php composer.phar install" to install the development versions of all packages.

In related news, I was earlier a fan of the idea of making PEAR packages available. Given how tough it has been to work with PEAR/Pyrus, and how surprisingly straightforward it was to work with Composer, I am no longer committed to PEAR packaging. If someone else wants to do so, please feel free to dive in, and I'll be happy to receive it.

Many thanks to @harikt for driving the Composer work, and to @shama for writing up an Aura-specific installer for Composer.


-- pmj

[George Papadopoulos]

Hi all!

Regarding the composer-based install procedure, may i propose an alternative solution?

It is possible to not have Aura packages resides into the package/ dir, but instead to use the default composer's vendor/ dir.

One way to do that is to alter the Aura.Framework's Framework, Config and System classes, to store a list of Aura.* packages internally. The rest (eg the bootstrap sequence) is trivial.

I think this way is more easy to add external Aura style packages into an Aura-based project. Don't forget (if i miss something here please correct me), an Aura style package is a package supplying it's own config/default-or-whatever-mode.php file to get included in the initial startup procedure - aka bootstraping.

Finally, i've made up a fork of Aura.Framework and start experimenting. You can see an example implementation at https://github.com/gpapadopg/Aura.Framework in gpapadopg branch. Then, to create a new Web application, the following index.php file can be used:

<?php

use Aura\Framework\Bootstrap;

// set config mode
$_ENV['AURA_CONFIG_MODE'] = 'www';

// initialize composer autoloader. Be careful with the path :-)
include dirname(__DIR__) . '/vendor/autoload.php';

// set system root...
$root = dirname(dirname(__DIR__));
// ...and aura packages to use
$packages = [
    'Aura.Autoload'     => 'vendor/aura/autoload',
    'Aura.Di'           => 'vendor/aura/di',
    'Aura.Framework'    => 'vendor/aura/framework',
    'Aura.Http'         => 'vendor/aura/http',
    'Aura.Router'       => 'vendor/aura/router',
    'Aura.Signal'       => 'vendor/aura/signal',
    'Aura.Uri'          => 'vendor/aura/uri',
    'Aura.View'         => 'vendor/aura/view',
    'Aura.Web'          => 'vendor/aura/web',
];

// create framework bootstrap
$bootstrap = new Bootstrap($root, $packages);
// and handle/execute web context
$bootstrap->execWeb();

What do you think?

[Hari K T]

Hey George Papadopoulos , 

Thank you for your mail and I hope this may be your first mail to the group, so I welcome you :) .

I will reply inline . Please find it .

Regarding the composer-based install procedure, may i propose an alternative solution?

Sure every one can suggest alternatives :) .

 

It is possible to not have Aura packages resides into the package/ dir, but instead to use the default composer's vendor/ dir.

Yes of-course it can. But why we kept is for aura doesn't want a vendor lock on composer . When building the initial version @Paul's idea and many of them agreed that, aura packages as a drop and work package.

That means if I download via git in package folder it should work , if you download and unzip and place it in package it should work. 

 

One way to do that is to alter the Aura.Framework's Framework, Config and System classes, to store a list of Aura.* packages internally. The rest (eg the bootstrap sequence) is trivial.

I think this way is more easy to add external Aura style packages into an Aura-based project.

Its still possible with composer . For eg : when you add the name to system/composer.json and if the type of that should be aura-package .

An example is https://github.com/harikt/Hari.Contact/blob/master/composer.json#L4

 

Don't forget (if i miss something here please correct me), an Aura style package is a package supplying it's own config/default-or-whatever-mode.php file to get included in the initial startup procedure - aka bootstraping.

Yes you are right.  It also is named currently Vendor.Package .

I am not sure which is the best way. But I still feel the current way works for we don't make a vendor lock on composer even if composer dies.

Thank you

[George Papadopoulos]

I'll also reply inline :-)

Τη Κυριακή, 2 Δεκεμβρίου 2012 2:32:01 μ.μ. UTC+2, ο χρήστης Hari K T έγραψε:

Hey George Papadopoulos , 

Thank you for your mail and I hope this may be your first mail to the group, so I welcome you :) .

I will reply inline . Please find it .

Regarding the composer-based install procedure, may i propose an alternative solution?

Sure every one can suggest alternatives :) .

 

It is possible to not have Aura packages resides into the package/ dir, but instead to use the default composer's vendor/ dir.

Yes of-course it can. But why we kept is for aura doesn't want a vendor lock on composer . When building the initial version @Paul's idea and many of them agreed that, aura packages as a drop and work package.

That means if I download via git in package folder it should work , if you download and unzip and place it in package it should work. 

I think the proposed solution also works with the way you describe... In my web example, you just change the paths and you 're done. This is agnostic to whatever package manager you use.
 

 

One way to do that is to alter the Aura.Framework's Framework, Config and System classes, to store a list of Aura.* packages internally. The rest (eg the bootstrap sequence) is trivial.

I think this way is more easy to add external Aura style packages into an Aura-based project.

Its still possible with composer . For eg : when you add the name to system/composer.json and if the type of that should be aura-package .

 

I must RTFM :-)

It's composer agnostic (as i said earlier). Also you may have multiple applications (eg CLI's, or Web Frontend, Admin Interface and REST api) and in each one you may use/initialize/include (you name it) only the Aura packages you need to use. This way, you can avoid the cost of 'include everything' from Aura packages in each bootstrap/initialization file. Eg, a possible web API interface doesn't need a session, although a Web Frontend does (usually :-).

What do you think?
 

[Hari K T]

Hi , 

 

I think the proposed solution also works with the way you describe... In my web example, you just change the paths and you 're done. This is agnostic to whatever package manager you use.

/me nods 

 

It's composer agnostic (as i said earlier). Also you may have multiple applications (eg CLI's, or Web Frontend, Admin Interface and REST api) and in each one you may use/initialize/include (you name it) only the Aura packages you need to use. This way, you can avoid the cost of 'include everything' from Aura packages in each bootstrap/initialization file. Eg, a possible web API interface doesn't need a session, although a Web Frontend does (usually :-).

What do you think?

About your thought of including everything, Its just a single config file, and we don't create objects . It all with the help of di, and I am not a bench mark guru.

Also I am not sure how different will it become including it as you mentioned  :( .

I can hear other's thoughts on this. 

[George Papadopoulos]

Hari,

in my humble opinion this is just another form of configuration.

This approach is more powerful (i think) in the sense that the developer has complete control on what (or not) to include. Yes, the automatic Aura.*/config/default.php inclusion is the easiest way to go but you lose control when you want to do some overrides on certain packages - maybe there is some form of doing this but i don't now of that yet :-).

Also, you can have Aura packages installed system wide :-P

Last but not least: one misconfiguration on composer.json (or an accidentally downloaded package into the packages/ dir) may can cause trouble as the inclusion of packages is a little bit 'magic' - magic in the sense of not actually seeing from the code what's going on. But please correct me if i am wrong here.

And as far as i remember some major frameworks do something like this by telling you to create the foo app bootstrap and in the list of available classes insert the 'acme' bundle and do this and that, etc

It will become different because you have to specify which packages to use.
 

[Hari K T]

Hi, 

 

in my humble opinion this is just another form of configuration.

Yes, yes I am getting you man. I am not sure whether my message was harsh, sorry if so.

 

This approach is more powerful (i think) in the sense that the developer has complete control on what (or not) to include. Yes, the automatic Aura.*/config/default.php inclusion is the easiest way to go but you lose control when you want to do some overrides on certain packages - maybe there is some form of doing this but i don't now of that yet :-).

From the next release onwards when you look into the config/_packages file 

https://github.com/auraphp/system/blob/develop/config/_packages

you can see the order of inclusion of the aura related package . So you want to add those your package when you need it .

I agree, your idea seems good for it can be placed anywhere. I too was discussing with Paul regarding the more configuration option and at some point I felt like it will become more complex for people to understand and follow.

 

Also, you can have Aura packages installed system wide :-P

Last but not least: one misconfiguration on composer.json (or an accidentally downloaded package into the packages/ dir) may can cause trouble as the inclusion of packages is a little bit 'magic' - magic in the sense of not actually seeing from the code what's going on. But please correct me if i am wrong here.

I feel this is about the installation problem. For when you downloaded and trying to make use of it in development environment. I feel the developer will be able to fix it. No ?

 

And as far as i remember some major frameworks do something like this by telling you to create the foo app bootstrap and in the list of available classes insert the 'acme' bundle and do this and that, etc

Yes Symfony2 tells you to add your bundle in the autoloader yourself. But iirc lithium was also a plug and play.

 

Also I am not sure how different will it become including it as you mentioned  :( .

It will become different because you have to specify which packages to use.

So do you think we need another file 

https://github.com/auraphp/system/blob/develop/config/_packages.web so it will be Web related and 

https://github.com/auraphp/system/blob/develop/config/_packages.cli , so its cli related stuffs etc ?

It seems horrible for me. I will wait for what @pmjones thinks about this . As he is in a conference you want to wait for few days.

Thank you for bringing this before the next release of Aura.Framework . He may be able to help you with this :) .

[George Papadopoulos]

Hari,

Τη Δευτέρα, 3 Δεκεμβρίου 2012 3:27:48 π.μ. UTC+2, ο χρήστης Hari K T έγραψε:

Hi, 

 

in my humble opinion this is just another form of configuration.

Yes, yes I am getting you man. I am not sure whether my message was harsh, sorry if so.

Harsh? no, no, of course not :-)
 

 

This approach is more powerful (i think) in the sense that the developer has complete control on what (or not) to include. Yes, the automatic Aura.*/config/default.php inclusion is the easiest way to go but you lose control when you want to do some overrides on certain packages - maybe there is some form of doing this but i don't now of that yet :-).

From the next release onwards when you look into the config/_packages file 

https://github.com/auraphp/system/blob/develop/config/_packages

you can see the order of inclusion of the aura related package . So you want to add those your package when you need it .

I agree, your idea seems good for it can be placed anywhere. I too was discussing with Paul regarding the more configuration option and at some point I felt like it will become more complex for people to understand and follow.

Yeah, got your point, but any other alternative with some clear instructions, i don't think will confuse people around.
 

 

Also, you can have Aura packages installed system wide :-P

Last but not least: one misconfiguration on composer.json (or an accidentally downloaded package into the packages/ dir) may can cause trouble as the inclusion of packages is a little bit 'magic' - magic in the sense of not actually seeing from the code what's going on. But please correct me if i am wrong here.

I feel this is about the installation problem. For when you downloaded and trying to make use of it in development environment. I feel the developer will be able to fix it. No ?

Hmm, i don't know... it depends on the problem. Usually it is 'oh my, it doesn't run' and start looking around. Talking about this discussion, say eg that you included an Aura package that breaks something - or need an alternative params/default.php. You don't have a way to exclude it except by physically moving/removing it.
 

 

And as far as i remember some major frameworks do something like this by telling you to create the foo app bootstrap and in the list of available classes insert the 'acme' bundle and do this and that, etc

Yes Symfony2 tells you to add your bundle in the autoloader yourself. But iirc lithium was also a plug and play.

 

Also I am not sure how different will it become including it as you mentioned  :( .

It will become different because you have to specify which packages to use.

So do you think we need another file 

https://github.com/auraphp/system/blob/develop/config/_packages.web so it will be Web related and 

https://github.com/auraphp/system/blob/develop/config/_packages.cli , so its cli related stuffs etc ?

It seems horrible for me. I will wait for what @pmjones thinks about this . As he is in a conference you want to wait for few days.

It seems horrible for me also... but, i am not talking about these files. I propose the alternative that instead of creating your app like this (taken from https://github.com/auraphp/system/blob/develop/web/index.php):

<?php
require dirname(__DIR__) . '/package/Aura.Framework/src/Aura/Framework/Bootstrap/Factory.php';
$factory = new \Aura\Framework\Bootstrap\Factory;
$web = $factory->newInstance('web');
$web->exec();

You do it like this:

<?php
use Aura\Framework\Bootstrap;

// initialize composer autoloader

include dirname(__DIR__) . '/vendor/autoload.php';
// set system root

$root = dirname(__DIR__);
// and aura packages

$packages = [
    'Aura.Autoload' => 'vendor/aura/autoload',
    'Aura.Di' => 'vendor/aura/di',

    'Aura.Web' => 'vendor/aura/web',
    'MyPackage' => 'path/to/my/package',
    // etc...
];

$bootstrap = new Bootstrap($root, $packages);

$bootstrap->execWeb();

just to clarify any misunderstanding :-)

Thank you for bringing this before the next release of Aura.Framework . He may be able to help you with this :) .

:-)

[Paul Jones]

Hi George,

First, thanks for taking the time to work with Aura. I appreciate it. :-)

Now, on to your main request. I'm going to sum up, so let me know if this is in accurate. Basically, you're saying things would be better (for various reasons) if we retire the "package/" directory in favor of using the Composer-standard "vendor/" directory structure. In particular, you want control over what config files get processed, and in what order, so you can do overrides. Also, you think the "web" and "cli" portions should be able to have independent package loading, so that things only needed in one don't get configured in the other. Is that about right?

It's hard for me to answer in-line all the points you raised in the earlier discussion, so I'm going to try and give a summary response. Let me know if I missed anything you think was important, and please don't take this as a dismissal. I want to make sure your questions are fully answered, even if you don't like the answers. ;-)

* * *


Order of Config Loading
-----------------------

Part of the idea with Aura was that you'd be able to download a package to the "package/" directory, and the framework would automatically pick up its config without you needing to do anything. That worked great for two years, until we started going stable on everything, and Hari realized that doing overrides would be too hard. I stalled on it for a while but eventually understood that Hari was right. So, we added two files:

config/_mode
config/_packages

The first one makes the config mode something you can see in your project, and that doesn't depend on a particular environment variable that might be available to a web app but not to a cli one.

The second one specifies the order in which package configurations are loaded. This lets you set which package configs overrides which other ones. By default, the "installer-system" Composer installer writes to this file in the order packages are installed, so it happens automatically when you install packages. You can of course edit the file to customize the order. If you don't list a package there, then the config doesn't get loaded, so that does add one extra step when you add a package manually.


Only Load Some Configs
----------------------

As far as the "only configure some packages" issue, I don't have a good answer for you there. I'd like to avoid making people edit the bootstrapping files. The overhead from loading config files, relatively speaking, is pretty low; if that becomes the biggest single bottleneck in an application that uses a database, that's a fantastic level of performance already. Well-constructed config files shouldn't be doing much other that setting scalars, lazy-calls, and closures; those don't have the overhead of, say, including other files.

The config files can be cached, too, so that instead of reading 12 or 20 or more config files from packages, the system only reads one. (Cf. Aura\Framework\Cli\CacheConfig, which is not documented yet.)

So, while I get the idea of loading different config files for a web vs a cli app, it doesn't seem like a gigantic optimization. It'll be *some* optimization, but I don't think it'll be a huge performance gain relative to all the other things that can go on (like hitting a database).


* * *

Again, please let me know if this begins to address your concerns; I'm happy to take it up again as you see fit.

And thanks for looking at Aura in such depth!


-- pmj

[George Papadopoulos]

Paul, i'll respond inline...

Τη Παρασκευή, 7 Δεκεμβρίου 2012 1:17:09 π.μ. UTC+2, ο χρήστης pmjones έγραψε:

Hi George,

First, thanks for taking the time to work with Aura.  I appreciate it.  :-)

:-)
 

Now, on to your main request.  I'm going to sum up, so let me know if this is in accurate.  Basically, you're saying things would be better (for various reasons) if we retire the "package/" directory in favor of using the Composer-standard "vendor/" directory structure.  In particular, you want control over what config files get processed, and in what order, so you can do overrides.  Also, you think the "web" and "cli" portions should be able to have independent package loading, so that things only needed in one don't get configured in the other. Is that about right?

Yep, exactly.
 

It's hard for me to answer in-line all the points you raised in the earlier discussion, so I'm going to try and give a summary response. Let me know if I missed anything you think was important, and please don't take this as a dismissal.  I want to make sure your questions are fully answered, even if you don't like the answers. ;-)

* * *


Order of Config Loading
-----------------------

Part of the idea with Aura was that you'd be able to download a package to the "package/" directory, and the framework would automatically pick up its config without you needing to do anything.  That worked great for two years, until we started going stable on everything, and Hari realized that doing overrides would be too hard.  I stalled on it for a while but eventually understood that Hari was right.  So, we added two files:

    config/_mode
    config/_packages

The first one makes the config mode something you can see in your project, and that doesn't depend on a particular environment variable that might be available to a web app but not to a cli one.

The second one specifies the order in which package configurations are loaded.  This lets you set which package configs overrides which other ones.  By default, the "installer-system" Composer installer writes to this file in the order packages are installed, so it happens automatically when you install packages.  You can of course edit the file to customize the order.  If you don't list a package there, then the config doesn't get loaded, so that does add one extra step when you add a package manually.

OK, I understand this - but I was working with the beta releases for some time ago which didn't have this mode of operation (it was "just load everything from packages/*/config/default.php"). Then start experimenting to get rid of that. With the continuous development on Aura packages, I was left behind and didn't actually see the changes on Aura.Framework - Lack of time :-(
 

Only Load Some Configs
----------------------

As far as the "only configure some packages" issue, I don't have a good answer for you there.  I'd like to avoid making people edit the bootstrapping files.  The overhead from loading config files, relatively speaking, is pretty low; if that becomes the biggest single bottleneck in an application that uses a database, that's a fantastic level of performance already.  Well-constructed config files shouldn't be doing much other that setting scalars, lazy-calls, and closures; those don't have the overhead of, say, including other files.

Yes I am aware of that
 

The config files can be cached, too, so that instead of reading 12 or 20 or more config files from packages, the system only reads one.  (Cf. Aura\Framework\Cli\CacheConfig, which is not documented yet.)  

So, while I get the idea of loading different config files for a web vs a cli app, it doesn't seem like a gigantic optimization.  It'll be *some* optimization, but I don't think it'll be a huge performance gain relative to all the other things that can go on (like hitting a database).

In my point of view it's not so about optimization but on the control of what to have available in my environment to work on.

Let me give you an example:
If I were to use only the Aura.Di in my project, I expect it to be into vendor/<somewhere> folder and (most importantly) not having a dependency on aura/installer-default - which I think semantically is redundant. And in that perspective only the Aura.Framework must depends on aura/installer-default.

And another example:
I am using the Aura packages some time now on a project because of the code elegance and decoupling. The code is created in such a way that is shared across the environments I need to run.

Eg, the main web frontends are the www for the visitor, the admin for administration purposes and an api providing a REST-like interface. The API must not have at all access to session - I mean I am not and I don't want to use session. In that perspective, I think the (automatic) inclusion of session package may lead to problems if I accidentally call somewhere in my code the $user->save() which then tries to save the user to the session environment. The same applies for the Cli interface.

One more thing. The project structure I am using is something like this:
- bin/
  - cli1.php         <-- not using session, web and router
  - cli2.php         <-- not using session but needs uri
- config/
  - www.php          <-- configuration for the WWW
  - admin.php        <-- configuration for the ADMIN
  - api.php          <-- configuration for the API
  - default.php      <-- Common configuration
- include/
- src/
  - mylibs/          <-- code base
- web/
  - www/
    - index.php      <-- handler for WWW. The configuration must include all of the
                       - aura packages, the config/default.php and config/www.php
  - admin/
    - index.php      <-- almost the same as before but with config/admin.php
  - api/
    - index.php      <-- and config/api.php
... etc ...

And the bootstraping is done like that:

<?php

use Aura\Framework\Bootstrap;

// set config mode
$_ENV['AURA_CONFIG_MODE'] = 'api';

// initialize composer autoloader
include dirname(__DIR__) . '/../vendor/autoload.php';

// set system root
$root = dirname(dirname(__DIR__));

// and aura packages
$packages = [
    'Aura.Autoload'=>'vendor/aura/autoload',
    'Aura.Di'=>'vendor/aura/di',

    'Aura.Framework'=>'vendor/aura/framework',
    'Aura.Http'=>'vendor/aura/http',
    'Aura.Router'=>'vendor/aura/router',
    'Aura.Signal'=>'vendor/aura/signal',
    'Aura.Uri'=>'vendor/aura/uri',

    'Aura.View'=>'vendor/aura/view',
    'Aura.Web'=>'vendor/aura/web',

];

$bootstrap = new Bootstrap($root, $packages);
$bootstrap->execWeb();

With Aura.Framework code from my fork (https://github.com/gpapadopg/Aura.Framework)

* * *

Again, please let me know if this begins to address your concerns; I'm happy to take it up again as you see fit.

Of course, the existing solution on framework loading, can do what I want. But it bugs me a little this kind of "magic inclusion".

I mean, if we can have a solution to allow the inclusion (or not) of specific packages - configuration values - call it whatever you want, wouldn't be great?
More specifically (as I am more convinced on what happen behind the scenes based on your arguments) a style of "yes, the framework does all the bootstraping for you but you have also the option to total control your environment" is acceptable?
 

And thanks for looking at Aura in such depth!  

Glad to help :-)
 

-- pmj

[George Papadopoulos]

And there is also another argument I can think of: The Aura.Framework composes the Aura packages (helps with the bootstrap and configuration process) into a ready to use environment. If you notice, you can use it with these packages: Aura.Di, Aura.Autoloader, Aura.Signal and Aura.Web for web apps or Aura.Cli for Cli apps. These are the minimal dependencies. So, if a developer need the basic framework, he/she downloads only these. If he/she needs the Sql package he/she only needs to specify it into the composer.json.

So to summarize my thoughts, I think that:
 - the dependency on aura/installer-default is unneeded in most of the packages (all except the Aura.Framework).
 - the Aura.Framework must have dependencies only on Aura.Autoload, Aura.Di, Aura.Signal, Aura.Web and Aura.Cli.
 - the aura/installer-default must not install aura style packages into the package directory, but into the directory composer uses by default (remember: a user can override that) and keep track of them (the aura ones) by the means of _package and _mode files. The later may be renamed to something like .aura.packages and .aura.mode residing into the composer's install directory.
 - if a developer needs additional Aura packages he/she may able to install them by adding them to the composer.json file.
 - and at last, we may be able to do the Aura.Framework bootstrap process configurable in some way, by providing the ability to load configuration files from other sources, or exclude some of the defaults.

I hope that my thoughts on this conversation are more clear and understandable now. If not, don't hesitate to bother me :-P

What do you think?

[Paul Jones]

On Dec 7, 2012, at 12:48 PM, George Papadopoulos wrote:

Hi George,

> So to summarize my thoughts, I think that:
> - the dependency on aura/installer-default is unneeded in most of the packages (all except the Aura.Framework).
> - the Aura.Framework must have dependencies only on Aura.Autoload, Aura.Di, Aura.Signal, Aura.Web and Aura.Cli.
> - the aura/installer-default must not install aura style packages into the package directory, but into the directory composer uses by default (remember: a user can override that) and keep track of them (the aura ones) by the means of _package and _mode files. The later may be renamed to something like .aura.packages and .aura.mode residing into the composer's install directory.
> - if a developer needs additional Aura packages he/she may able to install them by adding them to the composer.json file.
> - and at last, we may be able to do the Aura.Framework bootstrap process configurable in some way, by providing the ability to load configuration files from other sources, or exclude some of the defaults.

I'll address these out of order; some of them are easier to answer together.

* * *

> - the Aura.Framework must have dependencies only on Aura.Autoload, Aura.Di, Aura.Signal, Aura.Web and Aura.Cli.

If we do that, we pretty severely restrict the usefulness of the framework system "out of the box". There are no templates for views and layouts, no translation, no validating/sanitizing of user input, no session management, no routing of incoming URIs to controllers, no connection to SQL databases of any sort, and no HTTP transport of web responses. All of those things, in varying ways, have been critical for every project I've worked on for at least the past 5 years. To me, the real minimum looks more like this:

- Aura.Autoload
- Aura.Cli
- Aura.Di
- Aura.Filter
- Aura.Http
- Aura.Intl
- Aura.Router
- Aura.Session
- Aura.Signal
- Aura.Sql
- Aura.View
- Aura.Web

That leaves out only Aura.Marshal and Aura.Uri. Adding those two on, after all the others, is of such little impact config-wise that they might as well be included.


* * *

> - the dependency on aura/installer-default is unneeded in most of the packages (all except the Aura.Framework).
>

> - the aura/installer-default must not install aura style packages into the package directory, but into the directory composer uses by default (remember: a user can override that) and keep track of them (the aura ones) by the means of _package and _mode files. The later may be renamed to something like .aura.packages and .aura.mode residing into the composer's install directory.
>
> - if a developer needs additional Aura packages he/she may able to install them by adding them to the composer.json file.

For the record, I hate that "aura/installer-default" is currently necessary. Here's the history:

We wanted Aura packages to be installable in non-Aura systems, *but* we wanted them to be installed to "package/" in Aura systems. (We identify Aura packages in Composer as `"type" : "aura-package"`.)

The problem, as it was explained to me by Composer people, is that Composer needs something to override if you want to treat packages differently at different times. It's all because Composer doesn't necessarily install things in the order listed in composer.json; it finds prerequisites first, but that's independent of their listing order in composer.json.

That means it wasn't enough to have an "aura/installer-system" at the system level to put packages in "package/". There needed to be something for in the packages themselves that acts as a prerequisite, otherwise "aura/installer-system" might get installed last, meaning that the Aura packages won't go in the right place.

Thus, "aura/installer-default" has to be in place in each package. It does absolutely nothing other than extend the default Composer library installer without overriding anything. That makes it a prerequisite when we get to the system framework level. Then the system composer.json overrides it with "aura/installer-system" to put packages in the "package/" directory. Because it's overriding a prerequisite, it gets loaded first, and then all the "aura-package" packages go in the right place.

/me wipes brow

Now: I get you on having a single standard place for all Composer installed libraries (i.e., the "vendor/" directory). I tried it for a while myself. I discovered very quickly that when developing packages, it's really nice to have them *outside* the influence of Composer. It's also nice to be able to download them directly without Composer and drop them into place (with the newly added step that we have to edit the config/_packages file to make sure the system picks it up).

If we put packages in "vendor/aura/package-name" that adds more layers to the directory structure than I'd like to see in daily development. It adds somewhat more to the number of things you have to remember to do when developing a new package. It *possibly* (maybe) interferes with how Composer works by default, when you try to add things there that Composer didn't put there itself.

With all that in mind, I think the "package/" directory is here to stay, along with the detested but necessary-for-Composer "aura/installer-default".


* * *

> - and at last, we may be able to do the Aura.Framework bootstrap process configurable in some way, by providing the ability to load configuration files from other sources, or exclude some of the defaults.

On this one, I can see your point. Perhaps we can work out some way that involves editing *only* config files to achieve it, rather than editing the delivered boilerplate system and Aura.Framework files.

Alternatively, there may be some way for you to write up a framework system of your own that combines the Aura packages in a way you find more conducive to your own way of working. The way the packages are set up makes that very possible, although I'd hate for it to go that way.

* * *

Man, that was long. Let me know what you think.



-- pmj

[George Papadopoulos]

Yeah, very log indeed :-)

Well, I don't want, by any means, put more pressure on you or any other involved in aura development. My opinions are based mainly on the idea of decoupled components - eg replace Aura.Router in a project with some other routing mechanism but use everything else from aura packages.

You have understand my point of view, and I yours :-) So I'll try to work on my fork of Aura packages and develop something useful. Until then we keep in touch by this discussion and PR's on github.

[Hari K T]

Hey George,

We love you to be part of Aura.

There can be some views different, but that doesn't mean we are completely against. 

But you still can make your forks, ie a great place to experiment.

Thank you and hope to see more mails and discussions on your experiments and findings.

Hari K T
M: +91-9388758821 | W: http://harikt.com/blog

[Hari K T]

By the way I forgot to ask you once question from your example 

"Aura.Router in a project with some other routing mechanism but use everything else from aura packages."

Is there something you really feel? If so can we hear why and for what reasons you think so? Is there something we can improve?

Thank you for your time.

Hari K T
M: +91-9388758821 | W: http://harikt.com/blog

[George Papadopoulos]

Hari, in my point of view, all this It's about to make everything better, so, of course no prob :-)

As for the router package, it's just fine. Minimal, elegant and does the job. It was just an example of "using all those libs together but replaced this and that". And (in theory) should be applicable in every framework working with interfaces/abstractions as eg in zf2 authentication component. And as far as I've seen, aura is also done that!

[Hari K T]

to add I also discussed recently with Paul for not fixing it just at package folder as you mentioned. 

The implementation was something different though.

Its like setting the package directory somewhere ( probably a config/somewhere ) , and then set the package path there.

In the Aura\Framework\System

public function getPackagePath($sub = null)

    {

        return $this->getSubPath('package', $sub);

    }

we change the hard coded value package with the one injected.

That makes things easier, but it will not address what you tried to say like making things just for api, load only for admin like that.

Thanks