[RFC] - Namespace Standard for CMS code

[Michael Babker]

We've reached a point where we should start considering the namespacing of our CMS code (this has already happened with Framework classes) and as such we need to define a standard for namespacing of the CMS code base and a transition plan for achieving such.  Right now I'm only going to focus on the libraries folder and its class naming; extensions are a completely different beast and especially with components I don't believe this will be solved in the existing MVC classes.  With that, there are a few things to consider.

1) The "top level" namespace of CMS code.  The three feasible options are to just use Joomla as the namespace (which leaves the possibility of colliding with Framework class names in some packages), using JoomlaCMS as a top level namespace, or using a "sub-namespace" like Joomla\CMS

2) The autoloader to use.  Our JLoader class supports namespace autoloading following the PSR-0 standard.  Composer supports both PSR-0 & PSR-4.  Given we've begun transitioning to Composer supported infrastructure, it may make sense to use their autoloader and plan a deprecation and transition from JLoader.

3) The transition plan.  This affects two aspects of code contributions in the libraries folder; new code and existing code.  For new code, I would suggest that once decided and the structure in place, all new code contributions would be required to be namespaced.  For existing code, we have a couple of options.  One of those options is similar to how we've managed transitions on Framework code and using class aliases to proxy the old code to new (the biggest negative thing here is losing IDE hinting).  A second option is to move the code into a namespaced class while keeping the legacy class in place as a proxy (this has been used when renaming classes previously, see JWeb -> JApplicationWeb).  This one gets tricky though when you consider packages with a class hierarchy (i.e. JInstallerComponent -> JInstallerAdapterComponent -> JInstallerAdapter -> JAdapterInstance) as in this structure changing the parent class of any of the objects would inherently break typehinting somewhere along the way.

Thoughts on the above and a path forward?

[Rob Clayburn]

+1 For namespaces.

I would prefer Jooma\CMS as the top level namespace

+1 for using composer and deprecating JLoader, I've a personal preference for psr-4 but Im not religious about it

Transition plan - I think the class alias way sounds simpler, might give 3rd party devs an additional reason to upgrade their code as well :)

[Johan Janssens]

Hi Michael,

Some thoughts. I think it's time to deprecate the framework and put all focus on moving Joomla forward as a cms. Following this idea, using a Joomla\Cms namespace is overhead. Instead I would use following namespace structure :

Vendor namespace : \Joomla

Library namespace : \Joomla\Library\ or \Joomla\Framework 

Note : To get rid of the name 'framework' I would consider using a second level namespace called 'Library' which would hold all the different Joomla libraries. This falls inline with the naming of the "libraries" folder we introduced in 1.5.

To solve potential naming conflicts only the framework packages that have been integrated in the cms already would need to get their namespace changed. This can easily be doing using a transition strategy.

As for using Composer and PSR-0 or PSR-4. An autoloader that supports both is very easily written, you don't need Composer for that. Instead of using the Composer autoloader, JLoader can be improved to have support for PSR-0 and PSR-4, and transiationed into a Joomla\Library\Loader package.

Finally, for using Composer I suggest that the /libraries/vendor folder is moved to /vendor where it belongs, and the /libraries folder is used to hold the Joomla core libraries (former framework packages).

This restructuring would give you a much leaner and meaner codebase and bring focus back to developing Joomla as the worlds leading open source content management system. 

Johan

[Michael Babker]

So there's a few things you and I will never agree on:

- Path to where the vendor installed dependencies are, they're perfectly OK under libraries IMO as long as our entire application is expected to be in the web root.  Make the CMS so only web assets are there, we can talk.  Until then, every top level folder is a path unusable in a web address (and yes I think vendor has a more decent chance of being used in URLs compared to libraries).

- Writing Joomla solutions when there are viable third party solutions.  Composer's autoloader would work for the core libraries just fine.  There's no reason to bulk up JLoader to do something code we already distribute can handle just fine.  We'd be better off stripping Composer back out if we're going to go that route.

You should poke into JAB next week.  I talk about why exactly the mentality of everything in Joomla's code stack being written by us for being hacked to fit our structure is a bad idea.

--
You received this message because you are subscribed to the Google Groups "Joomla! CMS Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-cm...@googlegroups.com.
To post to this group, send email to joomla-...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-cms.
For more options, visit https://groups.google.com/d/optout.

[Johan Janssens]

Hi Michael,

I completely agree that Joomla should make more use of well established third party libraries and refrain from the NIH syndrome. It's more about doing a proper integration of Composer while solving the different issues you mentioned. 

Putting a Composer PSR autoloader in the core does have it benefits, all in we are talking about 20 lines of code. Try to stick with me here.

There are two scenario's how Joomla is being used :

1. A developer deploy scenario, where Joomla codebase is checked out from Git and deployed onto a production server. In this scenario developers would like to use Composer in the most flexible way and we can assume they have full control over their environment. 

2. A user scenario, where Joomla codebase is downloaded as an archive from Joomla.org and uploaded onto a server using FTP. In this scenario CLI access is very likely not available and it doesn't make sense to provide composer support. User will also not have enough knowledge or potentially access to their environment to make changes. We want things to just work here.

Both scenario's can be catered for by making a clear difference between checking code out from Git and downloading an installable package. The installable package would contain all the dependencies while the code in Git would not. 

Installable packages can be build using a package process; Phing comes to mind. The build process can be run as part of the CI chain, packages can be made available as nightly builds or as releases.

The installable package would not contain a composer.json file (as it does now), and would have all the dependencies included as part of the package. In this scenario dependencies can go into /libraries/vendor to prevent the /vendor path issue (as you mentioned).

What benefits does this approach give :

1. No longer need to commit all the dependencies in Git, making the Git repo more lean and mean and only containing Joomla specific code.

2. Developers are giving freedom to choose where to install their dependencies when checking out from Git. By default the /vendor would be used; if they wish they could use /libraries/vendor or they can easily move their dependencies below root where they should be.

3. Users would not have any potential conflicts with /vendor as in the installable package the dependencies are moved into /libraries/vendor, where the Joomla core autoloader would pick them up. 

This approach would give you the full benefits of Composer, without any drawbacks. Developers would gain additional flexibility, for users things would work as they want. 

Additionally (and with a bit of extra work) this can move Joomla in the direction of a web application platform. If core components become installable using Composer developers can easily choose to remove them. The installable package that users download would have them included. 

Give it some though, just trying to think a bit of the box on my end.

Johan

[Andrew Eddie]

On Monday, 25 May 2015 07:39:18 UTC+10, Michael Babker wrote:
> - Path to where the vendor installed dependencies are, they're perfectly OK under libraries IMO as long as our entire application is expected to be in the web root.  Make the CMS so only web assets are there, we can talk.  Until then, every top level folder is a path unusable in a web address (and yes I think vendor has a more decent chance of being used in URLs compared to libraries).

Are we talking trying to fit this into Joomla 3.x, or having some latitude to actually improve on past mistakes and rearrange things in Joomla 4 - even if Joomla 4 is a very gentle change to make a few key changes?

If the former, there is no wiggle room. If the latter, we can talk about moving things web server resources into a /public/ folder or something.

> - Writing Joomla solutions when there are viable third party solutions.  Composer's autoloader would work for the core libraries just fine.  There's no reason to bulk up JLoader to do something code we already distribute can handle just fine. 

Agree.

Regards,
Andrew Eddie

[Andrew Eddie]

> 1) The "top level" namespace of CMS code.  The three feasible options are to just use Joomla as the namespace (which leaves the possibility of colliding with Framework class names in some packages), using JoomlaCMS as a top level namespace, or using a "sub-namespace" like Joomla\CMS

I think it's important to realise that "CMS" is the name of the application you are building. It's not the name of an API. The API is what you wire up to build the CMS.

I think the Composer standard should be followed: Joomla/PACKAGE where PACKAGE accurately reflects the purpose of the code contained there in.

Joomla/Extension/Component
Joomla/Installer/ModuleInstaller
Joomla/Router/MenuRouter
Joomla/File/YamlReader

whatever.

The user of the API should be oblivious to whether the API they are using comes from the Joomla CMS download, or from the /vendor/ folder as sourced from the Composer distribution system.

> 2) The autoloader to use.  Our JLoader class supports namespace autoloading following the PSR-0 standard.  Composer supports both PSR-0 & PSR-4.  Given we've begun transitioning to Composer supported infrastructure, it may make sense to use their autoloader and plan a deprecation and transition from JLoader.

PSR-4 for our own code - no point in writing our code to PSR-0. Composer can handle external PSR-0 libraries.

> 3) The transition plan.  This affects two aspects of code contributions in the libraries folder; new code and existing code.  For new code, I would suggest that once decided and the structure in place, all new code contributions would be required to be namespaced. 

For new, namespaced code, I suggest "<install path>/api/PACKAGE".

The new /api/ folder is a public declaration that this IS the Joomla API you should be using to develop your extension. It is our contract to you.

Package names should be intuitive and descriptive. CMS is not a package name, it's the name of the application you are building.

> A second option is to move the code into a namespaced class while keeping the legacy class in place as a proxy (this has been used when renaming classes previously, see JWeb -> JApplicationWeb).  This one gets tricky though when you consider packages with a class hierarchy (i.e. JInstallerComponent -> JInstallerAdapterComponent -> JInstallerAdapter -> JAdapterInstance) as in this structure changing the parent class of any of the objects would inherently break typehinting somewhere along the way.

It does get tricky, but it's the best of a bad bunch of options. You eventually get to the point where you need to bump the major version, but I think this will work the best for many packages overall.

Regards,
Andrew Eddie

[Michael Babker]

On Sun, May 24, 2015 at 7:41 PM, Andrew Eddie <mamb...@gmail.com> wrote:

On Monday, 25 May 2015 07:39:18 UTC+10, Michael Babker  wrote:
> - Path to where the vendor installed dependencies are, they're perfectly OK under libraries IMO as long as our entire application is expected to be in the web root.  Make the CMS so only web assets are there, we can talk.  Until then, every top level folder is a path unusable in a web address (and yes I think vendor has a more decent chance of being used in URLs compared to libraries).

Are we talking trying to fit this into Joomla 3.x, or having some latitude to actually improve on past mistakes and rearrange things in Joomla 4 - even if Joomla 4 is a very gentle change to make a few key changes?

If the former, there is no wiggle room. If the latter, we can talk about moving things web server resources into a /public/ folder or something.

I wouldn't suggest making changes to the filesystem structure in 3.x at this point.  That whole B/C promise and whatnot being the biggest hurdle and right now the trend is if it even smells like a B/C break it's going to be difficult to push it through.  So I think discussion on that topic (separate thread from this) fits into the 4.0 discussion, if that ever "officially" starts.

[Michael Babker]

On Sun, May 24, 2015 at 7:33 PM, Andrew Eddie <mamb...@gmail.com> wrote:

> 1) The "top level" namespace of CMS code.  The three feasible options are to just use Joomla as the namespace (which leaves the possibility of colliding with Framework class names in some packages), using JoomlaCMS as a top level namespace, or using a "sub-namespace" like Joomla\CMS

I think it's important to realise that "CMS" is the name of the application you are building. It's not the name of an API. The API is what you wire up to build the CMS.

I think the Composer standard should be followed: Joomla/PACKAGE where PACKAGE accurately reflects the purpose of the code contained there in.

Joomla/Extension/Component
Joomla/Installer/ModuleInstaller
Joomla/Router/MenuRouter
Joomla/File/YamlReader

whatever.

The user of the API should be oblivious to whether the API they are using comes from the Joomla CMS download, or from the /vendor/ folder as sourced from the Composer distribution system.

So how would you distinguish some of our very obviously "CMS only" classes then?  For example, we have JApplicationCms extending JApplicationWeb (Joomla\Application\AbstractWebApplication in the Framework).  Would you call that Joomla\Application\AbstractCmsWebApplication or Joomla\CMS\Application\AbstractWebApplication?  True, CMS most likely isn't needed in the entire API naming structure, but if we continue forward with the modularized Framework pieces and have application specific things added into the mix, somehow we'll need a way to distinguish them.  The Framework's v2 package has a LanguageHelper class right now, the CMS has one too with different functions, so one of the two will need a unique name.

> 3) The transition plan.  This affects two aspects of code contributions in the libraries folder; new code and existing code.  For new code, I would suggest that once decided and the structure in place, all new code contributions would be required to be namespaced. 

For new, namespaced code, I suggest "<install path>/api/PACKAGE".

The new /api/ folder is a public declaration that this IS the Joomla API you should be using to develop your extension. It is our contract to you.

I don't know if I would use a folder explicitly named API.  Unless you plan on dropping the Composer dependencies in there too, it could get confusing to explain in some ways.  Are developers only allowed to extend code from this API or are those vendors also considered part of the API?  I personally look at the entire top level API as small libraries, and in my "Next" repo, I broke the libraries folder into two pieces; core and vendor (core being the application specific stuff and vendor being Composer stuff).  It still keeps all of the code logically grouped together and since we don't have the luxury of shipping the code outside the web root right now and using more commonly accepted lib or src folders for our stuff, this feels like an appropriate compromise.

[Nils Rückmann]

I know it's a big "no way Joomla is doing this", but why not just seeing Joomla as a community and not a product ? The whole namespace collision dilemma would be gone if the Framework and CMS would be different products with different names.  

[Andrew Eddie]

> So how would you distinguish some of our very obviously "CMS only" classes
> then?

Sorry, I was so fixated on library code that I forgot to mention. My
practice, and one I see a lot, is to have an application specify
namespace for the bespoke application code. Usually a tutorial will
have "use MyApp\Application as Application" or similar.

> For example, we have JApplicationCms extending JApplicationWeb
> (Joomla\Application\AbstractWebApplication in the Framework). Would you
> call that Joomla\Application\AbstractCmsWebApplication or
> Joomla\CMS\Application\AbstractWebApplication?

I'd probably do, in principle, something like:

<?php
namespace Cms;

class TheApp extends \Joomla\Application\AbstractWebApplication
?>

> True, CMS most likely isn't
> needed in the entire API naming structure, but if we continue forward with
> the modularized Framework pieces and have application specific things added
> into the mix, somehow we'll need a way to distinguish them.

There are only two types of code. a) the package you import from
Composer, and b) the code you write that runs the application.

The first lives in /vendor/, the second lives in /api/ (in Joomla 4+).

> The Framework's
> v2 package has a LanguageHelper class right now, the CMS has one too with
> different functions, so one of the two will need a unique name.

If we are talking about Joomla 4, you just put the CMS's version into
a legacy layer.

> I don't know if I would use a folder explicitly named API. Unless you plan
> on dropping the Composer dependencies in there too, it could get confusing
> to explain in some ways.

No, it's really quite simple:

/api - The developers API that the application writes to run itself
/config - File based application configuration files (separate file
for a different configuration namespace, e.g. db.json, mail.json, etc)
/extensions - Where you install addons
/libraries - Legacy library support
/vendor - Imported dependencies
/www - Web root resources

Probably a few more but you get the idea.

> Are developers only allowed to extend code from
> this API or are those vendors also considered part of the API?

Both.

> I personally
> look at the entire top level API as small libraries, and in my "Next" repo,
> I broke the libraries folder into two pieces; core and vendor (core being
> the application specific stuff and vendor being Composer stuff).

Bingo - same concept, different labels.

> It still
> keeps all of the code logically grouped together and since we don't have the
> luxury of shipping the code outside the web root right now and using more
> commonly accepted lib or src folders for our stuff, this feels like an
> appropriate compromise.

If you look at the Node or Bower world for example, /node_modules/ is
top level and everyone knows they are the dependancies and you don't
edit them, but you can include them wherever you want. If in a Joomla
4 scenario, we have the luxury of doing a /www/ folder, then I think
it makes sense to put your custom code (/api/ or whatever) and your
imported dependencies (/vendor) at the top level.

Note, this can't be achieved in 3.x.

Those are my thoughts anyway.

Regards,
Andrew Eddie

[Michael Babker]

So... Can we salvage anything useful out of this thread or is namespacing CMS code that isn't already by way of the Framework off the table?

[George Wilson]

I think anything that's currently there will stay non-namespaced until Joomla 4. When we get to the point of starting to backport Joomla 4 features into the 3.x branch (where appropriate) we will keep those classes namespaced. The question is what to do with new library code being designed specifically for the 3.x series.

Kind Regards,

George

[Michael Babker]

::facepalm::

Migrating to namespaced code can start TODAY.  Look at how JRegistry -> Joomla\Registry\Registry was done.  We can also leave empty classes in place and extend them from new objects as appropriate.  If we don't start transitioning NOW then either v4 will have to retain support for the non-namespaced class names or there is going to be a hellacious B/C break forcing folks to rename classes.

There is a reason I brought up the topic now.  It is NOT dependent on the v4 architecture.  It CAN be managed in a B/C manner in a lot of places.  We need to PLAN and EXECUTE it.  Something this project is a massive failure at.

[Andrew Eddie]

>> 1) The "top level" namespace of CMS code. The three feasible options are
>> to just use Joomla as the namespace (which leaves the possibility of
>> colliding with Framework class names in some packages), using JoomlaCMS as a
>> top level namespace, or using a "sub-namespace" like Joomla\CMS

For anything that is shipped with the CMS, not as a package, I'd go
with the application name as the top level namespace (so "CMS" or
"JoomlaCMS" works).

>> 2) The autoloader to use. Our JLoader class supports namespace
>> autoloading following the PSR-0 standard. Composer supports both PSR-0 &
>> PSR-4. Given we've begun transitioning to Composer supported
>> infrastructure, it may make sense to use their autoloader and plan a
>> deprecation and transition from JLoader.

Yes, that (let Composer handle it) is a no brainer.

>> 3) The transition plan. This affects two aspects of code contributions in
>> the libraries folder; new code and existing code. For new code, I would
>> suggest that once decided and the structure in place, all new code
>> contributions would be required to be namespaced.

I agree.

>> For existing code, we
>> have a couple of options. One of those options is similar to how we've
>> managed transitions on Framework code and using class aliases to proxy the
>> old code to new (the biggest negative thing here is losing IDE hinting). A
>> second option is to move the code into a namespaced class while keeping the
>> legacy class in place as a proxy (this has been used when renaming classes
>> previously, see JWeb -> JApplicationWeb). This one gets tricky though when
>> you consider packages with a class hierarchy (i.e. JInstallerComponent ->
>> JInstallerAdapterComponent -> JInstallerAdapter -> JAdapterInstance) as in
>> this structure changing the parent class of any of the objects would
>> inherently break type hinting somewhere along the way.

My thoughts are probably similar to George's, but for different
reasons. I would hold of on making any changes to J3 until we sort out
what J4 looks like, and then change J3 to suit how J4 turns out. If
you start changing things now, it means that a poor call could either
lock J4 into a less desirable path, or require it to apply yet another
compatibility layer.

I feel strongly that J4 needs to be given the maximum amount of
freedom to change things in a way that makes sense for what
applications should look like during its usable life (in other words,
the architecture needs to be forward thinking, not just catching up).

Yes, it could be we find it didn't matter after all and we could have
changed things now, but I also don't think anything is lost by waiting
and concentrating efforts completely on J4 or the new Web Services app
(which is, hopefully, a J4-esque application anyway).

Regards,
Andrew Eddie