Discussion - Framework v2 Roadmap

[Michael Babker]

Please use this thread to discuss the roadmap for version 2 of the Joomla! Framework.  The roadmap is published at http://developer.joomla.org/news/603-framework-v2-roadmap.html.

[Hils]

On Thursday, 12 February 2015 15:18:01 UTC, Michael Babker wrote:

Please use this thread to discuss the roadmap for version 2 of the Joomla! Framework.  The roadmap is published at http://developer.joomla.org/news/603-framework-v2-roadmap.html.

Great news!

Will the CMS use this new version of the Framework? I know that ideally both should dovetail but ultimately, is it the CMS or Framework that will take priority if there is a conflict.

Again - great news :)

[Michael Babker]

I'd say it should be a goal, but it isn't a decision for me to make anymore.  Each package is going to have to be looked at on a case-by-case basis.  In CMS 3.4, there are a few packages integrated where it's been relatively simple to do so, and there are more packages where this is a possibility without breaking B/C in the CMS.  Others will have to wait for 4.0 to see full integration.

Pain points in the CMS are going to be the cache, event, and profiler packages.  Those received major overhauls and integrating those may be difficult if it is desired.  In the case of the event package, this affects CMS plugins.  For packages with a clear hierarchical tree, like the application package, it would be a break to adjust the CMS classes in this tree to use the Framework classes immediately.

--
Framework source code: https://github.com/joomla-framework
Visit http://developer.joomla.org for more information about developing with Joomla!
---
You received this message because you are subscribed to the Google Groups "Joomla! Framework Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-frame...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-framework.

--
- Michael

Please pardon any errors, this message was sent from my iPhone.

[George Wilson]

Will the CMS use this new version of the Framework? I know that ideally both should dovetail but ultimately, is it the CMS or Framework that will take priority if there is a conflict.

Hi Hils,

Tough question. It is certainly (from the framework teams perspective) our intention for the CMS to use as much of the code as possible.

However many of these classes are not compatible with the CMS as we can take the liberties of making the backwards incompatible changes that were just not possible with version 3.x of the CMS (as an example will completely remove support for the "_QQ_" constant in the language package). In the majority of cases whilst minor it makes it pretty hard to implement and so it is better left to keep the CMS as is and make the breaks in 4.x. In cases where changes are minor the framework packages are already in place (for example the JUri class will extend the frameworks Uri from 4.x) and there are one or two more packages where we can probably do similar things. There will finally be a few packages where we will be making much more major changes (for example the Form package). In that case the likelihood is that the CMS will maintain it's own form package or if using the frameworks will only loosely use it.

From a PLT perspective I should also add no firm decision has been made for version 4.x of the CMS. However I'd suggest that the likihood is that at that time the majority (if not all) the framework packages will be used in the CMS

Kind Regards,

George

[Dimitris]

First of all I have to state that I haven’t play lately with framework (shame on me, I know :) ) so my suggestions might be already implemented (?).

Anyhow what I found very useful and I would like to see them implemented first on Framework and then on cms are:

1. Ability to have development and live environments

2. Ability to fulfill the db with fake data

3. CLI app for creating app scaffoldings

[klas berlič]

Regarding cache package - it highly depends on 4.x CMS architecure what is needed to implement it. Current framework cache package is nothing more than 80% or less of current cache storage layer (called handlers in csm) features, while it is totaly lacking what is called controllers in currents cms cache - higher level...you can even call them helpers.. functions that help cms to do caching. Those are tightly coupled with cms architecture, so it is impossible to predict what will be needed without determining that first, but surely there will be a need for some sort of cms caching layer as framework only takes care of sets and gets.

Regards,

Klas

--

[Michael Babker]

On Thu, Feb 12, 2015 at 3:01 PM, Dimitris <d.gram...@gmail.com> wrote:

1. Ability to have development and live environments

I don't know if this is necessarily a framework's responsibility.  In practice, I'm able to deploy the same code to staging servers and production by hooking each of my environments up to my git repository and managing them through git and Composer.  There are some ways the Framework may be able to differentiate between those environments, and there are ways we can build tools to make this easier (one of the biggest challenges is if you're managing database schemas), but ultimately a full fledged toolset in the Framework for managing deployments might be a bit too much.

2. Ability to fulfill the db with fake data

Again, this is something the database package is really weak in.  Using Symfony and Doctrine together, you can build out a simple command to load up sample data using fixture classes.  In truth I'm not 100% certain if this is a feature supported in their "core" platforms or if it's accomplished by addon tools developed through their communities, but I know it's easy to accomplish in that environment.

3. CLI app for creating app scaffoldings

A post 2.0 goal for me is to start building something similar to Symfony's Standard Edition; an application you can clone out of GitHub, answer some questions, and you have the base scaffolding for an application generated for you following our best practices.  Considering the only two live demos I can point to for Framework apps are the Issue Tracker and Framework site itself, it's a little tricky to say "here's how you get started" at the moment and it's an issue we need to address.

[Michael Babker]

For the Cache package specifically, I think it might be possible in CMS v4 to re-work the internals of its JCache and JCacheController APIs to work with the Framework's handlers.  Admittedly that isn't a part of the CMS or Framework I'm all that familiar with, but I do see the couplings to the infrastructure that you mention.  I took a look at a few packages that came up in a search for "cache" on Packagist and didn't really see anything that jumped out as having that type of logic, so it may be that the Framework doesn't necessarily need to ship those style classes but leave them to an application to implement as needed.  Or maybe we can build a top level interface and base class to build a base API for them.  Plenty of possibilities IMO.

[Adam Bouqdib]

I think the forms package needs a complete overhaul really. It would be great to be able to create forms from JSON or registry objects.

[Rob Clayburn]

Hi

That sounds a really great road map. I'd be really interested in helping out with the forms and database stuff.

[Andrew Eddie]

My 2c worth.

## Consolidated Code Repository

I personally don't think this is optimum. While, sure, it makes checking out the Framework easy, it does complicate matters. 

It doesn't promote isolation of the code (we've had cases where tests passed in the full stack, but didn't on an individual package because some test scaffolding bled into other tests). 

It makes the tests for each package longer to run (if I want to make a quick fix to Keychain, I have to wait for all the other tests to run first).

It doesn't allow for incremental changes to individual packages to happen when they are needed (for example, many packages could go 2.0 right now, but it's not happening because some packages are not quite ready). 

My personal preference, and it's come from working in both the Composer and NPM universes, is that individual package are the point of truth, manage their own dependencies (the less the better) and their own lifecycle (some packages will get a lot of support and change a lot, some won't - so be it).

In terms of being able to download the whole Framework, this can be done with Composer's create-project feature. In fact, you don't want to do that at all. You want to spin up a half-dozen examples of starter applications (command line, REST, web site, et al) that people can pull down and have a skeleton application up-and-running very quickly.

## Documentation

Given the previous comment, I feel strongly that documentation should still live with each package.

However, the intention was always so be able to build the programmers API into one site (from the DocBlocks), and also compile all of the package documentation into one master site (something like what Laravel does would be what I would aim for).

The advantage of leaving the docs in the same repo as the code is that you only need one pull request to check that the documentation has been adjusted appropriately for any given code change. 

To put it another way, it's easy to automated collecting docs from several sources. It's not easy to automate checks that there is a PR for code (and tests), and a PR for docs. The split strategy has never, ever worked for the CMS, but having docs and code in one contributor PR has worked in the Framework for many years.

## Minimum Supported PHP Version

I think each package should be able to govern what it needs as a minimum version, and 5.4/5.5 should be allowed on merit.

## Package Specific Goals

### Cache

I've actually been thinking lately that a Cache is just a storage layer, and that this can be combined with the Database abstraction layer.

### Filesystem

I really question whether this is necessary any more. At it's heart, it's really just a storage adapter.

### Form

This needs a lot of work and split into several packages: data validation and HTML form building.

### Language

I'd love to see Language converted to an adapter pattern so the the API can use any sort of language file/storage format, not just the Joomla .ini definition. It would then be a good candidate for a PSR.

### Log

This is a bit of a cross-over comment, but I think it would be a good exercise to implement Monolog in the CMS, and build out a legacy JLog interface layer. 

To be honest, I think that strategy should be used more and more to replace all the old junk in the CMS.

... 

That will do for now :)

Regards,

Andrew Eddie

[klas berlič]

Cache indeed needs storage to do its job, but it is not just that, it has numerous functions that come on top of storage the same with databases, you can't degrade them both just to storage, it's like you would say car is the same as engine.
Examples of such functionalities of caching package that separate it from what I would cosider pure read/write storage: expiration control, id management, auto garbage cleaning,and convenience functions for on-demand management. Controllers are coupled with application and should be part of application library as Michael suggested, with the possible exception of function/callback cache which is usefull in wide array of applications, here I would +1 on top level interface.

But indeed having a common storage package is a must, such package would be implemented not only by database or cache, but also by session. And for filesystem - +1 for storage adapter.

Cache            >                

Database        >   Storage  >  Filesystem

Session          >

One more suggestion for Input. This is more application stuff, but simple modification to implement bookkeeping of inputs requested trough API (from quick look it seems only method to return inputs array is needed) would create a base for auto id calculation for view cache in application, dropping the need for manually listing those safe parameters in display method.

Regards,

Klas

[Michael Babker]

On Mon, Feb 16, 2015 at 5:30 PM, Andrew Eddie <mamb...@gmail.com> wrote:

## Consolidated Code Repository

I personally don't think this is optimum. While, sure, it makes checking out the Framework easy, it does complicate matters. 

It doesn't promote isolation of the code (we've had cases where tests passed in the full stack, but didn't on an individual package because some test scaffolding bled into other tests). 

It makes the tests for each package longer to run (if I want to make a quick fix to Keychain, I have to wait for all the other tests to run first).

It doesn't allow for incremental changes to individual packages to happen when they are needed (for example, many packages could go 2.0 right now, but it's not happening because some packages are not quite ready). 

My personal preference, and it's come from working in both the Composer and NPM universes, is that individual package are the point of truth, manage their own dependencies (the less the better) and their own lifecycle (some packages will get a lot of support and change a lot, some won't - so be it).

In terms of being able to download the whole Framework, this can be done with Composer's create-project feature. In fact, you don't want to do that at all. You want to spin up a half-dozen examples of starter applications (command line, REST, web site, et al) that people can pull down and have a skeleton application up-and-running very quickly.

Some of my griping about the current structure is based on how disconnected everything feels and how difficult it is in this structure to get attention around updates.  Since Framework 1.0 was released, there have been a grand total of zero announcements regarding package updates.  Granted, it's as much my fault as anyone's (if not more), but it's really difficult to promote package releases when effectively what we're doing is "hey, we merged a pull request, here's an updated package!".  I doubt anyone wants to see 3 dozen "Framework <Package> 2.0 Released" tweets, nor would it do well for us to have a single "Framework 2.0 Released" announcement when a majority of the stack has had 2.0 for potentially weeks or months (the DateTime package from last year's GSoC was released as 2.0 in September with no fanfare).  The only unifying thing about our present structure is the brand name, and I'd go so far as to say what we have isn't framework material nor is it being managed in a way where it could be considered such.

## Documentation

Given the previous comment, I feel strongly that documentation should still live with each package.

However, the intention was always so be able to build the programmers API into one site (from the DocBlocks), and also compile all of the package documentation into one master site (something like what Laravel does would be what I would aim for).

The advantage of leaving the docs in the same repo as the code is that you only need one pull request to check that the documentation has been adjusted appropriately for any given code change. 

To put it another way, it's easy to automated collecting docs from several sources. It's not easy to automate checks that there is a PR for code (and tests), and a PR for docs. The split strategy has never, ever worked for the CMS, but having docs and code in one contributor PR has worked in the Framework for many years.

That works well for package specific documentation.  Where do you put documentation on general best practices for the entire code stack, or tutorials on building applications or resources with our code?  The only points of truth we have right now are the API site (with the automated doc block parsing) and the 3 dozen README files.  Laravel (https://github.com/laravel/docs) and Symfony (https://github.com/symfony/symfony-docs) both maintain docs in a consolidated location on GitHub and publish them via their main website.

I don't think we need automated checks on whether a pull request has documentation either.  What we do need is an established contributor workflow that is crystal clear on all requirements.  Symfony has it right (http://symfony.com/doc/current/contributing/code/patches.html) IMO; I spent longer writing the documentation pull request than I did putting together my first code pull request because of how well everything was explained.

[Andrew Eddie]

On 17 February 2015 at 19:48, klas berlič <klas....@gmail.com> wrote:
> Cache indeed needs storage to do its job, but it is not just that, it has
> numerous functions that come on top of storage the same with databases, you
> can't degrade them both just to storage, it's like you would say car is the
> same as engine.

That analogy isn't quite right. It's saying if you are designing a
car, choosing "which engine" is an implementation details, and all you
provide is an interface to the transmission and fuel system. And in
the process of design, you want to delay the decision of selecting
which engine you want to use for as long as possible (cf Clean or
Hexagonal Architecture).

So in terms of getting some data, my API should not care, initially,
or as far as my unit tests are concerned, whether data is coming from
a JSON file, canned data in memory, or a full blown instance of
Teradata with multiple caching layers.

The point is caching is an implementation detail but at it's heart
it's a type of storage. For example, a "find" method in a model should
connect to a single storage adapter. That adapter may choose to
implement a caching layer, but the model should not have to worry
about both a persistent storage layer and a caching layer (that's an
implementation detail it should not care about).

> Cache >
> Database > Storage > Filesystem
> Session >

Almost.

Project 1: Model -> Storage Interface > Filesystem
Project 2: Model -> Storage Interface > Database
Project 3: Model -> Storage Interface > Cache > Database (implements
CacheableDatabase)

The point is that the model is interchangable across projects, and is
very easy to test. But the base code for a, for example, Redis cache
should extend from StorageInterface is my point. It's just CRUD.

> One more suggestion for Input. This is more application stuff, but simple
> modification to implement bookkeeping of inputs requested trough API (from
> quick look it seems only method to return inputs array is needed) would
> create a base for auto id calculation for view cache in application,
> dropping the need for manually listing those safe parameters in display
> method.

On that one I think I'd start to follow Symfony's request and response
models (the Node Express package uses this pattern also).

Regards,
Andrew Eddie

[Andrew Eddie]

On 18 February 2015 at 00:28, Michael Babker <michael...@gmail.com> wrote:
> Some of my griping about the current structure is based on how disconnected
> everything feels

It should if you are adhering ruthlessly to the Single Responsibility
Principle. That's actually a really good problem to have :)

> and how difficult it is in this structure to get attention
> around updates. Since Framework 1.0 was released, there have been a grand
> total of zero announcements regarding package updates.

I believe I sent in PR to move Keychain to 2.0 some time ago ;)

> Granted, it's as
> much my fault as anyone's (if not more), but it's really difficult to
> promote package releases when effectively what we're doing is "hey, we
> merged a pull request, here's an updated package!". I doubt anyone wants to
> see 3 dozen "Framework <Package> 2.0 Released" tweets,

No, that's not what I'm thinking at all, except perhaps for day 0.
After that, each package is master of its own destiny, which may
include bumping major versions from time to time, or accepting that
its season of usefulness is at an end.

> nor would it do well
> for us to have a single "Framework 2.0 Released" announcement when a
> majority of the stack has had 2.0 for potentially weeks or months (the
> DateTime package from last year's GSoC was released as 2.0 in September with
> no fanfare). The only unifying thing about our present structure is the
> brand name, and I'd go so far as to say what we have isn't framework
> material nor is it being managed in a way where it could be considered such.

No, and I think a mindset change is needed. The overriding fact is we
are never going to have the resources to do a full-stack framework -
and that's mostly due to the fact that we have no major corporate
users to back the Framework up.

To solve that, I think you need to change tack and use the Framework
to upgrade the CMS one package at a time, using a 1.5-style legacy
layer to bridge the changes. That way, the CMS is using the code and
there is no stupid argument about whether the Framework deserves to
have the name Joomla.

Yes, that was how Platform 11.1 was supposed to work but the problem
with that was the monolithic release system. Composer support for the
CMS fixes that, but I still think trying to synchronise the versions
of 30 packages, where there is mostly no change in any of the
packages, is a messy system to support.

If you go fully modular, a bug fix for a package can be merged and
released on the same day - simple! And semantic versioning is easy to
work out. The alternative, which is the current situation, is PR's
wait until you get to a point where you "feel" like a release is
needed, or you are constantly argue to delay the release to slip in
another fix. Why not just fix each package and bump the package's
version when you need to?

>> ## Documentation

> Laravel
> (https://github.com/laravel/docs) and Symfony
> (https://github.com/symfony/symfony-docs) both maintain docs in a
> consolidated location on GitHub and publish them via their main website.

Yes, and both of those projects have solid companies backing them so
they have the resources to maintain superb docs. They are also mature,
full stack offerings. The Joomla Framework isn't anywhere close to
being able to claim that. When the CMS is entirely built off Framework
packages, we can revise the status :)

Anyway, the extra docs you talk about go into the framework.joomla.org
site itself, separate from the individual package docs. That site
aggregates both the PHPDoc API pages and also package docs themselves.

That way, the rules of contribution are simple:

1. The developer MUST include DocBlocks (that pass linting rules).
2. The developer MUST provide explanatory notes about the basic usage
of the package with the PR.
3. The developer MAY choose to add additional user tutorials for one
or a combination of many packages.

At the end of the day, I could live with docs all in one repo,
providing reviewing strictly enforces that docs are a requirement with
code changes. But, I think a solid README is still vitally important
for the developer doing a search on Packagist.org and coming to the
package on Github and finding out what it does. If Packagist follows
the NPM example, it will eventually start consuming, and potentially
indexing that content (at least, that's what I'd do).

My main concern is a) encouraging (forcing?) developers to have good
documentation habits, and b) keeping the number of places they need to
contribute to 1 if at all possible. I just think 1 PR and a script to
aggregate docs is a better contributor experience than 2 PR's more
often than not. The code reviewer also only has to look in one place
to find out how new features work.

Regards,
Andrew Eddie

[Michael Babker]

On Tue, Feb 17, 2015 at 5:11 PM, Andrew Eddie <mamb...@gmail.com> wrote:

On 18 February 2015 at 00:28, Michael Babker <michael...@gmail.com> wrote:
> Some of my griping about the current structure is based on how disconnected
> everything feels

It should if you are adhering ruthlessly to the Single Responsibility
Principle. That's actually a really good problem to have :)

The packages themselves are fine in terms of dependencies and independent use/testing.  But there's 30+ resources moving in 30+ directions with no cohesion between them.  It feels like to me we have a collection of packages that are thrown under a brand name for the sake of grouping them.  PHP League already has claim to that organizational structure and are doing great things with it ;-)

> and how difficult it is in this structure to get attention
> around updates.  Since Framework 1.0 was released, there have been a grand
> total of zero announcements regarding package updates.

I believe I sent in PR to move Keychain to 2.0 some time ago ;)

You did.  And I pretty much sat on it for the sole reason that it was at a point in time where I was getting burned out by the project, by life circumstances, and felt like I was the only person really using or maintaining the code.  And there was a good chunk of time where I really considered just pulling the plug on it all and getting what we could reintegrated to the non-namespaced CMS classes and letting someone else absorb what was salvageable (maybe the League could have absorbed our GitHub, & Profiler packages, the former with a bit of retooling).  Right now I feel a little better about things.

> Granted, it's as
> much my fault as anyone's (if not more), but it's really difficult to
> promote package releases when effectively what we're doing is "hey, we
> merged a pull request, here's an updated package!".  I doubt anyone wants to
> see 3 dozen "Framework <Package> 2.0 Released" tweets,

No, that's not what I'm thinking at all, except perhaps for day 0.
After that, each package is master of its own destiny, which may
include bumping major versions from time to time, or accepting that
its season of usefulness is at an end.

> nor would it do well
> for us to have a single "Framework 2.0 Released" announcement when a
> majority of the stack has had 2.0 for potentially weeks or months (the
> DateTime package from last year's GSoC was released as 2.0 in September with
> no fanfare).  The only unifying thing about our present structure is the
> brand name, and I'd go so far as to say what we have isn't framework
> material nor is it being managed in a way where it could be considered such.

No, and I think a mindset change is needed. The overriding fact is we
are never going to have the resources to do a full-stack framework -
and that's mostly due to the fact that we have no major corporate
users to back the Framework up.

A lot of my gripe in this block is all around marketing, believe it or not.  How do we promote the Framework, updates of the code, or highlight new features when everything is moving at its own pace and everything basically equates to a merged PR is a new release?  There is just no feasible way right now to gather any attention around what is happening in the code.  You can't run posts like https://make.wordpress.org/core/2015/02/09/emoji-feature-plugin-for-4-2/ or http://symfony.com/blog/a-week-of-symfony-424-9-15-february-2015 highlighting anything and there's no enticement from third parties to follow a blog feed for the Framework or any social media outlet if people are just going to be told "follow GitHub, it's our only communication source" (I know that isn't the case, but dumbing things down to a least effort required scenario).  Short of spamming the Twitter feed with the GitHub activity, I don't have an answer to "how can we promote this project".

You say we can't do a full-stack framework.  I say we can.  Joomla's been doing it for the last decade.  Granted, that framework has been geared toward a more specific purpose and implementation (the CMS), but we already have the baseline of tools needed to build from.  What needs to happen is some of that code just needs some cleaning up to be less coupled to its original implementation in the CMS (I think the Form package is the most guilty of this right now) and we need to be looking at ways to improve the code which helps our packages evolve and in turn creates a stronger developer API for the CMS.  Regardless of the Framework existing or not, our database package needs a lot of work as it pertains to database schemas and handling schema updates between application releases; there is no good reason to require raw SQL as the schema manager in today's ecosystem.

To solve that, I think you need to change tack and use the Framework
to upgrade the CMS one package at a time, using a 1.5-style legacy
layer to bridge the changes. That way, the CMS is using the code and
there is no stupid argument about whether the Framework deserves to
have the name Joomla.

Already working on that where we can.  3.4 got a handful of Framework packages (ones that were in essence straight drop-in replacements).  So far they've been simple exercises.  I'm interested in stepping up the complexity another level; how can we integrate packages which don't have a strict hierarchy without breaking compatibility in the CMS?  This excludes the application and database classes at the moment, but I'm thinking we could integrate other packages (Date or Language for example) in a way similar to how we handled the URI package (dropping it in, keeping the JUri class for CMS features, and relying on the rest to come from the Framework package).

Yes, that was how Platform 11.1 was supposed to work but the problem
with that was the monolithic release system. Composer support for the
CMS fixes that, but I still think trying to synchronise the versions
of 30 packages, where there is mostly no change in any of the
packages, is a messy system to support.

If you go fully modular, a bug fix for a package can be merged and
released on the same day - simple! And semantic versioning is easy to
work out. The alternative, which is the current situation, is PR's
wait until you get to a point where you "feel" like a release is
needed, or you are constantly argue to delay the release to slip in
another fix. Why not just fix each package and bump the package's
version when you need to?

What we end up with in a fully modular scenario isn't a framework, and if that's the direction we're heading then we need some re-branding to not use that term to describe what we're doing.  Which is in essence where we are today and have been since Framework 1.1's release.  Everything has been moving at its own pace (if even moving at all) and we depend almost entirely on Packagist and Google searches to even find us or what's going on with our code.

>> ## Documentation
> Laravel
> (https://github.com/laravel/docs) and Symfony
> (https://github.com/symfony/symfony-docs) both maintain docs in a
> consolidated location on GitHub and publish them via their main website.

Yes, and both of those projects have solid companies backing them so
they have the resources to maintain superb docs. They are also mature,
full stack offerings. The Joomla Framework isn't anywhere close to
being able to claim that. When the CMS is entirely built off Framework
packages, we can revise the status :)

Anyway, the extra docs you talk about go into the framework.joomla.org
site itself, separate from the individual package docs. That site
aggregates both the PHPDoc API pages and also package docs themselves.

I'd suggest just writing the documentation before figuring out how many places we want to duplicate it in.  And I would seriously argue against duplicating the PHPDoc stuff between the API & Framework sites; we already have enough problems with Joomla related content being spread across too many domains, let's not add to that.

Building applications, using the database API, or working with the Crypt package requires you to reverse engineer the code to figure out how to implement the tools.  It's been an issue for years and we just keep saying "well we'll require docs with new code"; nobody is addressing the decade of missing documentation we have in the project and short of doing a code-freeze until everything is properly documented, I don't see it happening anytime this decade.

That way, the rules of contribution are simple:

1. The developer MUST include DocBlocks (that pass linting rules).
2. The developer MUST provide explanatory notes about the basic usage
of the package with the PR.
3. The developer MAY choose to add additional user tutorials for one
or a combination of many packages.

It's that MAY that gets us.  I can almost guarantee you that next to nobody is aware that Registry 1.3 (included in CMS 3.4) can be iterated over or counted, or that outputting the data to a PHP file now supports namespaced classes, and the only way to figure that out is to follow the commit history.  But all of those changes meet the MUSTs of your list of rules.

At the end of the day, I could live with docs all in one repo,
providing reviewing strictly enforces that docs are a requirement with
code changes. But, I think a solid README is still vitally important
for the developer doing a search on Packagist.org and coming to the
package on Github and finding out what it does. If Packagist follows
the NPM example, it will eventually start consuming, and potentially
indexing that content (at least, that's what I'd do).

Personally, I don't think a README file (or doc blocks) should be the point of truth for documentation on any code.  IMO a repo's README file should be an executive summary of what the package is doing, what you need in order to use the package, and links to appropriate resources (full documentation, third party APIs, example applications, support queues, etc.).  Cramming everything into a README is just piss-poor documentation management IMO, and if you actually have a strong level of documentation (something 90% of our code lacks), those files are going to be excessively long and have conflicts all the time if the Framework code repos were a quarter as active as the CMS or any other proper Framework stack.

[Andrew Eddie]

That's all just my opinion :P You don't have to take any of it on board ;)

Regards,
Andrew Eddie

[Nils Rückmann]

Am i the only person who's asking why there is a discussion about the roadmap AFTER the official announcement ?

Besides that, i think it's way more productive to have one discussion for each thread.

And because i'm not getting tired to annoy people with my opinion, here it comes:

### Relicensed to LGPL

+1

### Consolidated Code Repository

The current amount of packages is rediculous, indeed. But instead of throwing everything in one repository, we should have one repository for each logical block. MVC is a good example. It's one block, no one wants to use a model from framework x and a view from framework y. I've never understood why those classes were separated in the first place.  

### Documentation

Documentation is essential for every piece of code, even if you have a big brand like "Joomla". And if Joomla-FW separates the documentation and code, it will be the same mess like Joomla-CMSs documentation. Having documentation and code in the same repository doesnt't mean that there can't be another repository for global documentations like tutorials or guidelines.

### Minimum Supported PHP Version

Definitly 5.4 or even 5.5

### Path Constants

+1

### Database / Filesystem

There are plenty other packages which are doing a great job (e.g. doctrine or flysystem), but a stable framework should use his own api to be flexible. So why not creating a persistence api like Typo3 Flow ?

[Michael Babker]

On Wed, Feb 18, 2015 at 11:19 AM, Nils Rückmann <ma...@nueckman.de> wrote:

Am i the only person who's asking why there is a discussion about the roadmap AFTER the official announcement ?

The intent was to gather additional ideas from interested folks.  What we published was only based on a few individuals collaborating on ideas, I wanted to ensure we gave the opportunity to seek out additional ideas before committing to anything firm as there is the possibility someone has an idea where we all just think "why didn't I think of that before!?".

### Documentation

Documentation is essential for every piece of code, even if you have a big brand like "Joomla". And if Joomla-FW separates the documentation and code, it will be the same mess like Joomla-CMSs documentation. Having documentation and code in the same repository doesnt't mean that there can't be another repository for global documentations like tutorials or guidelines.

I think as long as it's all at least logically grouped and machine parsable (I want to actually use the Framework site for something more than a marketing landing page as this thing grows, using our GitHub package to sync it to the site should be possible), I can live with whatever the choice may be.  For me the important things are getting things documented and having a structure to expand the documentation without self-imposed limitations.

### Minimum Supported PHP Version

Definitly 5.4 or even 5.5

Personally, I think if you're still stuck in PHP 5.3 or 5.4 land, you're safe using Framework 1 (in part because the CMS is starting to consume it so it'll most likely be supported so long as CMS 3 has support).  Framework 2 can make the jump to 5.5 without an issue.

### Database / Filesystem

There are plenty other packages which are doing a great job (e.g. doctrine or flysystem), but a stable framework should use his own api to be flexible. So why not creating a persistence api like Typo3 Flow ?

As far as the filesystem code goes, there are a lot of "CMS-isms" present to deal with sub-par hosting environments.  Otherwise, the only coupling to it in our existing packages is in the Form package for the file/folder list fields.  I'm not at the point where I'd say we HAVE to retain those packages (actually, I'm leaning closer to Filesystem could be deprecated or be used as a wrapper around Flysystem (in the case of the CMS) than saying it has to be updated), but I would say any feasible idea is welcome.

[Andrew Eddie]

On Thursday, 19 February 2015 02:19:48 UTC+10, Nils Rückmann wrote:

### Consolidated Code Repository

The current amount of packages is rediculous, indeed. But instead of throwing everything in one repository, we should have one repository for each logical block. MVC is a good example.

I was writing to disagree with you, but then I decided you are actually right. Those packages act as a single, highly coupled MVC triad so it would be fine to combine them. 

It could make for messy namespaces though. 

\Joomla\MVC\Model is a bit clunky.

It would almost be better to throw them back into the Application package.

\Joomla\Application\Controller

\Joomla\Application\Model

\Joomla\Application\View

Or, you use Action-Domain-Response as a super container (ADR is, in my opinion a better triad to describe how web applications work).

\Joomla\Action\Controller (CLI? REST? Who cares)

\Joomla\Domain\Model (and now you can add a Service, a Factory, etc)

\Joomla\Response\View (or a response Template, or even a Theme)

Huh. I like it.

Regards,

Andrew Eddie

[Nils Rückmann]

Well, i like the ADR concept, but to define what's best for a Joomla Framework, there should be a definition of its goals.

The Joomla! Framework™ is a new PHP framework (a collection of software libraries/packages) for writing web and command line applications

(http://framework.joomla.org/) 

First i would disagree that a framework is just a bunch of packages. But if Joomla does see it this way, there is no reason why MVC, ADR or XYZ Packages can't be part of the framework at the same time.

IMO a framework is on top of that. It's not just a collection of packages, it's an engine for developers to build (more or less) awesome applications.

But back to topic: It would be great if we could offer multiple architectural patterns for the same application. So an application can change it's pattern in relation to it's context.

Something like:

\Joomla\Architecture\MVC

\Joomla\Architecture\ARD

 

[Michael Babker]

If Joomla's approach is to the MVC triad, then I think our framework code should focus on that first.  With that said, if it's feasible to have an ADR layer without making things complex, I think that would interesting to incorporate.

Jumping subjects for a minute (not really roadmap, but the change does have v2 B/C breaking implications)...

In our social packages, we only return a formatted response body from our methods, which makes the rest of the response data inaccessible.  In the case of GitHub, this makes it difficult to access the response headers to check the different data they send back with requests (such as pagination links for list APIs or rate limits).  So we need to make an adjustment to those APIs IMO to give access to the full response data.  Either by storing the last response as an accessible API point or adjusting all of our endpoints to return the full response (including the formatted body).

[Andrew Eddie]

On 23 February 2015 at 08:01, Michael Babker <michael...@gmail.com> wrote:
> In our social packages, we only return a formatted response body from our
> methods, which makes the rest of the response data inaccessible. In the
> case of GitHub, this makes it difficult to access the response headers to
> check the different data they send back with requests (such as pagination
> links for list APIs or rate limits). So we need to make an adjustment to
> those APIs IMO to give access to the full response data. Either by storing
> the last response as an accessible API point or adjusting all of our
> endpoints to return the full response (including the formatted body).

I'd probably return either a response model (something with functions)
or a response entity (something with just properties). The model is
probably better as you can define an interface.

You definitely need the ability to inspect headers.

Regards,
Andrew Eddie

[Nils Rückmann]

+1 For response model.  First step might be to expand the http package, because that's where abstract/base models should be part of.

[Michael Babker]

George has done some stuff on that, see https://github.com/joomla-framework/http/pull/9

--

[Nils Rückmann]

I will take a look into. 

PS: The PR is a good example how docs belongs to the package ;) Well done.

[piotr_cz]

I wouldn't mind if the packages would be managed in Laravel style:

- whole framework in one repository,

- but packages available to install separately

[Nils Rückmann]

Now i'm confused. Were there any plans to not offering install-packages separately ? If there were plans, and please excuse my wording, it would be ridiculous and against everything (composer, PHP-FIG, etc.) the PHP community worked for in the last year(s). 

Back to the roadmap: Will "Joomla Framework" be a part of GSoC 2015 ? Some goals of the roadmap could be "outsourced" to students.

[Michael Babker]

Never any plans to not offer the packages separately.

If we can come up with proposals for GSoC, they can surely be added to the ideas list - https://docs.joomla.org/GSOC_2015_Project_Ideas

On Tue, Feb 24, 2015 at 9:14 AM, Nils Rückmann <ma...@nueckman.de> wrote:

Now i'm confused. Were there any plans to not offering install-packages separately ? If there were plans, and please excuse my wording, it would be ridiculous and against everything (composer, PHP-FIG, etc.) the PHP community worked for in the last year(s). 

Back to the roadmap: Will "Joomla Framework" be a part of GSoC 2015 ? Some goals of the roadmap could be "outsourced" to students.

--

[Nils Rückmann]

Well, i think you are more up to date with the current progress. But the form package could be something for GSoC, or the social packages if the http package is ready.

[Michael Babker]

Seeing your post on the GSoC list reminded me I forgot to come back here.  Sorry about that.

I think we'd need to be clear on what goals we want to accomplish for the Form package before putting an idea out.  Incidentally we should know by this time tomorrow if we've been accepted for participation in GSoC so looking forward to that announcement.

One idea that's already gotten onto the ideas list is expanding the package to work with formats beyond XML.

On Tue, Feb 24, 2015 at 9:43 AM, Nils Rückmann <ma...@nueckman.de> wrote:

Well, i think you are more up to date with the current progress. But the form package could be something for GSoC, or the social packages if the http package is ready.

--

[Nils Rückmann]

Hi Michael, i proposed the form-idea a few days ago, but still no feedback. Basically it was about interfaces, the ability to modify forms/fields and a new layer to allow different configurations (XML, JSON, YAML, Annotation, whatever..) and different renderers (no hardcode html, different markuo frameworks, etc.). But you're right, we will see what tomorrow brings.

BTW: I've opened a new topic (https://groups.google.com/forum/#!topic/joomla-dev-framework/nX7KO2SKKyA) a few days ago which belongs to the configuration issue.

[Walt Sorensen]

Here is my input.

### Minimum Supported PHP Version

Definitely drop 5.3 as it is a year past it's end of life

5.4 is questionable since it's EOL is this fall, but would be the lowest acceptable minimum

5.5 would be a preferred minimum even with the EOL in Summer 2016

### Database
As per the Framework roadmap "For version 2, we are looking at ... potentially consuming a third party database system."
Suggestion to incorporate Doctrine DBAL in V2 (suggested on github)

Doctrine DBAL seems like it might be a good candidate for consideration as it is an "Abstraction layer and access library for relational databases. A thin layer on top of PDO with a lot of additional, horizontal functionality."

This would fill some of the holes  

 

[Andrew Eddie]

Just wanting to touch base with has anything progressed in terms of executing the roadmap and what one thing can everyone focus on till the end of this month and just "finish"?

Regards,

Andrew Eddie

[Michael Babker]

Umm, yea, about that...

The Language package has gotten a fair bit of attention, including some documentation work (in the model I'd suggest is adopted Framework wide).  The View package has had a bit of a refactoring and brought about a Renderer package as a result.  Beyond that, not much tangible work has started AFAIK.  I'm toying around a bit with the Session package and maybe after JAB I can find time to push that into a PR on the repo.

With the PSR-7 vote passing, we need to make a decision whether it's something to apply to our code base, and if so, what effort that will require.

For the HTTP package stuff referenced here earlier (https://github.com/joomla-framework/http/pull/9), looking back now I think George and I started cramming too much into one PR and it might be a good idea to break that up into smaller incremental pieces to make sense of what's trying to be accomplished in there.  That may also be dependent on a PSR-7 decision.

Lastly, regardless of other activity going on, there's PHP 7 compatibility to keep in mind.  The String package already had a class renamed to deal with that and we've already made some syntax changes to support it as needed, but we still have some issues.  Our unit tests that essentially test the typehinting on a method now cause fatal errors on PHP 7 and can probably go away.  There's a failure in the Form package (https://travis-ci.org/joomla-framework/form/jobs/61620446) and I thought there was one with the Utilities package but I can't find the failing builds anymore (or it was something with the CMS' version of JArrayHelper, who knows).

--