KISS: A Sustainable CMS Strategy

[Nick Savov]

Hi all,

In my opinion, the Joomla CMS’s upgradability, stability, and maintainability can and should be improved greatly. I think this is essential to our project’s future. I think that those three items can be improved substantially without sacrificing innovation. In fact, adjusting the way we currently deal with these three processes will set the foundation for more sustainable innovation and competitive involvement on the web.

The current development strategy results in fragmented Joomla CMS major versions that are extremely difficult, time-consuming, and costly to migrate from one version to another. Our users are not able to keep up with the pace of change, which is evident by the statistic that about 55% of our users are still on Joomla 1 (http://w3techs.com/technologies/details/cm-joomla/all/all).

Fragmented Joomla major versions also make it difficult for core developers/designers and 3rd party developers/designers to maintain their software, since they need to support multiple packages and do testing on multiple versions. It results in at least double the work for maintainers and testers.

It also results in a bloated CMS that’s difficult (or near impossible) to maintain long-term.

Finally, the last major issue is that it results in confusion as to which Joomla CMS version to use and when. Should a user migrate from Joomla 1.5 to Joomla 2.5 or Joomla 3? If so, when? How long will each be supported? What’s the best strategy to take? What’s LTS? What’s STS? For how long? We’ve resorted to answering these type of questions with complicated paragraphs, diagrams, charts, infographics, videos, etc. Even with all the effort we’ve put into explaining Joomla versions, our users as a whole still don’t understand because it’s too complicated.

That being said, I’d like to propose and get community feedback on my suggested changes to our development strategy for the CMS, which would resolve the above mentioned issues. These are merely my personal ideas as a community member and they are not on behalf of PLT. I would like to gather more community feedback before presenting my ideas for consideration to PLT.

The basic premise is to keep things simple, easy, and sustainable. All terms with an * are defined at the bottom of the doc. So with that in mind, here are my suggestions:

There should only be one active Joomla CMS version, “the latest and greatest”, which everyone should use and which should always be stable and user-friendly. (Note, this is the same approach as Google Chrome and Wordpress)

In order to achieve this, the following policy is required:

• Backward compatibility* must always be maintained, even across major versions

• The Developer API* (i.e. public API) should never break

• Developer API should be clearly stated and documented.

• Developer API should be fully unit tested to maintain integrity

• Deprecations* are maintained at least 2 years before being considered for removal .

• Deprecations should be clearly documented

• Removal of deprecated code is not mandated after 2 years, but merely allowed if necessary

• Deprecations can only be removed during major releases.

• Removal of deprecations are not considered breaks in backward compatibility

• The Designer Media* must never “break”

• Designer Media should be clearly stated and documented.

• Deprecations* are maintained at least 2 years before being considered for removal .

• Deprecations should be clearly documented

• Removal of deprecated media is not mandated after 2 years, but merely allowed if necessary

• Deprecations can only be removed during major releases.

• Removal of deprecations are not considered breaks in backward compatibility

• Public Database Schema* must be maintained

• Database schema should be clearly stated and documented.

• Deprecations* are maintained at least 2 years before being considered for removal

• Deprecations should be clearly documented

• Removal of deprecations is not mandated after 2 years, but merely allowed if necessary

• Deprecations can only be removed during major releases.

• Removal of deprecations are not considered breaks in backward compatibility

• Refactoring* should only be done when needed

• Backward compatibility is non-negotiable. If that means we acquire technical debt for our users (end-users, site builders, designers, developers, etc), so be it.

• Release a new minor (e.g. 3.4) or major (e.g. 4.0) version about every 6 months

• The degree of change should determine if it’s a minor or major release. For example, an overhaul of the admin area could trigger a major release.

• Alpha should be released around 3 months before the release

• Beta should be released around 2 months before the release and no more new features can be added after beta is released. Periodic betas should be released.

• An RC should be released at least two weeks before the release and only when ready

• A stable release should only be released when ready

• If any submitted features are not ready, they must wait until the next release (around 6 months later)

• Maintenance releases (e.g. 3.1.1) should only fix security issues and critical bugs

• Joomla core should be kept light and flexible

• Non-essential extensions (for example, com_banners, com_contact, com_newsfeeds, etc) should be made as installable extensions via the Install from Web feature and maintained independently of the Joomla core.

• This includes extensions that are used as “best practices” guides for new developers. A category on the JED can be created for “best practice” extensions and we can highlight them in the Install from Web feature if needed.

• This type of thinking can be used for distributions. Just have a light core and install "sets" of extensions from the Install from Web feature.

--------------------------------

Definitions:

1) Backward compatibility:

Backward compatibility refers to the ability of a Joomla version N to accept extensions or data that worked under version N - 1. By this definition, if previous versions (N - 1, N - 2, etc.) were also backward compatible, which is often the case, then, by induction, version N will also accept input that worked under any prior version after, and including, the latest one that was not backward compatible. However, in practice, features are often deprecated and support is dropped in a later release, which is yet thought of as backward compatible.

Note, this is taken from http://en.wikipedia.org/wiki/Backward_compatibility, but modified slightly with “Joomla” and “extensions” as references instead.

2) Deprecation:

Deprecation is status change of a supported practice to a non-supported practice, which keeps in place the previously supported practice for at least one version to provide backward compatibility. That being said, with the deprecation of at least 2 years, you're actually getting at least N-4 versions.

3) Developer API:

The API that Joomla is agreeing to support for developers to use. This is our contract between Joomla and developers for API that we’re agreeing will be available to them and stable.

4) Designer Media:

The media that Joomla is agreeing to support for designers to use. This is our contract between Joomla and designers that we’re agreeing will be able to them and useable.

5) Public Database Schema:

The database schema that Joomla is agreeing to support for developers and others to use. This is our contract between Joomla and those that desire to use Joomla’s database schema.

6) Refactoring:

According to http://refactoring.com/, refactoring is a "disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior". Another way to say that is that refactoring is a technique that changes code, but produces no breaks for its users.

____________

FAQs:

1) What is the main advantage of the proposed changes?

The main advantage is that it’s simple. There’s only one Joomla version to market, to explain, and to maintain. This produces a ton of benefits for maintainers and users alike.

The Joomla core team would only have one Joomla version to maintain (rather than 2 or 3, as is possible now). That version would be light, so there would be less code to maintain.

Extension developers (include template designers) would only have one version to support, thus they wouldn’t need to maintain several different code bases (as was the case from J1.5 to J2 for everyone and J2 to J3 for template designers).

2) Won’t we get stuck with a lot of outdated, junky code?

No, that’s where deprecation comes in. It allows us to responsibly remove outdated/junky code, while stilling giving people enough time (at least 2 years) to adjust.

3) What is the relationship between backward compatibility and deprecation?

Backward compatibility is in reference to supported practices of the previous version (i.e. N-1). On the other hand, deprecation is status change of a supported practice to a non-supported practice, which keeps in place the previously supported practice for at least one version to provide backward compatibility. That being said, with the deprecation of at least 2 years, you are actually getting at least N-4 versions.

4) Isn’t removing deprecated code a break in backward compatibility?

That depends on what definitions you’re using. If you’re using the industry standard of N-1 for backward compatibility of supported practices, then by definition removing deprecated code is not a break in compatibility.

This is why it’s essential for our project to define our terms properly, no matter what direction we take.

5) Can you give an example of a break in backward compatibility?

Sure. An example of a break in backward compatibility would be an API that used to do a certain function in version V that suddenly stops doing the same function or produces a fatal error in version V+1.

However, if an API gets deprecated in V and gets removed in version V+5, when V+6 comes along, that is not considered a break in compatibility since the API was deprecated in V, worked in V+1, and was removed in V+5 and is not supported in V+6.

5) Why not just use semantic versioning instead of the proposed changes?

Semantic versioning doesn’t tell you whether or not you should have backward compatibility changes. It merely tells you what you should do *if* you have breaks in backward compatibility. My suggestion transcends semantic versioning and states that we should never have breaks in backward compatibility.

Furthermore, semantic versioning is for APIs whereas Joomla contains APIs and more. There’s UI to take into account, etc, that needs to be properly communicated. For example, an overhaul of the admin area could trigger a major release (e.g. J4).

6) What happens to STS and LTS?

Those terms are confusing and unneeded. With the proposed changes, there’s only the “latest Joomla”.

7) What would a move from a Joomla 3 to Joomla 4 for users look like under the proposed changes?

It would be a one-click upgrade.

Currently around 55% of our users are on Joomla versions that are EOL, unsupported, and insecure:

http://w3techs.com/technologies/details/cm-joomla/all/all

The propose changes would greatly mitigate this types of issues from occurring in the future.

8) Aren’t time-based releases causing rushed, sloppy code and review?

No, time based releases do not require submitted code to get in. If it's not ready (even for a committer to review it), it must wait 6 months until the next release.

That being said, having alpha 3 months before the release and having beta 2 months before, gives plenty of breathing room, especially with a lighter core and unit tested developer API.

----

Your feedback is welcomed and appreciated. Thank you for taking the time to read through this and give it consideration.

Sincerely,

Nick Savov

[brian teeman]

Before commenting in depth could you clarify a little bit more what you mean by "Designer Media" I'm a bit lost.

[Nick Savov]

Sure thing. It's basically the same concept as our currently JUI:
https://github.com/joomla/joomla-cms/tree/master/media/jui

CSS, Javascript, images, etc, that Joomla is agreeing to support for
designers to use.

Kind regards,
Nick

>
> Before commenting in depth could you clarify a little bit more what you
> mean by "Designer Media" I'm a bit lost.
>

> --
> You received this message because you are subscribed to the Google Groups
> "Joomla! CMS Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to joomla-dev-cm...@googlegroups.com.
> To post to this group, send an email to joomla-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/joomla-dev-cms.
> For more options, visit https://groups.google.com/groups/opt_out.
>

[brian teeman]

On Thursday, 12 December 2013 21:39:15 UTC, Nick Savov wrote:

• 
  Maintenance releases (e.g. 3.1.1) should only fix security issues and critical bugs

Do you really mean that the users have to wait for 6 months for a simple bug fix. Thats what it reads like but I'm sure you dont really mean that as you quoted exampled of google chrome and wordpress dont do that 

[brian teeman]

On Thursday, 12 December 2013 22:04:01 UTC, Nick Savov wrote:

Sure thing. It's basically the same concept as our currently JUI:
https://github.com/joomla/joomla-cms/tree/master/media/jui

CSS, Javascript, images, etc, that Joomla is agreeing to support for
designers to use.

How can that work? These are in most cases external libraries eg jquery that we dont have control over and would mean that we would go back to the days of Joomla extensions installing multiple versions of those libraries because the one in the core is upto 2 years out of date. 

 

[Nick Savov]

Initially, until our upgrades get better, yes. I'm fine with moving up
every 2 months, 3 months, or 4 months though

Google Chrome and Wordpress are much better at not producing breaks in
backward compatibility with their upgrades. They've also both now started
doing automatic upgrades (though I'd say Chrome is years ahead on this).

By the way, Wordpress does indeed only fix security issues and critical
bugs in their maintenance releases.

Kind regards,
Nick

>
>
> On Thursday, 12 December 2013 21:39:15 UTC, Nick Savov wrote:
>>
>>
>>
>>

>> * - Maintenance releases (e.g. 3.1.1) should only fix security issues
>> and
>> critical bugs*

>>
>
> Do you really mean that the users have to wait for 6 months for a simple
> bug fix. Thats what it reads like but I'm sure you dont really mean that
> as
> you quoted exampled of google chrome and wordpress dont do that
>

[Donald Gilbert]

Backward compatibility* must always be maintained, even across major versions

Your provided definition of BC doesn't meet this requirement. If you are always maintaining BC, you can never remove deprecated functions. You acknowledge this by specifically stating "Removing deprecated functions is not considered a BC break" which is, according to your definition, entirely false.

My thoughts on BC is that it both CAN and WILL happen, as and when needed. If we follow a standard versioning approach (Semantic Versioning) then it is very clear to the end user and to our developers when they should expect possible breaking changes.

Since you specifically state that BC must always be maintained, then if a plugin works on 3.2, then it must by your definition, also work on 8.99. Otherwise, BC is not always maintained. You can say that you clarify it in your definition, but why not just call a spade a spade instead of trying to make it into marketing speak that our users want to here, saying "We never break BC*" <-- where the asterisk says we do break bc, when removing deprecated code.

The degree of change should determine if it’s a minor or major release. For example, an overhaul of the admin area could trigger a major release.

I disagree here. The determining factor for whether or not a release is considered Minor or Major is spelled out by the industry standard Semantic Versioning. It's a minor release when there are new features added that do not break backwards compatibility, regardless of the "degree of change". Likewise, a new release is a Major release when there are BC breaks in the code. A release is considered a bug fix or patch release if it fixes bugs or addresses security concerns in a backwards compatible way, typically by deprecating old and using new.

If Joomla were to adopt this industry standard approach, we would have much more stable software, as well as much happier extension developers, because they would know exactly when to expect breaking changes in the code. This approach also makes it much easier to document such changes. Also under this approach, we can have a CMS release really whenever the code is ready to go and has undergone sufficient testing.

Joomla core should be kept light and flexible

I agree with your comments in the section.

To answer your FAQ section -

4) Isn’t removing deprecated code a break in backward compatibility?

Yes, it actually is. Backwards compatibility does not only apply to the previous version, since you said "must always" be maintained.

5) Why not just use semantic versioning instead of the proposed changes?

Yes, why don't we? Your understanding of Semantic Versioning is incorrect, and causes you to come up with your own definitions to try to explain it.

8) Aren’t time-based releases causing rushed, sloppy code and review?

Yes, the actually do, as can be evidenced by Joomla! CMS 3.2

My Thoughts for a Joomla Dev Strategy

I feel that taking a strict Semantic Versioning approach to our development strategy would be a great approach, instead of coming up with yet another list of Joomla-specific definitions and re-definitions of common industry practices. *drops mic*

[brian teeman]

Sorry are we trying to copy wordpress. I thought we were joomla.

Seriously though I'm not fine and I doubt real users are fine with waiting your stated 6 months for a non critical bug fix

[Nick Savov]

We control when they are upgraded and added in our CMS though. That's
where we'd need proper planning, dependency management, and deprecation.

Kind regards,
Nick

[Michael Babker]

And your back in the days of old where developers were removing the core media in their templates and loading whatever they wanted to do.  Yes, we can control what media is upgraded when in the CMS, but shipping outdated media for the sake of B/C is crap.

[Nick Savov]

I was just pointing to Google Chrome and Wordpress as real case examples.

If people aren't fine with that, we can move it up to 2-3 months :)

Kind regards,
Nick

>> > email to joomla-dev-cm...@googlegroups.com <javascript:>.

>> > To post to this group, send an email to

>> joomla-...@googlegroups.com<javascript:>.

[Donald Gilbert]

On Thursday, December 12, 2013 4:14:13 PM UTC-6, brian teeman wrote:

Sorry are we trying to copy wordpress. I thought we were joomla.

This was the response of many of the people Nick presented his solutions to. 

[Michael Babker]

On Thu, Dec 12, 2013 at 4:12 PM, Donald Gilbert <dilber...@gmail.com> wrote:

8) Aren’t time-based releases causing rushed, sloppy code and review?

Yes, the actually do, as can be evidenced by Joomla! CMS 3.2

My only public comment here is that the number of people actually reviewing and merging code is far too little while the number of people testing and commenting on the details of a feature or function, or even if it should be merged into core (we can't get an update to our .htaccess in for what reason!?) is far too great.  When there are single individuals devoting full-time work hours to preparing a release and volunteers telling those individuals they need to back off their commitments and let others pick up the slack (who won't do it by the way), there are other issues that need to be addressed.

 

/rant

[Donald Gilbert]

For the curious, the PLT has been working on a revised development strategy, of which the Work In Progress can be seen here: https://gist.github.com/dongilbert/fb5d5107f28c5392dd59

[Steve]

I'm not heavily involved in development, so I'll keep my comments short and to the point.

1. Nick and Donald ... I think you guys agree on more than you realize. You're talking past each a little, but slow down and you'll see that you share a lot of ideas.
2. The current release cycle is working. Users appreciate the predictability that Joomla has provided lately. 
3. Please let's not go back to releasing major versions increment when we need them to". Witness the old struggles with Joomla 1.5 and 1.6 and the more recent struggles of Drupal 8. Witness the Drupal community realizing that their current release cycle is broken and they need to move to incremental releases. Deadlines have been good for Joomla and we've achieved much more with them than without them. 

[Amy Stephen]

On Thursday, December 12, 2013 3:39:15 PM UTC-6, Nick Savov wrote:

It also results in a bloated CMS that’s difficult (or near impossible) to maintain long-term.

And yet, you are calling for keeping it all. Encourage you to think about that point because I believe there are some tough choices facing Joomla that involve deep cuts in this area, otherwise, I don't think things will ever be manageable. It'll just be more and more and more failures.
 

Finally, the last major issue is that it results in confusion as to which Joomla CMS version to use and when. Should a user migrate from Joomla 1.5 to Joomla 2.5 or Joomla 3? <snip> Joomla versions, our users as a whole still don’t understand because it’s too complicated.

It really is a problem. Total Agreement.

In order to achieve this, the following policy is required:

• Backward compatibility* must always be maintained, even across major versions

• The Developer API* (i.e. public API) should never break

• Developer API should be clearly stated and documented.

• Developer API should be fully unit tested to maintain integrity

• Deprecations* are maintained at least 2 years before being considered for removal .

In all of these proposals, this is the only one that is calling for defining the API, and IMO, clarifying the API is the only possible way out.

But, I would not go further than defining the API at this time. I would get an agreement on that -- and the very act of doing so will mean BC breaks. There are far, far, far too many ways that developers can use the environment and those should be significantly slashed and Interfaces should be defined for the CMS to act as clear contracts as to what developers should (and should not be using.) 

As an example of how this would help -- take email as a service and offer a single method "send()" with no parameters. Inject all input using the framework's DIC and the API is one method. Now, what the framework folks do inside is irrelevant to the CMS. If they offer options beyond phpMailer -- including Swiftmail -- and Zendmail -- it won't change the API.

I'm very concerned at all of these proposals I am seeing which continue to avoid getting down to a cleaned up, scaled back, well documented, fully tested API. When the API is agreed too, those other matters, like how long support lasts and whether or not Semantic Versioning will be used, etc., etc., will be easier to tackle.

But first, what is the API? Does everyone agree on that? That alone will be a big job but it's the way forward, avoiding it means more of the same in terms of problems, quality issues, BC breaks.

It's definitely worth looking at why WordPress is able to upgrade without problems. For a very long time, WP has had a clearly defined API. And, that isn't all public functions -- it's a clear set of developer functions, nothing less than what they need and nothing more.

On the issue of BC support, yes, WP is the one to emulate. But, before Joomla is in a position to do what WP is successfully doing, there will need to be cleanup changes, there is just no way around that.

[Donald Gilbert]

We're not trying to be like Drupal. We've tried strictly featured based releases. We've tried strictly timed based releases. Neither of them have worked for our community in the long run (if they had, we wouldn't be having this conversation) the balance is somewhere in the middle. Let's find that.

[brian teeman]

On Thursday, 12 December 2013 22:38:51 UTC, Donald Gilbert wrote:

We're not trying to be like Drupal. We've tried strictly featured based releases. We've tried strictly timed based releases. Neither of them have worked for our community in the long run (if they had, we wouldn't be having this conversation) the balance is somewhere in the middle. Let's find that.

Things evolve and change over time and we must respond and react to that - or even better pre-empt it. Its one of the exciting things about working on the internet - who knows what will happen tomorrow.

[Donald Gilbert]

Fully agree Amy, we really must define our public API in order to move forward.

As for the users being confused as to what release to use, under Semantic Versioning, the answer is "always use the highest version available" if you're starting a new site. If you have an existing site, it's more complicated, but the basic answer is you should always use the latest version within your major series. 

This is accomplished via Semantic Versioning by guaranteeing Backward Compatibility within a major series. 3.99 is compatible all the way back to 3.2. Said another way, an extension written for 3.2 will work all the way up to 3.999999 (if we make it that far), guaranteed. 

[Amy Stephen]

So, where is the disagreement?

Nick - if you want to avoid BC breaks, then embrace Semantic Versioning, it grew out of frustrations with API breaks -- it is nearly exclusively focused on a disciplined approach to deciding -- is it worth breaking BC?

Either way, sounds like there is common ground on defining the API. Wouldn't that be step 1 regardless? Is there reason to go further without an agreement on the API, in hand?

If there is interest in collaborating on the CommonApi Interfaces I have extracted from my work in Molajo, I would welcome the involvement. That might help jump start the effort and I am open to changing any Interfaces. Or, if the project wants to fork my work and go their own way with it, feel free. 

[Andrew Eddie]

On Friday, 13 December 2013 07:39:15 UTC+10, Nick Savov wrote:

The current development strategy results in fragmented Joomla CMS major versions that are extremely difficult, time-consuming, and costly to migrate from one version to another.

While it's true for 1.5 to 2 or 3, it's not strictly true for 1.6+ to 3.x. The reality is though, there IS a cost associated with upgrading or migrating between major versions. I'm not sure what you mean by "fragmented" though.

 

Our users are not able to keep up with the pace of change, which is evident by the statistic that about 55% of our users are still on Joomla 1 (http://w3techs.com/technologies/details/cm-joomla/all/all).

You can't draw that conclusion from that data. The data is silent on whether people need to change at all. The only thing we can conclude is that we need more data to work out "why" the versions are distributed the way they are.

 

In order to achieve this, the following policy is required:

• Backward compatibility* must always be maintained, even across major versions

While it should a goal to reduce the number of changes, it's ridiculous to suggest we should never remove code. It might not be what you mean, but that's the end result of this requirement.

 

• The Developer API* (i.e. public API) should never break

That's an unusual statement because the API follows OOP rules. Public is public, protected is protected and private is private. The statement is redundant because your first point (always helpful to number them rather than using dot points). Maybe you mean particular types of files?

 

• Developer API should be clearly stated and documented.

• Developer API should be fully unit tested to maintain integrity

You'd need to define "fully". 

 

• Deprecations* are maintained at least 2 years before being considered for removal .

Contradicts with your opening bullet point. Removing deprecated code breaks backward compatibility. 

 

• Deprecations should be clearly documented

• Removal of deprecated code is not mandated after 2 years, but merely allowed if necessary

Unnecessary statement.

 

• Deprecations can only be removed during major releases.

Agree but you need to fix up your idea of what deprecation and backward compatibility is. You have conflicting requirements.

 

• Removal of deprecations are not considered breaks in backward compatibility

They are in my book. You should just keep it simple and say deprecated code is maintained for the life of a minor series, and is removed in the next major version. That's easy to follow, easy to understand.

 

• The Designer Media* must never “break”

• Designer Media should be clearly stated and documented.

• Deprecations* are maintained at least 2 years before being considered for removal .

• Deprecations should be clearly documented

• Removal of deprecated media is not mandated after 2 years, but merely allowed if necessary

• Deprecations can only be removed during major releases.

• Removal of deprecations are not considered breaks in backward compatibility

This whole section is clumsy. You need to define what you actually mean. I'd talk to someone experienced with design about it.

 

• Public Database Schema* must be maintained

Must be maintained for? To do?

 

• Database schema should be clearly stated and documented.

• Deprecations* are maintained at least 2 years before being considered for removal

• Deprecations should be clearly documented

• Removal of deprecations is not mandated after 2 years, but merely allowed if necessary

• Deprecations can only be removed during major releases.

• Removal of deprecations are not considered breaks in backward compatibility

Same problems as previous rules. You'd be better to set basic principles for b/c and deprecation and then list the things that apply, for example:

a. PHP code that is not private.

b. JavaScript functions and objects that are not private. 

c. Database schema. 

d. XML schema. 

e. Language keys

 

• Refactoring* should only be done when needed

That's sort of like saying "breath when needed". Refactoring is a natural part of any development process.

 

• Backward compatibility is non-negotiable. If that means we acquire technical debt for our users (end-users, site builders, designers, developers, etc), so be it.

Yes but you you need to reconcile it with what you are saying about deprecation. You view on deprecation not being a compatibility issue is just plain wrong in my opinion.

 

• Release a new minor (e.g. 3.4) or major (e.g. 4.0) version about every 6 months

You need to clean up this statement. It could mean that we release a new major every 6 months. But then again, when you aren't removing code, you don't have an issue under your strategy.

 

• The degree of change should determine if it’s a minor or major release. For example, an overhaul of the admin area could trigger a major release.

I don't agree. You should only change the major version when you introduce a backward incompatible change. If you overhaul the admin area I would actually do that in a new template allowing people to migrate from the old admin to the new admin at their leisure. This can be done within a minor series easily.

 

• Alpha should be released around 3 months before the release

I wouldn't set a time limit on it. Alpha's should be released when you want to show the development community your ideas and get feedback. They should not be locked into the calendar.

 

• Beta should be released around 2 months before the release and no more new features can be added after beta is released. Periodic betas should be released.

Again, it depends. 1.6 was in beta for 6 months, and rightly so. Beta isn't a time period, it's a "state".

 

• An RC should be released at least two weeks before the release and only when ready

You should define what "ready" means. I define it as no P1 or P2 issues pending.

 

• A stable release should only be released when ready

• If any submitted features are not ready, they must wait until the next release (around 6 months later)

Hang on. That's part of your beta phase of development. Beta is "feature complete" but buggy. You would be better to say that once the software is in beta, you have to wait for the next minor release to add new features.

 

• Maintenance releases (e.g. 3.1.1) should only fix security issues and critical bugs

When you you fix the non-critical bugs? In minor release every 6 months?

 

• Joomla core should be kept light and flexible

• Non-essential extensions (for example, com_banners, com_contact, com_newsfeeds, etc) should be made as installable extensions via the Install from Web feature and maintained independently of the Joomla core.

Agree.

 

• This includes extensions that are used as “best practices” guides for new developers. A category on the JED can be created for “best practice” extensions and we can highlight them in the Install from Web feature if needed.

Don't like that idea.

 

• This type of thinking can be used for distributions. Just have a light core and install "sets" of extensions from the Install from Web feature.

Agree.

 

1) Backward compatibility:

Backward compatibility refers to the ability of a Joomla version N to accept extensions or data that worked under version N - 1. By this definition, if previous versions (N - 1, N - 2, etc.) were also backward compatible, which is often the case, then, by induction, version N will also accept input that worked under any prior version after, and including, the latest one that was not backward compatible. However, in practice, features are often deprecated and support is dropped in a later release, which is yet thought of as backward compatible.

Note, this is taken from http://en.wikipedia.org/wiki/Backward_compatibility, but modified slightly with “Joomla” and “extensions” as references instead.

I prefer the way Semver defines it. Your definition is better suited to the days when we compiled code. Apart from that, I've had to read that paragraph a number of time to even start to "get it". Why is the following so hard:

* New patch version - fully backward compatible and contains bug fixes.

* New minor version - fully backward compatible, may include newly deprecated code, new features and bug fixes.

* New major version - not backward compatible with previous major versions, previously deprecated code removed.

Keep it simple!

 

2) Deprecation:

Deprecation is status change of a supported practice to a non-supported practice, which keeps in place the previously supported practice for at least one version to provide backward compatibility. That being said, with the deprecation of at least 2 years, you're actually getting at least N-4 versions.

This is overcomplicated because you are using WordPress's way of versioning. Compared to Semver, Wordpress is really just a series of minor versions. It's like extending our 3.x to infinity. I would find this utterly confusing.

 

3) Developer API:

The API that Joomla is agreeing to support for developers to use. This is our contract between Joomla and developers for API that we’re agreeing will be available to them and stable.

The code you distribute IS the API. You are getting hung up on a term that does not exist in practice. The API is every public method and property in executable code, be it PHP or JavaScript. Protected methods in PHP would also be covered.

 

4) Designer Media:

The media that Joomla is agreeing to support for designers to use. This is our contract between Joomla and designers that we’re agreeing will be able to them and useable.

This is a really fully area because it's so broad. I agree, however, that you need to define critical parts of markup that other parts of the system rely on.

 

The main advantage is that it’s simple. There’s only one Joomla version to market, to explain, and to maintain. This produces a ton of benefits for maintainers and users alike.

I think you have some reasonable ideas, but as I have told you before, you need to think about how to set your ideas out better and work out your inconsistencies. Nick, we spent over 2 hours going through all these issues but you have not made any changes.

 

The Joomla core team would only have one Joomla version to maintain (rather than 2 or 3, as is possible now). That version would be light, so there would be less code to maintain.

I'm not a fan of the WordPress model.

 

Extension developers (include template designers) would only have one version to support, thus they wouldn’t need to maintain several different code bases (as was the case from J1.5 to J2 for everyone and J2 to J3 for template designers).

Then maybe they should jump ship and go to WordPress.

 

2) Won’t we get stuck with a lot of outdated, junky code?

No, that’s where deprecation comes in. It allows us to responsibly remove outdated/junky code, while stilling giving people enough time (at least 2 years) to adjust.

Um, Nick, we already deprecate code quite well. It's removed at each major version. I think this is a straw man.

 

3) What is the relationship between backward compatibility and deprecation?

Backward compatibility is in reference to supported practices of the previous version (i.e. N-1). On the other hand, deprecation is status change of a supported practice to a non-supported practice, which keeps in place the previously supported practice for at least one version to provide backward compatibility. That being said, with the deprecation of at least 2 years, you are actually getting at least N-4 versions.

Again, that's fine if you are WordPress. Semver works much better in my opinion.

 

4) Isn’t removing deprecated code a break in backward compatibility?

That depends on what definitions you’re using. If you’re using the industry standard of N-1 for backward compatibility of supported practices, then by definition removing deprecated code is not a break in compatibility.

I really feel you are adding an unnecessary layer of complexity here. Drop your 2 years requirement and I agree completely with your intention. You only need your 2 years because you are using the WordPress, not the Semver model.

 

5) Why not just use semantic versioning instead of the proposed changes?

Semantic versioning doesn’t tell you whether or not you should have backward compatibility changes.  

It merely tells you what you should do *if* you have breaks in backward compatibility. My suggestion transcends semantic versioning and states that we should never have breaks in backward compatibility.

Again, you have your idea of breaking and compatibility and versioning all messed up. The major increment IS advertising the software should have backward compatibility changes.

 

Furthermore, semantic versioning is for APIs whereas Joomla contains APIs and more. There’s UI to take into account, etc, that needs to be properly communicated. For example, an overhaul of the admin area could trigger a major release (e.g. J4).

Nick, you really need to broaden your experience because that statement is just pain wrong on so many levels. The reality is the major build tools like Composer, Ant, Bower, Grunt and others are ALL using Semver. Semver IS the industry standard, not WordPress.

Bottom line is this is a list of idea, not a cohesive strategy document. I think you need to clean it up a lot before we can compare competing strategies. The irony is though, Semver gives you exactly what you want (and so does my draft strategy :P).

Regards,

Andrew Eddie

[kisswebdesign]

On a related matter, and something that has been mentioned a few times, is the lifecycle of each major series.

I have written a gist https://gist.github.com/KISS-Web-Design/440a89cc54913a8ba4d5#file-j-version-strategy-md that explains my idea for a 5 year lifecycle, comments are welcome on the gist (or here is you must, but lets not completely hijack the thread).

The versioning and lifecycle do form part of the development strategy, but I have tried to isolate them from the other issues (distributions, joomlacode/github, chicken/egg, core/plugin, etc) - although there is a little crossover.

Hopefully the idea makes sense whatever the resolution to any of the other issues happens to be.

This is a first pass at the idea, it is not comprehensive enough for a proposal or RFC, and I am sure there are some details that need to be added and/or clarified.

Chris.

[Andrew Eddie]

On 13 December 2013 12:14, kisswebdesign <> wrote:
> I have written a gist https://gist.github.com/KISS-Web-Design/440a89cc54913a8ba4d5#file-j-version-strategy-md that explains my idea for a 5 year lifecycle, comments are welcome on the gist (or here is you must, but lets not completely hijack the thread).

Yep, did see that, just haven't had a chance to fully digest and
comment ;) I think it's good to have a spread of idea. It's not hard
to see the common themes.

Regards,
Andrew Eddie

[Nick Savov]

For anyone following along by email only, looks like Google Groups editor
messed up the formatting. You can see the formatted posts at
https://groups.google.com/forum/#!topic/joomla-dev-cms/4GZxFXO8zD4

Kind regards,
Nick

>
>
>
> *Hi all,In my opinion, the Joomla CMS�s upgradability, stability, and

> maintainability can and should be improved greatly. I think this is
> essential to our project�s future. I think that those three items can be
> improved substantially without sacrificing innovation. In fact, adjusting
> the way we currently deal with these three processes will set the
> foundation for more sustainable innovation and competitive involvement on

> the web.The current development strategy results in fragmented Joomla CMS

> major versions that are extremely difficult, time-consuming, and costly to
> migrate from one version to another. Our users are not able to keep up
> with
> the pace of change, which is evident by the statistic that about 55% of
> our
> users are still on Joomla 1
> (http://w3techs.com/technologies/details/cm-joomla/all/all
> <http://w3techs.com/technologies/details/cm-joomla/all/all>).Fragmented
> Joomla major versions also make it difficult for core developers/designers
> and 3rd party developers/designers to maintain their software, since they
> need to support multiple packages and do testing on multiple versions. It

> results in at least double the work for maintainers and testers.It also

> results in a bloated CMS that�s difficult (or near impossible) to maintain

> long-term.Finally, the last major issue is that it results in confusion as

> to which Joomla CMS version to use and when. Should a user migrate from
> Joomla 1.5 to Joomla 2.5 or Joomla 3? If so, when? How long will each be
> supported? What�s the best strategy to take? What�s LTS? What�s STS? For
> how long? We�ve resorted to answering these type of questions with
> complicated paragraphs, diagrams, charts, infographics, videos, etc. Even
> with all the effort we�ve put into explaining Joomla versions, our users
> as

> a whole still don�t understand because it�s too complicated.That being

> said, I�d like to propose and get community feedback on my suggested
> changes to our development strategy for the CMS, which would resolve the
> above mentioned issues. These are merely my personal ideas as a community
> member and they are not on behalf of PLT. I would like to gather more
> community feedback before presenting my ideas for consideration to PLT.The
> basic premise is to keep things simple, easy, and sustainable. All terms
> with an * are defined at the bottom of the doc. So with that in mind, here
> are my suggestions:There should only be one active Joomla CMS version,
> �the
> latest and greatest�, which everyone should use and which should always be
> stable and user-friendly. (Note, this is the same approach as Google
> Chrome

> and Wordpress)In order to achieve this, the following policy is required:
> -

> Backward compatibility* must always be maintained, even across major

> versions- The Developer API* (i.e. public API) should never break-
> Developer API should be clearly stated and documented.- Developer API
> should be fully unit tested to maintain integrity- Deprecations* are
> maintained at least 2 years before being considered for removal .-
> Deprecations should be clearly documented- Removal of deprecated code is
> not mandated after 2 years, but merely allowed if necessary- Deprecations
> can only be removed during major releases.- Removal of deprecations are
> not
> considered breaks in backward compatibility- The Designer Media* must
> never
> �break�- Designer Media should be clearly stated and documented.-

> Deprecations* are maintained at least 2 years before being considered for

> removal .- Deprecations should be clearly documented- Removal of
> deprecated
> media is not mandated after 2 years, but merely allowed if necessary-
> Deprecations can only be removed during major releases.- Removal of
> deprecations are not considered breaks in backward compatibility- Public
> Database Schema* must be maintained- Database schema should be clearly
> stated and documented.- Deprecations* are maintained at least 2 years
> before being considered for removal- Deprecations should be clearly
> documented- Removal of deprecations is not mandated after 2 years, but
> merely allowed if necessary- Deprecations can only be removed during major
> releases.- Removal of deprecations are not considered breaks in backward
> compatibility- Refactoring* should only be done when needed- Backward

> compatibility is non-negotiable. If that means we acquire technical debt
> for our users (end-users, site builders, designers, developers, etc), so
> be

> it. - Release a new minor (e.g. 3.4) or major (e.g. 4.0) version about
> every 6 months- The degree of change should determine if it�s a minor or

> major release. For example, an overhaul of the admin area could trigger a

> major release.- Alpha should be released around 3 months before the
> release- Beta should be released around 2 months before the release and no

> more new features can be added after beta is released. Periodic betas

> should be released.- An RC should be released at least two weeks before
> the
> release and only when ready- A stable release should only be released when
> ready- If any submitted features are not ready, they must wait until the
> next release (around 6 months later)- Maintenance releases (e.g. 3.1.1)
> should only fix security issues and critical bugs- Joomla core should be
> kept light and flexible- Non-essential extensions (for example,

> com_banners, com_contact, com_newsfeeds, etc) should be made as
> installable
> extensions via the Install from Web feature and maintained independently
> of

> the Joomla core.- This includes extensions that are used as �best

> practices� guides for new developers. A category on the JED can be created
> for �best practice� extensions and we can highlight them in the Install

> from Web feature if needed.- This type of thinking can be used for

> distributions. Just have a light core and install "sets" of extensions
> from
> the Install from Web

> feature.--------------------------------Definitions:1)

> Backward compatibility:Backward compatibility refers to the ability of a
> Joomla version N to accept extensions or data that worked under version N
> -
> 1. By this definition, if previous versions (N - 1, N - 2, etc.) were also
> backward compatible, which is often the case, then, by induction, version
> N
> will also accept input that worked under any prior version after, and
> including, the latest one that was not backward compatible. However, in
> practice, features are often deprecated and support is dropped in a later

> release, which is yet thought of as backward compatible.Note, this is
> taken
> from http://en.wikipedia.org/wiki/Backward_compatibility
> <http://en.wikipedia.org/wiki/Backward_compatibility>, but modified

> slightly with �Joomla� and �extensions� as references instead.2)
> Deprecation: Deprecation is status change of a supported practice to a
> non-supported practice, which keeps in place the previously supported
> practice for at least one version to provide backward compatibility. That
> being said, with the deprecation of at least 2 years, you're actually

> getting at least N-4 versions.3) Developer API:The API that Joomla is

> agreeing to support for developers to use. This is our contract between
> Joomla and developers for API that we�re agreeing will be available to
> them

> and stable.4) Designer Media:The media that Joomla is agreeing to support

> for designers to use. This is our contract between Joomla and designers

> that we�re agreeing will be able to them and useable.5) Public Database

> Schema:The database schema that Joomla is agreeing to support for
> developers and others to use. This is our contract between Joomla and
> those
> that desire to use Joomla�s database schema.6) Refactoring:According to

> http://refactoring.com/ <http://refactoring.com/>, refactoring is a

> "disciplined technique for restructuring an existing body of code,
> altering
> its internal structure without changing its external behavior". Another
> way
> to say that is that refactoring is a technique that changes code, but

> produces no breaks for its users.____________FAQs:1) What is the main
> advantage of the proposed changes?The main advantage is that it�s simple.

> There�s only one Joomla version to market, to explain, and to maintain.

> This produces a ton of benefits for maintainers and users alike.The Joomla

> core team would only have one Joomla version to maintain (rather than 2 or
> 3, as is possible now). That version would be light, so there would be
> less

> code to maintain.Extension developers (include template designers) would

> only have one version to support, thus they wouldn�t need to maintain
> several different code bases (as was the case from J1.5 to J2 for everyone

> and J2 to J3 for template designers).2) Won�t we get stuck with a lot of
> outdated, junky code?No, that�s where deprecation comes in. It allows us

> to
> responsibly remove outdated/junky code, while stilling giving people
> enough

> time (at least 2 years) to adjust.3) What is the relationship between
> backward compatibility and deprecation?Backward compatibility is in

> reference to supported practices of the previous version (i.e. N-1). On
> the
> other hand, deprecation is status change of a supported practice to a
> non-supported practice, which keeps in place the previously supported
> practice for at least one version to provide backward compatibility. That
> being said, with the deprecation of at least 2 years, you are actually

> getting at least N-4 versions.4) Isn�t removing deprecated code a break in
> backward compatibility?That depends on what definitions you�re using. If

> you�re using the industry standard of N-1 for backward compatibility of
> supported practices, then by definition removing deprecated code is not a

> break in compatibility.This is why it�s essential for our project to

> define
> our terms properly, no matter what direction we take. 5) Can you give an

> example of a break in backward compatibility?Sure. An example of a break

> in
> backward compatibility would be an API that used to do a certain function
> in version V that suddenly stops doing the same function or produces a

> fatal error in version V+1.However, if an API gets deprecated in V and

> gets
> removed in version V+5, when V+6 comes along, that is not considered a
> break in compatibility since the API was deprecated in V, worked in V+1,

> and was removed in V+5 and is not supported in V+6.5) Why not just use
> semantic versioning instead of the proposed changes?Semantic versioning

> doesn�t tell you whether or not you should have backward compatibility
> changes. It merely tells you what you should do *if* you have breaks in
> backward compatibility. My suggestion transcends semantic versioning and
> states that we should never have breaks in backward

> compatibility.Furthermore, semantic versioning is for APIs whereas Joomla

> contains APIs and more. There�s UI to take into account, etc, that needs
> to
> be properly communicated. For example, an overhaul of the admin area could

> trigger a major release (e.g. J4).6) What happens to STS and LTS?Those

> terms are confusing and unneeded. With the proposed changes, there�s only

> the �latest Joomla�.7) What would a move from a Joomla 3 to Joomla 4 for

> users look like under the proposed changes?It would be a one-click

> upgrade.Currently around 55% of our users are on Joomla versions that are

> EOL, unsupported, and
> insecure:http://w3techs.com/technologies/details/cm-joomla/all/all
> <http://w3techs.com/technologies/details/cm-joomla/all/all>The propose
> changes would greatly mitigate this types of issues from occurring in the

> future.8) Aren�t time-based releases causing rushed, sloppy code and
> review?No, time based releases do not require submitted code to get in. If

> it's not ready (even for a committer to review it), it must wait 6 months

> until the next release.That being said, having alpha 3 months before the

> release and having beta 2 months before, gives plenty of breathing room,
> especially with a lighter core and unit tested developer API.----Your
> feedback is welcomed and appreciated. Thank you for taking the time to
> read

> through this and give it consideration.Sincerely,Nick Savov*

[Nick Savov]

@Amy
Great feedback!

For the bloated part, my suggestion was to lighten the Joomla core and get
rid of unneeded extensions from the core. We can make them as installable
extensions through the Install from Web feature instead.

I agree though that deeper cuts would probably need to be made and we
could start with Joomla 4. The core really should be a light platform (for
lack of a better term) that can be easily extended and customized through
a UI. We should be very conservative in what features get into the core.

That being said, I don't want us to re-rewrite the CMS from scratch and
not have a *very* easy upgrade path (and not just data; extensions need to
easily work too) from Joomla 3/2.5 to Joomla 4. It needs to be dead easy.
We honestly can't afford another massive migration and leave 100% of our
current users stranded again and hope they'll some day manage to get out
of the sinking sand on their own or go somewhere else.

High-five on the public API :) Great example! I wish we had more
discussions about potential solutions like this. Hopefully other
developers will chime in on it.

Concerning semantic versioning, it doesn�t tell you whether or not you

should have backward compatibility changes. It merely tells you what you

should do *if* you have breaks in backward compatibility. My suggestion is
to not have breaks in backward compatibility ever (this assume the
definition I gave).

It also doesn't tell you how big of backward compatibility changes you
should have or whether you should have an upgrade path.

Finally, it doesn't deal with anything other than public API. For example,
it doesn't deal with UI changes.

That being said, if we maintain backward compatibility for our users, I'm
fine with going with semantic versioning though, because it doesn't say
that "Major version X (X.y.z | X > 0) MUST NOT be incremented if only
backwards compatible changes are introduced to the public API." i.e. it
doesn't strictly disallow a major incrementation if you have a major new
features that are backwards compatible, but which you want to communicate
with a major version (e.g. a backward compatible admin UI overhaul). It
only states what you should do if you have a break in backward
compatibility. That's the way I read it anyway.

@Andrew,
Thanks for the feedback! You missed the forest for the trees though.
Please see each comment in the context of the rest of the information
surrounding it.

For example, I never suggested we shouldn't remove code, and in fact I
mentioned deprecation and removal of code several times, so statements
like "it's ridiculous to suggest we should never remove code" are invalid.

By fragmented, I mean that each Joomla version is its own little island
without a good bridge between each. We're getting better (e.g. 2.5 to 3 is
a easier than 1.5 to 1.6 in many respects), but it's still too hard to
migrate between major versions.

There's nothing unusual about wanting Joomla to declare a public API. Many
people don't share your view concerning the public methods (e.g. see the
other comments in this discussion concerning public API as well as the
comments at
https://groups.google.com/d/msg/joomla-dev-cms/YhPvWTJS4go/wgqVQbkXGPMJ).
The Joomla project needs to declare a public API that will be the contract

between Joomla and developers for API that we�re agreeing will be

available to developers and stable. If the project decides that it should
be all the methods that are declared public via OOP, so be it, however
there is currently nothing that states so and there isn't a consensus on
it. The Joomla project needs to declare what it's public API is and there
should be not questions about it.

Concerning definitions, to be honest, I'd be completely fine moving to
strict semantic versioning assuming we don't break peoples' sites and
upgrades (even across major versions) are extremely easy.

Concerning all your comments about Wordpress at the end, you're mistaken
and your comments are full of either red herrings or straw-mans fallacies.
I don't take your comments out of context and compare them to Drupal's
strategy and tell you to jump ship to there. I'd appreciate it if you'd be
as considerate. If you'd like to compare strategies to different projects,
I think that's good and you're encouraged to do so, however don't throw in
a bunch of invalid arguments in the middle.

Concerning semantic versioning not being just for API, you're very
mistaken. It's for API. It has no concern for UI. Some projects make use
it for their versioning mechanism, however that doesn't escape the fact
that semantic versioning as defined at http://semver.org/ is ONLY for API.
That's what the whole doc is about.

@Chris, great doc! I commented in the gist.

Kind regards,
Nick

>
> On Thursday, December 12, 2013 3:39:15 PM UTC-6, Nick Savov wrote:
>>
>>

>> *It also results in a bloated CMS that�s difficult (or near impossible)
>> to
>> maintain long-term.*

>>
>
> And yet, you are calling for keeping it all. Encourage you to think about
> that point because I believe there are some tough choices facing Joomla
> that involve deep cuts in this area, otherwise, I don't think things will
> ever be manageable. It'll just be more and more and more failures.
>
>
>>

>> *Finally, the last major issue is that it results in confusion as to

>> which
>> Joomla CMS version to use and when. Should a user migrate from Joomla
>> 1.5
>> to Joomla 2.5 or Joomla 3? <snip> Joomla versions, our users as a whole

>> still don�t understand because it�s too complicated.*
>>
>
> *It really is a problem.*
>
>
>
>
>
>
>
>
> * Total Agreement.In order to achieve this, the following policy is
> required: - Backward compatibility* must always be maintained, even across
> major versions- The Developer API* (i.e. public API) should never break-
> Developer API should be clearly stated and documented.- Developer API
> should be fully unit tested to maintain integrity- Deprecations* are
> maintained at least 2 years before being considered for removal .In all of

> these proposals, this is the only one that is calling for defining the
> API,

> and IMO, clarifying the API is the only possible way out.But, I would not

> go further than defining the API at this time. I would get an agreement on
> that -- and the very act of doing so will mean BC breaks. There are far,
> far, far too many ways that developers can use the environment and those
> should be significantly slashed and Interfaces should be defined for the
> CMS to act as clear contracts as to what developers should (and should not
> be using.) As an example of how this would help -- take email as a
> service
> and offer a single method "send()" with no parameters. Inject all input
> using the framework's DIC and the API is one method. Now, what the
> framework folks do inside is irrelevant to the CMS. If they offer options
> beyond phpMailer -- including Swiftmail -- and Zendmail -- it won't change
> the API. I'm very concerned at all of these proposals I am seeing which
> continue to avoid getting down to a cleaned up, scaled back, well
> documented, fully tested API. When the API is agreed too, those other
> matters, like how long support lasts and whether or not Semantic
> Versioning
> will be used, etc., etc., will be easier to tackle. But first, what is the
> API? Does everyone agree on that? That alone will be a big job but it's
> the
> way forward, avoiding it means more of the same in terms of problems,

> quality issues, BC breaks.It's definitely worth looking at why WordPress

> is
> able to upgrade without problems. For a very long time, WP has had a
> clearly defined API. And, that isn't all public functions -- it's a clear
> set of developer functions, nothing less than what they need and nothing
> more. On the issue of BC support, yes, WP is the one to emulate. But,
> before Joomla is in a position to do what WP is successfully doing, there

> will need to be cleanup changes, there is just no way around that.*

[Andrew Eddie]

On 13 December 2013 16:20, Nick Savov <> wrote:
> For the bloated part, my suggestion was to lighten the Joomla core and get
> rid of unneeded extensions from the core. We can make them as installable
> extensions through the Install from Web feature instead.

To be fair, that doesn't really solve anything. It just pushes code we
still have to maintain into another repository. That's completely ok
but you aren't technically "solving" bloat. The bloat still exists.

> That being said, I don't want us to re-rewrite the CMS from scratch and
> not have a *very* easy upgrade path (and not just data; extensions need to
> easily work too) from Joomla 3/2.5 to Joomla 4.

You can rewrite the CMS from scratch and still have an easy upgrade
path for the core and possibly a bridging layer. That's what we did
with 1.0 to 1.5 more or less, jsut without the easy migration path.

> Concerning semantic versioning, it doesn’t tell you whether or not you
> should have backward compatibility changes.

The draft says that Joomla software versioning follows Semver
principles. Here's the simple test:

a) Increment patch version is only bugs fixed.
b) Increment minor version if new features added.
c) Increment major version if you break b/c

So now you walk over to Github and examine all of the pull requests
that have been merged. Did any break b/c? No, then it's either a minor
or a patch version to release next. Simple!

> It also doesn't tell you how big of backward compatibility changes you
> should have or whether you should have an upgrade path.

Neither does yours. What are you going to do; jump from version 3 to
version 7 because it feels like that is a big enough gap?

> Finally, it doesn't deal with anything other than public API. For example,
> it doesn't deal with UI changes.

Semver says:
"Software using Semantic Versioning MUST ..."

The proposal I've outlined says that the software that Joomla releases
is covered by Semver. So regardless of what Semver says, I have
defined that the same principles apply to the version of the released
software. It doesn't really matter where you interpret Semver to
apply, I'm saying it MUST apply.

> For example, I never suggested we shouldn't remove code, and in fact I
> mentioned deprecation and removal of code several times, so statements
> like "it's ridiculous to suggest we should never remove code" are invalid.

Removing code by definition introduces a backward compatibility.
Remove your opening bullet point and the problem goes away.

> By fragmented, I mean that each Joomla version is its own little island
> without a good bridge between each. We're getting better (e.g. 2.5 to 3 is
> a easier than 1.5 to 1.6 in many respects), but it's still too hard to
> migrate between major versions.

Yes I understand that. That's why I'm proposing to extend the life of
the 3.x version so you can reclaim more of the ocean. Let we developer
that want to nuke the old code innovate.

> The Joomla project needs to declare a public API

Semver says:
"Software using Semantic Versioning MUST declare a public API. This
API could be declared in the code itself or exist strictly in
documentation. However it is done, it should be precise and
comprehensive."

We do "This API could be declared in the code itself" and we also
publish http://api.joomla.org. The code-style rules are also used to
enforce this. We use phpDocumentor to process it. We also follow
standard OOP architectural principles for visibility of classes and
properties.

Bottom-line: we already do it. Read the API docs.

> Concerning definitions, to be honest, I'd be completely fine moving to
> strict semantic versioning assuming we don't break peoples' sites and
> upgrades (even across major versions) are extremely easy.

You have to appreciate there is a lot of devil in the detail to that
comment. A major upgrade from, say, 3 to 4, should be easy if

a) the user has not hacked the core code
b) the user's extensions are not using code marked as deprecated in 3
c) the template is compatible

I agree, we don't want another migration like 1.5 to 1.6 for a long
time. However, that doesn't mean that all sites will be easy to
upgrade.

> Concerning all your comments about Wordpress at the end, you're mistaken
> and your comments are full of either red herrings or straw-mans fallacies.

You've made the point to me a number of times about Wordpress. There's
no shame in taking ideas from elsewhere so I'm not sure what your
problem is. Your strategy IS virtually the same as WordPress and you
HAVE referred to it to me and others before. In fact, we've already
had this conversation with the entire PLT and we are pointing out
exactly the same problems with your proposal.

> Concerning semantic versioning not being just for API, you're very
> mistaken.

My draft defines Semver as covering all of the software and also
outlines what is covered under b/c. You argument while possibly
literally correct (a developer would not interpret it the way you
are), is irrelevant because I clarify the scope of Semver.

> @Chris, great doc! I commented in the gist.

Don't do that because notifications are not available for gists.

Regards,
Andrew Eddie

[Andrew Eddie]

More on Public API

Nick, here an example of the documentation for a random method:
https://github.com/joomla/joomla-cms/blob/master/libraries/cms/application/administrator.php#L36

That's called a DocBlock (http://en.wikipedia.org/wiki/PHPDoc) and
that's what we developer consider "published" on it's own; but we go
one step further:
http://api.joomla.org/cms-3/classes/JApplicationAdministrator.html

That's the pretty version. However, the DocBlocks are all read by most
API so you can have the API documentation within your development
environment. The DocBlocks are mandatory for submitted code unless
someone changed the rules.

Next, you can read about visibility here:
http://php.net/manual/en/language.oop5.visibility.php

Our practice with visibility is that `public` and `protected` fall
under our consideration for backward compatibility, because both are
exposed to the PHP developer while `private` does not.

In JavaScript land things are a little trickier but similar
methodologies are followed. However, we don't do a good job of
publishing our JS API and that is something that needs to be improved.
There are tools and techniques out there, we just have to find them
(for example, this is obviously done with an automated tool
http://api.jquery.com/jQuery/ and that's what you call "the Public
API").

But just because Semver says it has to be done doesn't mean it's going
to magically appear because we say we are using Semver. We have to do
the work. Conversely, having poorly documented JS API doesn't
invalidate Semver. We humans still decide how to apply it.

I hope this clears up any misunderstanding you have about standard
developer practice.

Regards,
Andrew Eddie

[Andrew Eddie]

On Friday, 13 December 2013 08:34:35 UTC+10, Amy Stephen wrote:

In all of these proposals, this is the only one that is calling for defining the API, and IMO, clarifying the API is the only possible way out.

Amy, I don't understand why you need to define the API. The API is actually what we ship in the ZIp. It's the collection public and protected OOP stuff, plus Javascript and then CSS to a degree. Can you help me understand what you are getting hung up on because I've never, ever considered anything less that the whole is subject to backward compatibility assessment. I think I've said this before, defining Joomla's API is a bit like putting "contains nuts" labels on a bag of peanuts. I don't get it.

 

But, I would not go further than defining the API at this time. I would get an agreement on that -- and the very act of doing so will mean BC breaks. There are far, far, far too many ways that developers can use the environment and those should be significantly slashed and Interfaces should be defined for the CMS to act as clear contracts as to what developers should (and should not be using.)  

That's covered by OOP in my opinion. Public and protected visibility is fair game, private is not. I don't see why we can't keep it simple like that.

 

As an example of how this would help -- take email as a service and offer a single method "send()" with no parameters. Inject all input using the framework's DIC and the API is one method. Now, what the framework folks do inside is irrelevant to the CMS. If they offer options beyond phpMailer -- including Swiftmail -- and Zendmail -- it won't change the API. 

I totally agree but that's an architectural strategy, albeit one I want to head in. However, here is the pickle I'm in. I want to use the Framework because it's fresher and lighter and all that, but if I try to blend it with 3.x, I eventually collide with type hinting (JModel vs \Joomla\Model\Model as a simple example). So as someone passionate about innovation, what do I do? 4.0 is due next March and that's WAY too soon to blow up the code. I need to wait another 2 years. I can tell you I am not going to hang around no matter how much loyalty I have to the project. I'll probably go join the PageKit guys.

Alternatively, I can maybe lobby for changing the dev strategy. Do we need 4.0 next March? Is there a compelling reason to introduce a b/c break? I don't think so. So let's change to a Semver-based system and let 3.x carry on it's merry way under the current architecture. That will leave people like you and me to take bits and pieces of all the code we've played with and build something new over the course of next year. Let's do heaps of alpha's. By late 2014 maybe we are ready for beta - let's keep it there until it's stable.

 

I'm very concerned at all of these proposals I am seeing which continue to avoid getting down to a cleaned up, scaled back, well documented, fully tested API. When the API is agreed too, those other matters, like how long support lasts and whether or not Semantic Versioning will be used, etc., etc., will be easier to tackle.

I don't agree. As I've said the next major break is next March. and the next available choice after that is 2 years following. I don't see either of those dates being practical for dev's that want to work seriously on improving the architecture. The bottom line is if the developer strategy remains as is, then I'm moving on (we've already lost Louis, Rob, Sam and others for the same reasons - when do we decide to stop the bleeding?).

 

But first, what is the API? Does everyone agree on that? That alone will be a big job but it's the way forward, avoiding it means more of the same in terms of problems, quality issues, BC breaks.

I suspect what you really want to know is what "stuff" is covered by backward compatibility considerations. To that end I say:

* All non-private PHP

* All non-private JavaScript

* All XML and JSON schema

* All database schema

* All language keys

I've left off CSS because I don't know how to define how it could be covered, not because I don't think it's important.

 

It's definitely worth looking at why WordPress is able to upgrade without problems. For a very long time, WP has had a clearly defined API. And, that isn't all public functions -- it's a clear set of developer functions, nothing less than what they need and nothing more.

I don't think it has anything to do with how the API is defining. The important point is that they do not introduce b/c breaks at all; virtually never. Joomla has the same set of clear developer functions to use. Know, do we have good tutorials about them? No. But do we enforce DocBlocks? Yes (or at least we used to, things may have changed since 1.7).

 

On the issue of BC support, yes, WP is the one to emulate. But, before Joomla is in a position to do what WP is successfully doing, there will need to be cleanup changes, there is just no way around that.

I disagree completely because if that were try we should still be able to run Mambo 4.5.2 extensions in Joomla 3. So many technologies have moved on since 1.5 was conceived and we are still living with a lot of baggage from that version. All I'm proposing is we let 3.x live on and guarantee no b/c problems (with the possible exception of this Bcrypt fix) and let people that do want to break a few things work on 4.0 over the next year. The alternative is that all the innovators leave the project. Let that sink in because we've already lost too many.  What's the worst that can happen? We realise 4 is too big a jump and we spin it off as something else? That's better than plan B (a project fork) isn't it?

Regards,

Andrew Eddie

[Hannes Papenberg]

>
> I'm very concerned at all of these proposals I am seeing which
> continue to avoid getting down to a cleaned up, scaled back, well
> documented, fully tested API. When the API is agreed too, those
> other matters, like how long support lasts and whether or not
> Semantic Versioning will be used, etc., etc., will be easier to
> tackle.
>
>
> I don't agree. As I've said the next major break is next March. and
> the next available choice after that is 2 years following. I don't see
> either of those dates being practical for dev's that want to work
> seriously on improving the architecture. The bottom line is if the
> developer strategy remains as is, then I'm moving on (we've already
> lost Louis, Rob, Sam and others for the same reasons - when do we
> decide to stop the bleeding?).

Amy, consider that I've been trying to get essentially the same routing
changes accepted since 2 months before Joomla 1.6 was released. I didn't
push for it for 3.0 because I had other, personal stuff to attend to.
And thus I missed my only chance to get this in, it seems. So far it has
always been shot down with regards to b/c breaks and the comment, that
3.0 or 4.0 is not in development yet. I might be stupidly stubborn
sometimes, but consider me the next one to leave when Andrew packs his
stuff. And I agree with him here completely.
>
> *
>
> *On the issue of BC support, yes, WP is the one to emulate. But,

> before Joomla is in a position to do what WP is successfully
> doing, there will need to be cleanup changes, there is just no way

> around that.*
>
> *

>
> I disagree completely because if that were try we should still be able
> to run Mambo 4.5.2 extensions in Joomla 3. So many technologies have
> moved on since 1.5 was conceived and we are still living with a lot of
> baggage from that version. All I'm proposing is we let 3.x live on and
> guarantee no b/c problems (with the possible exception of this Bcrypt
> fix) and let people that do want to break a few things work on 4.0
> over the next year. The alternative is that all the innovators leave
> the project. Let that sink in because we've already lost too many.
> What's the worst that can happen? We realise 4 is too big a jump and
> we spin it off as something else? That's better than plan B (a project
> fork) isn't it?

Considering the quality of code that we have and the shit that WP has,
I'd choose the Joomla way over WP every time. Yes, we need to improve in
that regard, but WP is definitely not an example on how to handle b/c
and keep good code. I do have however a different idea about the whole
3.x thing than Andrew. To me, we have to move towards a point where we
say "3.x is now in maintenance mode and this is the version that the
normal users use to setup a site." No new features to the 3.x series
from that point forward and only then can we drop 2.5. All that said, I
think 3.x should be the last release of Joomla with any number behind
any dots. As I said earlier, I think we have to move to a distribution
model and we have to do that move in the 3.x series. That can be done in
a backwards compatible way. We could then call 3.7 or something the last
release in the 3.x series and every extension would have to support one
LTS release and one new development (at least for the time being. There
might be more LTS versions that you'd have to support as an extension.)
and when we say "There has been sufficient new development in some
areas", we release Joomla 4 as a new release of the distribution. All
bugfixes and new development between 3.x and 4 and between 4 and 5 are
done solely in the different extension-sub-projects.

Hannes

[Andrew Eddie]

Nick, reflecting more, I think you are trying to tie too much
architectural direction thinking into the development strategy as a
whole. I think you should separate those concerns much as we do with
code.

My view is the development strategy should define:

1. What the version numbers mean (major, minor, patch)
2. What the development stages mean (alpha, beta)
3. What the backward compatibility policy is
4. What the deprecation policy is
4. What the security policy is

That's is for the "development strategy".

Now, "ease to upgrade" is an "architecture strategy" and that is a
completely different kettle of fish, quite distinct from all the
points above. the architectural strategy would define how much change
do we tolerate between minor and major versions (but you need a simple
versioning strategy in place first). Mixing the two is just confusing
and that's what is happening here.

What I think you need to do is define behavioural goals for the future
of the CMS, and then we can see if we can codify them. For example,
these goals, which fall under and architecture strategy, might
include:

a) All patch and minor upgrades MUST happen via the Joomla Admin
b) We must implement version dependency checking so we tell users if
it is safe to upgrade
c) We MUST provide an upgrade path to the next major version before
the first RC of that major version
d) We SHOULD be moving towards using more of the Joomla Framework in the CMS

etc

Do you see the difference? Both are important, and I suspect we agree
on a lot of the points, but my opinion is they are separate but
equally important documents. One policy is used to set our versions.
The other policy is used to tell us whether or not we can actually
merge the code (no, this change is too big for this version we aren't
cutting another major any time soon) - that's a big difference.

But the pressing issue for me is what the hell is happening March next
year - 3.5 and a mandatory 4 or can we do it a better way? How about
we refactor the current development strategy first?

We both agree that b/c only happens at the major version.

We both agree deprecated code gets removed in the major version.

I think we agree that new features only get added in minors and bugs
gets fixed in patch releases.

I think we are 99% there if we separate out your "upgrading must be
easy" concerns into another policy. Let's get this easy one out of the
way. Architectural direction is harder to quantify and codify and we
are all going to have to talk to a lot of people about what they want.
For now that, just allowing 3.x to continue and 4.0 to arrive at a
date to be advised is a reasonable compromise don't you think?

Regards,
Andrew Eddie

[Leo Lammerink]

I think we are 99% there if we separate out your "upgrading must be
easy" concerns into another policy. Let's get this easy one out of the
way. Architectural direction is harder to quantify and codify and we
are all going to have to talk to a lot of people about what they want.
For now that, just allowing 3.x to continue and 4.0 to arrive at a
date to be advised is a reasonable compromise don't you think?

Regards,
Andrew Eddie

I cannot concur more!

Leo Lammerink
 

[Andrew Eddie]

On 13 December 2013 18:47, Hannes Papenberg <> wrote:
> and keep good code. I do have however a different idea about the whole
> 3.x thing than Andrew. To me, we have to move towards a point where we
> say "3.x is now in maintenance mode and this is the version that the
> normal users use to setup a site." No new features to the 3.x series
> from that point forward and only then can we drop 2.5. All that said, I
> think 3.x should be the last release of Joomla with any number behind
> any dots. As I said earlier, I think we have to move to a distribution
> model and we have to do that move in the 3.x series.

I think there are a number of ways to go from here, and I don't
disagree with your suggestion. It's something I have thought about.
The bottom line is we need to get the fixed release of 3.5 and 4.0 out
of the way to have any chance of achieving anything. After that, I
think many roads lead to Rome and the more people trying different
ideas the better. We do need to fix JPATH* hell in 3.x though - it's
killing the ability to do unit tests. We need a plan to fix our code,
our testing and our documentation. I am prepared to help but only if
we have a path to innovate and to deprecate with extreme prejudice
(with the caveat that the new major version MUST have an upgrade path
in place - we cheated with 1.6 but let's not do that again).

In all honesty, I think it would be easier to completely build 4.0
from scratch under the distribution model, have extension developers
convert all their extensions to the "4.0 way" and put the bridging
layers in 3.x and 2.5 (not the other way around). THAT, my young
friend, would be extremely cool to pull off :)

Baby steps though.

Regards,
Andrew Eddie

[Joomla Forms]

Hi @all,

I know it's always easy to say but the discussion now was predictable since Joomla! 1.6 came out.

You can't and you shouldn't ever tell people that their systems need a complete update every 2nd year. Be it at least for psychological reasons.

Or does anybody of you want to know what work needs to be done in 2 years (in best case)? Do you want to know what garbage you have to clean next week? I don't think so.

Joomla! is not a linux distribution or an enterprise portal system with expensive consulting contracts attached to it.

If you do so, then at least provide them with the promise that updating is really easy or at least not that painful (remember the really great 1.0 => 1.5 compatibility layer).

Like most of you, I am right at the spot, the end-users and they all complain about the same things in joomla since 1.6, namely:

 - Breaks on updates
 - Hard to use

If you compare Joomla! directly with WP in that matter then you see why WP is there where it is. Just yesterday when WP 1.8 came out I realized I didn't have to touch any of my plugins. And that since ages already (maybe a bit of CSS).

However, I largely prefer Joomla! over WP because it has the better concepts and scales a lot better than WP. That is incredible high value and just needs to be addressed properly to people.

I really hope the new marketing group will catch that up and spread the word about it.

The "hard-to-use" issue can be overcome, too. I don't think there need to be drastic changes except starting to open people's eyes about what Joomla! actually is.

So in short, my personal advise if anyone is interested into it:

1. Cancel the current release schedule, do updates as required and not as planned and focus back on the piece of software called Joomla! and what it stands for. If a major release will be made, check Joomla!'s history how it was addressed at that time and get back to it (no, times didn't change since that, see WP)

2. Make the promise - and keep it - to always provide a compat layer for the previous version and actually keep that layer updated

3. Simplify the community structure as seen from outside: I see a gazillion of responsibilities amongst the different parts of the project. I totally lost the overview what group is responsible for what and who is in charge. If I am not able to follow and get identified any longer with the project, how a regular joomla user should be? 

Put some persons on front who speak with a clear voice to the community. One for development, one for community and one for the entire project (aka marketing). Even extend that persons right's to decide things if a discussion leads to nowhere. In short: get rid of the current structures, entirely and replace it with something simpler, less regulated.

Less talk, more do

Had to let that out. I am getting a bit scared if long term members like Andrew, whose works I appreciate a lot, start to mention Pagekit as real perspective, it gets serious.

So, if Andrew and others here promise to keep giving it a try, I'll promise to join development and donate 20 hours / week for any kind of job that needs to be done. I don't care if it's bugfixing, developing, new features, less features, etc. Hell, I would even stand on the street and give away joomla shirts for free if necessary. There needs just to be one person who contacts me directly and say, guy here you go, that needs to be done, go ahead (I am not talking about links to some links to some links where there probably somebody is in charge would could actually ... send you a link).

Best Regards,

Markus 

[Joomla Forms]

...sorry, WP 3.8 of course

[Amy Stephen]

It's really too hard to talk in these discussions -- would be easier if we were all setting around a table.

Andrew - If your goal is 4.0 is a fresh start, I agree with that approach. Long time ago, I came to the conclusion that large architectural changes will be required for the Joomla CMS to move forward.

But, whether the project continues with the existing architecture or starts anew, I still believe that using a traditional definition that all private and public methods are the API is no longer appropriate and most projects are finding such to be true. In fact, if you want to talk about bleed rate, I would assert that using the traditional definition for the API is exactly what causes developers to leave. There are very, very few private methods in the code base -- and the traditional API means there is absolutely no room for innovation or change.

The reality is there is a very small subset of classes and methods intended for use by extension developers. Those include Module and Component helpers, the top layers of the MVC classes, the XML for the Authorisation system, the Tags help, and so forth. Those are the ones that should be clearly identified. Those are the ones that should be held in place by Interfaces and unit testing. Those are the contracts between the framework team and the CMS extension developers.

But the vast majority of classes and methods are not intended for use by extension developers and should not be considered part of the API. Extension developers have no business using JWebApplication directly, for example, though every single method and property is public or protected.

When the Dependency Injection Container is in use by the CMS, I would expect some sort of Service Provider interface that developers use to interact with the Container. That's what they should use -- not the Container itself.

You back yourself into a corner using visibility as the API marker. Essentially, since there is so little private code, what that says is it's all the API. If that's true, then nothing can ever change. That definition fails to inform CMS extension developers what code they should be using and it unnecessarily restricts the framework team from innovating. 

[Amy Stephen]

On Friday, December 13, 2013 12:20:12 AM UTC-6, Nick Savov wrote:

Concerning semantic versioning, it doesn�t tell you whether or not you

should have backward compatibility changes. It merely tells you what you
should do *if* you have breaks in backward compatibility. My suggestion is
to not have breaks in backward compatibility ever (this assume the
definition I gave).

My point is that Semantic Versioning numbers itself on API breaks. So, the focus is there.

Knowing the code base, it's doubtful to me that you can restrict BC changes. Setting a policy is easy. Making it happen, not so easy. Even with an agreement not to do so, my point is you'll never know if you did or not if you use public/protected as the definition and you don't have unit tests to hold the API in place. So, it's just a strong statement without any teeth to it. (If that makes sense.)

I think it's fair to say no one _wants_ to break the API. No one gets high moral ground here. People who are advocating change are doing so with an understanding of the architectural weaknesses and awareness of the problems it causes. They are not "playing" with code or treating Joomla as a toy (not saying you've said that but I've heard it so many times I could scream.)

Everyone wants stability but a policy doesn't make it so -- hard work, smart developers, strong architecture, makes it so. And, to get there, change will be required.

 

It also doesn't tell you how big of backward compatibility changes you
should have or whether you should have an upgrade path.

Finally, it doesn't deal with anything other than public API. For example,
it doesn't deal with UI changes.

Correct. There is a reason for that -- when the API is clearly defined, it can be "proven" with unit testing. The same is not true with UI changes. Therefore, it needs to be treated differently. Putting these together creates a muddled environment where people start to get subjective about API changes. Since the PHP API can be definitely defined and tested by code, get that in place, first, and secure it with unit testing. There is no wiggle room there.

Then, take your UI challenge and deal with it separately using whatever tools you can to help manage that environment. But it can't be treated the same.

For 4.0, if it's a departure from the current architecture, the best approach, IMO, is to set some coding practices -- for one, never ever ever render HTML in a PHP Class. All rendered output should be in a non-class view. That has big implications on the current HTML and JForm classes but it will separate the presentation layer so that it can be managed. Right now, it's not possible to do so.

[Donald Gilbert]

That being said, if we maintain backward compatibility for our users, I'm 
fine with going with semantic versioning though, because it doesn't say 
that "Major version X (X.y.z | X > 0) MUST NOT be incremented if only 
backwards compatible changes are introduced to the public API." i.e. it 
doesn't strictly disallow a major incrementation if you have a major new 
features that are backwards compatible, but which you want to communicate 
with a major version (e.g. a backward compatible admin UI overhaul). It 
only states what you should do if you have a break in backward 
compatibility. That's the way I read it anyway. 

Nick, you've read it wrong then.

You ONLY increment major version when you break BC. The whole point of SemVer is that it's an indication to the end users of when to expect change. If you're incrementing versions willy nilly as you suggest, you're doing it wrong.

Your understanding and definitions of semantic versioning and backwards compatibility are off. If you could see that, and accept the fact that when you remove deprecated code it is a BC break (which means it only happens in major versions) our visions for the development strategy would start to look a lot more similar. As long as you insist on defining things in your own way and trying to force that erroneous definition upon others, your strategy will not work, and can not work. I'm not sure what else needs to be said or who else needs to say it for you to get the point, Nick. 

[Donald Gilbert]

I don't take your comments out of context and compare them to Drupal's strategy and tell you to jump ship to there.

Our strategy looks nothing like Drupal's, so if you were to do such a thing, you'd look like a fool. 

[Nick Savov]

Sorry, but you don't understand what I'm saying.

Kind regards,
Nick

[Nick Savov]

At the very highest point of view, Drupal has big breaks in backward
compatibility between major versions and practically rewrite its CMS each
time.

Also, refer to https://drupal.org/node/2135189 to see the similarities of
their proposed changes.

Kind regards,
Nick

[Leo Lammerink]

Nick,
We all do understand a lot of what you are saying but you do not listen
to the majority of the (very experienced) posters. You want to keep this
position up and continue this and risking loosing a lot of the majority
of TOP contributors or want to "finally open up?" Since I will not allow
this (and with me many others!) to jeopardize this project!

Leo

[Michael Babker]

Well then clarify it, Nick.  We had this same discussion internally 3 weeks ago and now the same talking points and frustrations have spilled out onto this mailing list unchanged.  Your proposal here is almost verbatim the same as what you shared with the PLT originally; did you not take anything out of the discussions that were had then and try to refine your vision?  We have tried to explain to you why, as worded, your proposals cause concern to the DEVELOPERS.  Your proposal is so heavily aimed at USERS and MARKETING that it can't be taken seriously, at least by me, as a DEVELOPMENT strategy that the Joomla project should adopt.

As I also said internally yesterday, as a project there needs to be a single development strategy ratified for all of Joomla.  That means the same principles need to define how development is done, versions are numbered, and strategies communicated for the CMS, Framework, Issue Tracker, and anything else that we support under the Joomla banner.  Then, and ONLY then, should discussions begin on product specific strategies (which, IMO, will be necessary to an extent as the CMS and Framework are targeted towards separate audiences and will have separate goals and roadmaps; the product specific strategy should be an extension of the project's overall strategy, not overruling it).

[Leo Lammerink]

My very simple question is: " What are the qualities of a "Leader" ? Listen indeed.... now listen please to what is being said (kindly request still!)

Leo

[Nick Savov]

Clarify what? He was talking about Semantic Versioning in that "You ONLY
increment major version when you break BC". However, Semantic Versioning
does *not* state that. It even states that it has a "MUST NOT" term in the
first line of its "Semantic Versioning Specification (SemVer)", yet it
doesn't ever use that it in the context incrementing major versions.
Anyway, if that's going to cause such a big divide to make a statement
like that, I'll just drop it. I don't see the point of your guys'
aggressiveness though. It's disheartening.

Umm, I did take things out and I did refine it.

As to developers, I actually ran it by several trusted developers before
posting and they gave it hearty approval.

Kind regards,
Nick

[Nick Savov]

+1 on both points, Amy. Well said.

Cheers,
Nick

>
>
> On Friday, December 13, 2013 12:20:12 AM UTC-6, Nick Savov wrote:
>>
>>

>> Concerning semantic versioning, it doesn�t tell you whether or not you

[Leo Lammerink]

So what...You get half of the most valuable developers over you and
again it was already discussed before and you still want to score points
clearly and jeopardizing the Project (of which you are a 'Leader") Sorry
Nick...That is not good Leadership at all!

Leo

[Leo Lammerink]

Nick, can you please answer how you feel about being a member of the PLT
and not listening to the most senior members of the Production Team and
most Sr. Developers for the Joomla Project ? What is your agenda here?

Did you read what they stated? Do you care?

Leo

[Leo Lammerink]

As to developers, I actually ran it by several trusted developers before posting and they gave it hearty approval

Who, what, when...... Do not quote without being precise in these major important discussions? Quoting sources without mentioning your 'trusted (by whom?) sources is rather funny so to say

Leo

On Friday, December 13, 2013 10:25:56 PM UTC+7, Nick Savov wrote:

+1 on both points, Amy. Well said.

Cheers,
Nick

>
>
> On Friday, December 13, 2013 12:20:12 AM UTC-6, Nick Savov wrote:
>>
>>

>> Concerning semantic versioning, it doesn�t tell you whether or not you

[Nick Savov]

Hi Andrew,

Yes, we agree on a ton of things.

My biggest concern is our users and not leaving them stranded on EOL
versions. Around 55% of our users are currently in that situation
(http://w3techs.com/technologies/details/cm-joomla/all/all).

If there is not a guarantee of an easy upgrade path to Joomla 4 for our
users, then to be quite honest I don't even want to be having the
discussion of adjusting the development strategy to give a carte blanche
to "rewrite from scratch" and "blow things up".

We've been promising our users easier upgrades (even across major
versions) for some time now. We need to keep our promise.

Kind regards,
Nick

[Donald Gilbert]

Nick,

If several of the lead developers and contributors to the project are misunderstanding what you mean, don't you think it's remotely possible that you aren't actually explaining it correctly or that your understanding of it might be off a little?

Bewildered,

Don

[Donald Gilbert]

+1

[Nick Savov]

Yes, that's possible. Can we get back on topic now?

Cheers,
Nick

[Donald Gilbert]

Nick,

"Easy upgrade path" is an easy statement to say, but a hard promise to keep. As Andrew stated, the devil is in the details.

You're assuming:

1) Users haven't hacked core code

2) Extensions that the user is using is following recommended practices

3) The users web host actually supports the update

Any one of those things, outside control of the project, can make an easy upgrade path into a difficult one. Such a saying belongs in a "Joomla Core Philosophies" document, but not in a Development Strategy.

Thanks,

Don

[Donald Gilbert]

That IS the topic, Nick. You just can't see that for some reason. There is absolutely no point talking further about a development strategy until you understand these definitions properly instead of assigning your own definitions to them.

[Nick Savov]

Hi Don,

Yes, easy upgrade path for those that are currently following Joomla best
practices in the Joomla 3 series. We can safely ignore people that have
hacked core files.

Yes, it's a hard promise to keep, but we've made the promise as a project
already. We need to keep it now.

Are we going to guarantee an easy upgrade path or not?

Kind regards,
Nick

> Philosophies" document, but *not* in a Development Strategy.
>
> Thanks,
> Don
>

[Leo Lammerink]

Since the Joomla Project is a Community driven Project I assume that
this Community has some words to say on performance of the Leadership of
this Project? What we do if somebody in PLT is considered not to
represent the general opinion of this Community?

Leo

[Michael Babker]

Per the Semantic Versioning 2.0.0 specification, it summarizes that:

Given a version number MAJOR.MINOR.PATCH, increment the: 

1. MAJOR version when you make incompatible API changes,
2. MINOR version when you add functionality in a backwards-compatible manner, and
3. PATCH version when you make backwards-compatible bug fixes.

Where is the issue in this summary?  Don is 100% correct in his statement in that based on the specification, the major version number does not increment except for when backwards compatibility is broken.  Your proposal to increment the major version number when a feature which is backwards compatible is introduced to accomplish "if you have a major new features that are backwards compatible, but which you want to communicate with a major version" is not in line with this specification which is practically a de facto standard in the development world.

 

Care to share privately what you changed?  How the first post of this thread reads to me is awfully similar to what you originally shared with the PLT.

 

I'm also curious which developers you ran it by.  Not to name and shame or to say they're wrong in agreeing.  But I'm more curious if these are the same developers who are trying to help Joomla be a stable product that can be implemented in a variety of environments or if they are the same ones who have to release updates to their code at every release because their code implementations are not in line with what many would expect to be correct usage of the API.

 

Example: At http://forum.joomla.org/viewtopic.php?f=9&t=825287&start=0#p3103232 an issue is reported with an extension which broke during the 3.1 to 3.2 update.  After troubleshooting, I determined that the cause of the issue is in the developer calling a static method in object context and we did not implement the static method in the new application classes, causing the error noted by that user.  Is this a case where we, as the Joomla developers and leaders, and myself personally as the individual who wrote these classes, screwed up and broke backwards compatibility or is this the fault of the developer who incorrectly implemented the API?

 

As I've said elsewhere, you have concepts in your proposal which need to be addressed, but not all belong at the development strategy level.  And even with strategies in place, the humans enforcing those strategies are still liable to not always follow them or accidentally overlook something which goes against the strategies; what mitigating factors can you introduce to account for human action?

[Joomla Forms]

Hi Nick,

 

Are we going to guarantee an easy upgrade path or not?

I would say yes, but only if Joomla! 4 gets delayed if not put aside for a while, so everybody gets time for well deserved decompression. 

Core devs, extension devs and last but not least the users. Do you know how under what pressure everybody is now already?

Core devs gonna enjoy christmas/new year and get back to it with a fresh mind and how to solve all the different issues.

Don't forget, 3.2.0 contains some issues still that need to be addressed and I only see one thing after the other being thrown in.

Extension devs would also really appreciate a delay. I know colleagues with 14+ big extensions, all need to be rewritten from scratch.

Users will totally loose the faith because either their sites can't get upgraded and/or extensions are not J! 4 ready.

Nick, if you think more globally on things, please please, pretty please listen to the guys.

[Leo Lammerink]

 

I'm also curious which developers you ran it by.  Not to name and shame or to say they're wrong in agreeing.  But I'm more curious if these are the same developers who are trying to help Joomla be a stable product that can be implemented in a variety of environments or if they are the same ones who have to release updates to their code at every release because their code implementations are not in line with what many would expect to be correct usage of the API.

 

and I posted before which Nick is choosing to ignore as l:

 

Who, what, when...... Do not quote without being precise in these major important discussions? Quoting sources without mentioning your 'trusted (by whom?) sources is rather funny so to say

Leo

So NIck can you please be so good to provide your "sources" or are they "secret" ?

Leo

[Amy Stephen]

On Friday, December 13, 2013 9:52:31 AM UTC-6, Leo Lammerink wrote:

Since the Joomla Project is a Community driven Project I assume that
this Community has some words to say on performance of the Leadership of
this Project? What we do if somebody in PLT is considered not to
represent the general opinion of this Community?

Leo

Well, frankly, I think if you were to do a poll of all users of Joomla and it were worded like thus:

On Friday, December 13, 2013 9:49:10 AM UTC-6, Nick Savov wrote:

Hi Don,

Yes, easy upgrade path for those that are currently following Joomla best
practices in the Joomla 3 series. We can safely ignore people that have
hacked core files.

Yes, it's a hard promise to keep, but we've made the promise as a project
already. We need to keep it now.

Are we going to guarantee an easy upgrade path or not?

Kind regards,
Nick

You'd like find a majority support Nick.

Can we try really hard not to make this a witch hunt against someone who is trying to advocate for not leaving users behind?

What in the heck is wrong with saying - yes - an easy upgrade path [for core] must be in place?

Let's give and take, here. Try not to make it personal. I know all of you - you can each be difficult. =)

My full support on the architectural issues continues to go with the framework team. But find ways to hear Nick's point and satisfy his concerns - those are valid.

OK. I have offered Interfaces. I have in the past and will again today offer any or all of my Molajo code. But, I'm tired of offering and being ignored. Call me if you want my help.

Please be awesome to one another, you won't regret it.
 

[Amy Stephen]

On Friday, December 13, 2013 10:38:38 AM UTC-6, Leo Lammerink wrote:

So having said so with all frustrations reading this I simply would like to ask to Open Source Matters:

Take your responsibility as owners and why not erasing all this PLT/CLT etc etc and set these groups up by invitation ( you know who have been very good to the project from the beginning in 2005: People like Brian, Andrew, Sam, Michael, Don, Thomas etc). At present People are in the Production  Leadership Team (example) because (sometimes) they applied for it and not because they have been chosen or selected. These people are suppose to be Leaders and are showing not to be the persons who are pretending to represent us at present?

Leo

Go look at who is on the PLT (it includes some of those listed.) And, take it down a notch, bud. =) It's okay, it's Joomla. *hugs*

[Leo Lammerink]

Amy sorry no poll for me and I do not buy he 'conclusions'..... They are yours and yours only

Leo

[Leo Lammerink]

Amy, NO sorry but agree that it it is posted in general not to direct posts to individuals

[Amy Stephen]

On Friday, December 13, 2013 9:53:56 AM UTC-6, Michael Babker wrote:

 

Example: At http://forum.joomla.org/viewtopic.php?f=9&t=825287&start=0#p3103232 an issue is reported with an extension which broke during the 3.1 to 3.2 update.  After troubleshooting, I determined that the cause of the issue is in the developer calling a static method in object context and we did not implement the static method in the new application classes, causing the error noted by that user.  Is this a case where we, as the Joomla developers and leaders, and myself personally as the individual who wrote these classes, screwed up and broke backwards compatibility or is this the fault of the developer who incorrectly implemented the API?

Good question for Andrew. I'm not sure what method it is but it's obvious that it was a public or protected method or it would not have been accessible. Thus, according to Andrew's definition, it would have been part of the API. Therefore, it would be the fault of the project.

However, I believe your point, Michael, is that the method was never intended for use by the developer and, thus, was the developer's problem.

I would tend to agree with the second position -- IF -- the API had been clearly defined. But, if it's not, then how would anyone ever answer that question? Or, if the API is "all public and protected methods", then why wouldn't the developer be within his or her rights to do so?

How many public and protected methods are there? I'll bet it's thousands. Defined more specifically, I'll be the list could number 100 or so. Much more manageable.

Good example. In terms of common ground, I believe everyone is in favor of the clarity on those very important points. Yes?

[Michael Babker]

IMO, this

--

[Michael Babker]

Bad mouse on the work computer, darn...

 

IMO, this specific issue isn't about public/protected.  In 3.1, JFactory::getApplication() returned an instance of JApplication, so doing JFactory::getApplication()->isWinOS(); worked.  JApplication::isWinOS() (http://api.joomla.org/cms-3/classes/JApplication.html#method_isWinOS) is a static method but was accessible in object context because the application object was an instance of JApplication.

 

In 3.2, our application classes extend JApplicationWeb, and nowhere in that hierarchy is the isWinOS() method included.  Developers who expected it to be available via the application object instead of calling the static method in a static manner experienced a breakage.  So, it's still clear as mud to me; who accepts responsibility for that breakage?  The developer who is calling that static method (there are a couple others, but the forum post I linked used that method specifically) within object context or the project for not accomodating what I would consider incorrect use of the API in a refactoring of the application classes?

On Fri, Dec 13, 2013 at 11:04 AM, Amy Stephen <amyst...@gmail.com> wrote:

--

[Amy Stephen]

On Friday, December 13, 2013 11:11:56 AM UTC-6, Michael Babker wrote:

Bad mouse on the work computer, darn...

 

IMO, this specific issue isn't about public/protected.  In 3.1, JFactory::getApplication() returned an instance of JApplication, so doing JFactory::getApplication()->isWinOS(); worked.  JApplication::isWinOS() (http://api.joomla.org/cms-3/classes/JApplication.html#method_isWinOS) is a static method but was accessible in object context because the application object was an instance of JApplication.

 

In 3.2, our application classes extend JApplicationWeb, and nowhere in that hierarchy is the isWinOS() method included.  Developers who expected it to be available via the application object instead of calling the static method in a static manner experienced a breakage.  So, it's still clear as mud to me; who accepts responsibility for that breakage?  The developer who is calling that static method (there are a couple others, but the forum post I linked used that method specifically) within object context or the project for not accomodating what I would consider incorrect use of the API in a refactoring of the application classes?

So, how does a developer know? Is it written anywhere? Is there an API that says so?

By Andrew's definition if it came in the bag of peanuts and the visibility is public or protected -- the developer can use it. It fits within that standard.

FWIW, I agree with you. The API is something more specific than visibility and docblocks. Most of the core code shouldn't be used by CMS extension developers. In balance, only a few methods of the thousands of methods in the peanut bag are actually intended for use by extension developers. So, why not identify them? Publish that list specifically. It would be much easier to unit test those. It would be far easier to document those. That's the list Semantic Versioning would be based upon.

Several times people have said "We aren't trying to be WordPress!" And, that's fine, but it's kind of a weird response since it doesn't hurt to at least understand how they are able to avoid BC breaks. It's pretty simple -- they have clarity on their API. That's the big secret. They have a small set of functions developers are intended to use and they work very hard not to break those. Everywhere else in the core code, the core devs can do their thing. If someone's plugin breaks because they ventured into deep core waters, it would be bad on them. They weren't using the API.

[Michael Babker]

On Fri, Dec 13, 2013 at 12:17 PM, Amy Stephen <amyst...@gmail.com> wrote:

On Friday, December 13, 2013 11:11:56 AM UTC-6, Michael Babker wrote:

Bad mouse on the work computer, darn...

 

IMO, this specific issue isn't about public/protected.  In 3.1, JFactory::getApplication() returned an instance of JApplication, so doing JFactory::getApplication()->isWinOS(); worked.  JApplication::isWinOS() (http://api.joomla.org/cms-3/classes/JApplication.html#method_isWinOS) is a static method but was accessible in object context because the application object was an instance of JApplication.

 

In 3.2, our application classes extend JApplicationWeb, and nowhere in that hierarchy is the isWinOS() method included.  Developers who expected it to be available via the application object instead of calling the static method in a static manner experienced a breakage.  So, it's still clear as mud to me; who accepts responsibility for that breakage?  The developer who is calling that static method (there are a couple others, but the forum post I linked used that method specifically) within object context or the project for not accomodating what I would consider incorrect use of the API in a refactoring of the application classes?

So, how does a developer know? Is it written anywhere? Is there an API that says so?

 

The doc block for this specific method and the method declaration have always indicated it is a static method.  To me, that means I should always use JApplication::isWinOS().

 

As far as API documentation goes, there was a period of a year and a half where our API site was allowed to rust over and not updated.  It took a few months, but we were able to build a fresh template for the site and get current documentation posted with the 3.2 release.  We now have proper API docs (well, phpDocumentor based output) for CMS 2.5 & 3.2 and Framework 1.0 online, the first updates to the site since Platform 12.1 was tagged in May 2012.  So, we're slowly making progress on that small aspect of the overall larger issue.

 

By Andrew's definition if it came in the bag of peanuts and the visibility is public or protected -- the developer can use it. It fits within that standard.

 

And using it isn't the issue, aside from the method being deprecated (and a replacement documented in the doc block, and inherently API docs, as well), but rather how it's being accessed.

 

FWIW, I agree with you. The API is something more specific than visibility and docblocks. Most of the core code shouldn't be used by CMS extension developers. In balance, only a few methods of the thousands of methods in the peanut bag are actually intended for use by extension developers. So, why not identify them? Publish that list specifically. It would be much easier to unit test those. It would be far easier to document those. That's the list Semantic Versioning would be based upon.

 

To me, I consider everything in the libraries folder the API and have always treated it as such as it pertains to renaming classes, refactoring methods, unit testing, and deprecating code.  The libraries should not be bound to a single component or implementation, and where they are (tags and categories being good examples), we're failing developers by bounding the system to specific implementations.  Interfaces would certainly be a great thing to implement in many places to spell out the bare minimums of what should be done in certain places.  With that said, a developer implementing a module probably shouldn't be extending JSession and doing crazy things that would break sessions on a site, but I guess technically it is possible given the method declarations.  Likewise, I know there's a lot of comments about the HTML rendering code we have; I keep saying refactor to JLayout (and I've done some myself), but it takes time and volunteer hours to accomplish things like that to abstract both our PHP classes and markup from one another.

 

Several times people have said "We aren't trying to be WordPress!" And, that's fine, but it's kind of a weird response since it doesn't hurt to at least understand how they are able to avoid BC breaks. It's pretty simple -- they have clarity on their API. That's the big secret. They have a small set of functions developers are intended to use and they work very hard not to break those. Everywhere else in the core code, the core devs can do their thing. If someone's plugin breaks because they ventured into deep core waters, it would be bad on them. They weren't using the API.

 

There's a difference in learning from the successes of WordPress and just blindly adopting WordPress' development strategy.  I think that's a distinction that's being lost right now.  Andrew Nacin and myself had an extended discussion over lunch about our projects' strategies at JWC and came away learning something (at least I know I did); that's the type of collaboration and learning that needs to happen.

[Amy Stephen]

Good comments, Michael. You have a good handle on things and your ethics, intelligence, work output are second to none. I definitely listen to what you say and I advise others to do so, as well. I do hope you are able to cut back a little on what is the API because I think it will help everyone, but hopefully that point was heard.

Thanks.

[Andrew Eddie]

Nick, the bottom line here is you need to have another go at your
proposal. You need to completely rewrite it in policy-ese .You have
the existing policy, mine and Chris's to look at. What you have at the
moment is a collection of dot-point ideas, not a policy document.

You also need to number the sections and points so they can be easily
referred to when people a debating it.

Put the final result in a gist so that you can make tweaks as people
provide comment, and be prepared to negotiate and compromise. It you
aren't prepared to do that, then expect a lot of resistance.

On technical grounds you need to loose your first dot-point. You
aren't a developer and the developers are telling you the statement is
in conflict with the rest of your proposal. Either you magically know
better than them, or your don't and you have to suck it up.

I get that you are hung up on "easy upgrade path" but you are going to
have to think about how to specify that at a technical level. I've
already given you advice on how to thing about it and that it should
be in a separate policy.

Have another go thinking carefully about the advice you have been
given and see what you can come up with. To make it easy, you could
just fork Chris's or my proposal and just "tweak" the most important
parts of the documents (don't reinvent them just because you can and
want to try and prove something).

Here's a suggestion - take me gist
(https://gist.github.com/eddieajau/a1d2fa56d0e67b95096a), fork it and
add add section "2.6 Upgrade Path" and just work on that (in all
honesty Nick, your dot points are mostly covered by my draft anyway).
Have a think about the text you would add in there based on the advice
I have already given you (like, the upgrade method must be in place
with the first RC, etc). Avoid using subjective requirements like "The
upgrade path must be easy" - that means squat. Think about the
assessment criteria for a pull request - if I merge this will it
satisfy all 4 points of the Upgrade Policy?

Clear?

Thanks.

Regards,
Andrew Eddie

[Andrew Eddie]

On 14 December 2013 04:17, Amy Stephen <> wrote:
> So, how does a developer know? Is it written anywhere? Is there an API that
> says so?

Here's how I would treat it. If the code was added after changing to
PHP 5, I would assume normal oop rules apply and that static's must be
explicitly declared.

If it's old code then the DocBlock wins if it tells you how to use it.

If the DocBlock is not absolutely explicit, then the usage in the core
code wins.

If the usage in the core code is split between, for example, static
and non-static, we have a problem and fixing that would definitely
trigger a new major release. However, we've already fixed all those
problems as far as I'm aware.

> By Andrew's definition if it came in the bag of peanuts and the visibility
> is public or protected -- the developer can use it. It fits within that
> standard.

Amy, is there an example project out there you can point to that
defines it's public API selectively? The idea that only part of the
code shipped is the public API is just foreign to me. I've never, ever
seen anyone do that.

> Several times people have said "We aren't trying to be WordPress!" And,
> that's fine, but it's kind of a weird response since it doesn't hurt to at
> least understand how they are able to avoid BC breaks. It's pretty simple --
> they have clarity on their API. That's the big secret. They have a small set
> of functions developers are intended to use and they work very hard not to
> break those. Everywhere else in the core code, the core devs can do their
> thing. If someone's plugin breaks because they ventured into deep core
> waters, it would be bad on them. They weren't using the API.

Can you show me where they define that "API"?

Thanks in advance.

Regards,
Andrew Eddie

[Grigor Mihov]

Couldn't read the whole discussion, as it quite long. I have read the first post of Nick and while I have disagreement on a lot of things there, there is one thing I fully agree - Joomla! should have one stable version that is recommended by the project. Obviously (for me), it should be the latest one.

This allows more predictability for the whole project.

My 2 cents.

[Amy Stephen]

Andrew -

Earlier, a couple of times I believe, I provided two examples. Symfony2 uses @api in their doc blocks to further restrict what they consider to be their API. And, ZF2 makes clear their API is public methods, only. They use protected classes so that devs can extend/change methods but make it clear doing so is at their own risk. Symfony uses Private to prevent that possibility and that got a little pushback.

In both cases, though, that's framework code. It starts to get more complicated as you offer code in layers. Your framework packages each have an API and a lifecycle of their own. Are you going to tie the CMS and each of those packages together? Or, will the CMS have Interfaces to these packages and those Interfaces will form the API for the CMS -- and create independence from the packages? (I'd do that.)

I provided a direct link to the WordPress API in the quoted message. Please use that - don't want to look it up again.

[Andrew Eddie]

On 14 December 2013 09:38, Amy Stephen <> wrote:
> Earlier, a couple of times I believe, I provided two examples. Symfony2 uses
> @api in their doc blocks to further restrict what they consider to be their
> API. And, ZF2 makes clear their API is public methods, only. They use
> protected classes so that devs can extend/change methods but make it clear
> doing so is at their own risk.

Huh, well there you go. That's fine for them. I'll stick with the
classical approach :)

> Symfony uses Private to prevent that
> possibility and that got a little pushback.

Well that's just OOP 101 (unless some dingaling is using reflection).

> In both cases, though, that's framework code. It starts to get more
> complicated as you offer code in layers.

Not really. I've always considered all the code fair game and falling
under b/c consideration.

> Your framework packages each have
> an API and a lifecycle of their own. Are you going to tie the CMS and each
> of those packages together?

Quite possibly, and possibly some Symfony packages as well. What's
changed is a better way to handle the dependency management via
Composer (as opposed to the monolithic Joomla Platform scenario), and
I'm guessing the UI people will head into Bower and Grunt territory as
well.

> Or, will the CMS have Interfaces to these
> packages and those Interfaces will form the API for the CMS -- and create
> independence from the packages? (I'd do that.)

It's a case-by-case thing Amy. PHP-FIG works slow so we do our best
with our own interfaces. None of this will be solved overnight.

> I provided a direct link to the WordPress API in the quoted message. Please
> use that - don't want to look it up again.

Sorry, missed it in the embedded link, for example.
http://codex.wordpress.org/Options_API

I'm sorry (with a half chuckle) - I just can't take a non-OOP API
seriously in this day and age.

Regards,
Andrew Eddie

[Andrew Eddie]

On 14 December 2013 09:07, Grigor Mihov <> wrote:
> Joomla! should have one stable version that is
> recommended by the project.

I think you can have two but we need to do a better job at
communicating which one is the best fit for you. Of course, the gets
foggy if we embark on core and custom distributions because then the
"version number" relates to the base architecture. That will be a very
interesting problem to work on though.

Regards,
Andrew Eddie

[Michael Babker]

Someone pointed me to Ubuntu's download page yesterday - http://www.ubuntu.com/download/desktop - That does a pretty good job of describing the versions offered and the pros of each.  And there's no biased statement about which is recommended by the project like we have for 3.2.

[Andrew Eddie]

On 14 December 2013 10:51, Michael Babker <> wrote:
> Someone pointed me to Ubuntu's download page yesterday -
> http://www.ubuntu.com/download/desktop - That does a pretty good job of
> describing the versions offered and the pros of each. And there's no biased
> statement about which is recommended by the project like we have for 3.2.

I like the idea, but I think it's a bit harder for us if we drop the
idea of LTS (a good thing mind you).

Possibly a better way is to hook into the compatibility of the JED. For example:


Joomla 3.3.4 - 4,534 compatible extensions available, 12234 templates
<link to JED filter>
Joomla 4.0.14 - 24 compatible extensions available, 3 templates <links>

Oh dear, I know which one I'm picking.

... one year later

Joomla 3.4.6 - 5,123 compatible extensions available, 1,500 templates
<link to JED filter>
Joomla 4.5.2 - 1,247 compatible extension available, 900 templates <links>

Hrm, now it's starting to get interesting. If you could couple that
with the list of extensions I care about, that would be really
interesting.

Regards,
Andrew Eddie

[Amy Stephen]

Andrew -

I'm always puzzled why you ask when you just blow off my response, anyway (Especially when I had already answered the information -- you still must not believe I can code or that I have any value to offer.)

Oh well.

Go ahead and wait for FIG to define Interfaces. But, no one else is and I sure am not. I'll use them when they are available, but, until then, I see a real value in using Interfaces as a cushion between concrete classes and as the basis of an API.

[Andrew Eddie]

On 14 December 2013 11:45, Amy Stephen <> wrote:
> I'm always puzzled why you ask when you just blow off my response, anyway
> (Especially when I had already answered the information -- you still must
> not believe I can code or that I have any value to offer.)

I didn't intend to blow you off (you know me, I just wouldn't answer
if I wanted to blow you off). The reality is WordPress is a much
simpler beast than our codebase, mainly because it's procedural. I'm
sorry that you are offended that I can't take the comparison
seriously. Yes, I'm an OOP snob. I'm also the longest standing
developer here today so I think I should know a little about what we
define as the "public API". I shudder at the thought of the work that
would be required to segment parts of the API as trusted and not
trusted (boggles my mind). The only possible exception is layouts but
that's exactly why I and a few others invented the layout override
system, because they are brittle. I cannot emphasise enough how
foreign the concept of "this is API, this is not API" is to me, but it
is what it is.

> Go ahead and wait for FIG to define Interfaces. But, no one else is and I
> sure am not.

You are right. Interface talk is absent in the CMS as far as I can
tell (though, I could have missed some topics so apologies in
advance). For me what's blocking that discussion is the existing
version and release strategy hence the full frontal assault, then I'm
going to flank with code+tests+docs, then I take prisoners by forcing
hard and fast plans for what gets done in 2014. Now you know all my
secrets for world domination (you'd think I'd be a lot better at it
after 10 years of trying though - I really suck at world domination,
expect when playing Civilization). We've tried "do what you want" for
long enough. It's time to round the brumbies up.

> I'll use them when they are available, but, until then, I see a
> real value in using Interfaces as a cushion between concrete classes and as
> the basis of an API.

You and I both realise exactly how much work is behind those "when
available" words. I didn't think I had to spell it out for you but as
you took offence it appears I have to. Yes, I agree working out
interface will be a fun yet critically necessary job. Perfection for
me is rebuilding CMS 4 from the ground up and then adding
compatibility layers to 3 and 2.5 and we just let the people that want
to stay in PHP 5.2 land grow old, retire and pass away. But to do that
we have to fix the version strategy first. We just need to do it.

Regards,
Andrew Eddie

[Amy Stephen]

Andrew - I'm sorry, I over-reacted to your line of questioning, allowed it to feel personal. That's my mistake, not yours.

Joomla has a tough road ahead of it. Nick is right in that changes have created a lot of problems for Joomla users over the years. But, to say the architecture is in bad shape is a serious understatement. So, what to do? Radical changes to address the architectural issues will create more problems for users to stay current. Not doing so has other cost -- how long can new features be placed on these crumbling foundations?

I know what I think -- but no one knows for certain which way is best.

What I hope each of you do is consider your core strengths. Are you the right one to be setting the development strategy? Are you the right one to gauge the pulse of the user base? Try to empower one another - and try not to lead in an area you aren't qualified.

Don't think anyone needs to doubt anyone else intentions, though, but some of you are out of your element and that is creating a problem.

We'll see how this sorts and I'll be looking to see if there is a way I can help when the smoke clears.

Go Joomla!

[Andrew Eddie]

On 14 December 2013 16:26, Amy Stephen <> wrote:
> Andrew - I'm sorry, I over-reacted to your line of questioning, allowed it
> to feel personal. That's my mistake, not yours.

No problems. I try really hard these days to just stick to facts :)
Don't always succeed.

> Joomla has a tough road ahead of it. Nick is right in that changes have
> created a lot of problems for Joomla users over the years.

I prefer to look at it as seasons. For example:

* Contribution was once closed, it's now open.
* The development list was once closed, it's now open.
* The JBS was born out of having a lack of direction for development
and this delivered 1.5.
* The current development strategy was born out of a lack of innovate
in 1.5 and that gave us 1.6 to 3.2.
* A revision of the current strategy is needed for us to embark on a new season.

All that is telling me is that we are growing up :)

> architecture is in bad shape is a serious understatement. So, what to do?
> Radical changes to address the architectural issues will create more
> problems for users to stay current. Not doing so has other cost -- how long
> can new features be placed on these crumbling foundations?

I think we have two choices. Either you split the entire project in
half or you allow two versions of the CMS to be worked on: the 3.x
stable track and the 4.x innovation and experimentation track (there
we go, let's call 4.x experimental).

> I know what I think -- but no one knows for certain which way is best.

All I know is the status quo is the least best position to hold.

Regards,
Andrew Eddie

[kisswebdesign]

Thanks Nick,

I also responded in the gist, but as notifications on gists are not working at the moment here is a copy...
====

Thanks Nick,

I do have some ideas around migration/upgrade strategies - and may write another gist to explain my thoughts - but that is for another day.

As for the one version idea, I don't like it for a couple of reasons

• does not conform to semver
• sticks to a time-based release schedule

Semver is a great way to manage versions, with well defined rules regarding major.minor.patch, which go a long way to helping developers, site builders and 3rd party devs understand the scope of the changes...

e.g. A break in the backwards compatibility (of the API, or anything that breaks B/C) triggers a new major version.

Time based releases cause mad rushes to get new features included before they are ready, and if they miss by even 1 day there is a 6 month wait to have it added into the next release. Some do 'make it in' only to cause misery because they were not ready and had been rushed or bullied in because 'its needed, we already announced it and we can't delay the release any more'.

Having evolving releases that add features when they are ready gives a better product to the customers (builders, 3rd party devs, admins, etc). The features will be more stable, fully assessed, tested and ready for production.

Having a 2 year overlap in the releases gives plenty of time to plan migrations - and have fully supported major versions running in parallel, so nobody feels they must upgrade because their current version is EOL after only a couple of years.

===

Regards,

Chris.

On Friday, 13 December 2013 06:20:12 UTC, Nick Savov wrote:

@Chris, great doc! I commented in the gist.

Kind regards,
Nick

[Nick Savov]

It wasn't meant to be written in "policy-ese". It's meant to be in plain,
simple to understand English. If we can't agree on basic principles like
"an easy upgrade path", there's no point in talking "policy-ese".

I'm willing to compromise on everything, except an easy upgrade path.
We've made the promise as a project already. Yes, it's a hard promise to
keep, but we need to keep it. So, I'll ask the question again, are we

going to guarantee an easy upgrade path or not?

In my first post, I already mentioned how we can improve upgradability.
Let's use the API example.

1) Define a developer API that Joomla agrees to support for developers to
use. This is our contract between Joomla and developers for API that we�re
agreeing will be available to them and stable.
2) The developer API should be clearly stated and documented.
3) The developer API should be 100% unit tested

So let's say a PR comes in for a bug fix or new feature. If the PR breaks
the unit testing, don't merge.

If you do the above, that would make upgrading easier for developers and
it's a big step in the right direction.

By the way, specifying it at the technical level is a whole other
conversation in itself. If you haven't noticed, the Joomla development
strategy (http://developer.joomla.org/cms/development-strategy.html)
contains subjective language throughout it. It's a not a quantifiable
technical specifications doc. For example, take a look at all the mentions
of "stable" (it even uses "very stable") and "reliable", yet those aren't
quantified. So rather than critiquing my comments with one standard, while
not holding the same standard for the current dev strategy or your own
gist (which has the same issues), I'd suggest you use put your critical
analysis on your gist if that's what you're after. For what it's worth,
you're going to have very hard time getting rid of subjective language in
your gist and make everything "technical". For example, you have "VERY
easy to perform" in the security section. By your own standard, "that
means squat" :P

Sincerely,
Nick

[Andrew Eddie]

Nick, if you aren't willing to listen to the advice on how to improve
your proposal, there is no point in taking this discussion any
further.

Your proposal gets a -1 from me.

I hope others will follow this lead so we can move on.

Regards,
Andrew Eddie

[Michael Babker]

On Sat, Dec 14, 2013 at 12:52 PM, Nick Savov <ni...@iowawebcompany.com> wrote:

I'm willing to compromise on everything, except an easy upgrade path.
We've made the promise as a project already. Yes, it's a hard promise to
keep, but we need to keep it. So, I'll ask the question again, are we

going to guarantee an easy upgrade path or not?

Nobody is seeking to purposefully make it the most difficult thing in the world to upgrade.  Similarly, if we make a strict promise that upgrading will always be easy (what defines easy anyways?), we risk tying our hands and not being able to innovate for the sake of easy.  And the fact is, until we know what we're looking at in terms of an end product, every discussion about upgrade paths is just words without action.  Until someone starts plugging the Framework stack into the CMS and removing the Platform equivalents, we won't know with 100% certainty whether that process will be measured as easy, medium, or hard.  If it's medium difficulty, can we do it under your proposal?

In my first post, I already mentioned how we can improve upgradability.

Let's use the API example.

1) Define a developer API that Joomla agrees to support for developers to

use. This is our contract between Joomla and developers for API that we’re
agreeing will be available to them and stable.

2) The developer API should be clearly stated and documented.
3) The developer API should be 100% unit tested

And thus far you've gotten at least three answers to the definition.  All of them clear as mud.  Do any of those answers (Andrew's "everything that's public/protected in the full stack", my "everything in the libraries folder" answer, or Amy's "these projects say this is the API but these other methods are absolutely not") fall in line with your vision?  If not, what is your vision of the API?

How far do we go then to document this API?  Is api.joomla.org adaquete for what you would consider "clearly stated and documented" or do you require additional details?  For what it's worth, the current version of the API site and what it presents for the CMS matches how 1.5 (http://api.joomla.org/1.5/) is documented; the libraries folders being documented as the API (except I didn't include third party stuff in the rendering this time).

For the unit testing piece, this has been a work in progress for well over a year.  Quite frankly, much of our legacy code is not in a state where it can be unit tested.  Our goal on the Framework is 80% plus coverage on all new code; this is a viable goal for most of the libraries IMO; there are pieces like cache and session that need a lot of work to be testable right now.

So let's say a PR comes in for a bug fix or new feature. If the PR breaks
the unit testing, don't merge.

Current standing practice that has been in place for at least two years with the Platform and Framework and for the better part of the last year with the CMS.

If you do the above, that would make upgrading easier for developers and
it's a big step in the right direction.

Are we not progressing in this direction?  We've taken a lot of steps to improve internal and external procedures and raise the bar for contributions (from a quality aspect, not quantity).  As I have to be constantly reminded of, a lot of things in the CMS move slower than some would like in order to maintain things like backwards compatibility and not complicating things for community members and contributors.

[Amy Stephen]

On Saturday, December 14, 2013 12:52:35 PM UTC-6, Nick Savov wrote:

 If we can't agree on basic principles like
"an easy upgrade path", there's no point in talking "policy-ese".

I'm willing to compromise on everything, except an easy upgrade path.
We've made the promise as a project already. Yes, it's a hard promise to
keep, but we need to keep it. So, I'll ask the question again, are we
going to guarantee an easy upgrade path or not?

IMO on "what is an easy upgrade path":

The project should adopt as part of the development strategy any *core data* migration requirements needed to move data from one version to the next version be a requirement of the release. (In fact, I believe the project has put a lot of effort into ensuring such since the 1.6 release).

In terms of extensions, documentation and developer support for migrating extensions, the development strategy should clarify that it's up to the community of developers to step forward, organize, make the plans and assignments, in order to create and share those solutions. In this area, the project responsibility is to provide a place to communicate and a location to link to available solutions.

+++++

Nick - one point on the word "promise." There are no promises. There are just people, some of whom volunteer and give quite a bit of themselves, yourself included, and others who use the software and market solutions which indirectly benefit the ecosystem but never test a patch or report a problem. None of these people are better or worse than the other, no one is doing it right or wrong. But, there are no promises. Just enough crazy people who are, for whatever undiagnosed reason, willing to give generously of their free time to keep the ball moving forward. Anyone who starts to expect anything in this arrangement needs to ask themselves why they are so entitled. It's fine for the PLT to set high standards for itself. It is immoral (IMO) for anyone to feel entitled to anything, no matter what is said, NOTHING is promised. (Those are ramblings intended to help you understand, you are simply not responsible for those who freely choose to use Joomla, they are still responsible for themselves. Freedom is like that, wouldn't have it any other way.). (hugs for you.)

[Amy Stephen]

To add -- paying people to do work changes the equation.

If the community wants promises, then people must be paid.

[Leo Lammerink]

"People Must paid" :: We have had that in the past already and that is not on the plate imho nor desired at all so let's not go and discuss that again shall we?

Once again Nick, you have been asked by many people to listen and you are stubborn holding on to something that is not desired as very clearly stated by a majority of very mature people here. Listening is a virtue!

I follow Andrew's (and Michael's) Lead and your proposal is off for me (-1) as well

Leo

On 12/15/2013 7:57 AM, Amy Stephen wrote:

To add -- paying people to do work changes the equation.

If the community wants promises, then people must be paid.

[Michael Babker]

On Sat, Dec 14, 2013 at 8:59 PM, Leo Lammerink <gws...@gmail.com> wrote:

"People Must paid" :: We have had that in the past already and that is not on the plate imho nor desired at all so let's not go and discuss that again shall we?

Maybe how it was done the last time left a lot of people with mixed feelings about the topic, but if you take a look at the thread where potential 2014 goals were discussed (https://groups.google.com/d/topic/joomla-dev-general/p55iS6JddFU/discussion) you'll find the support for paid staff is surprisingly higher than you might think.  And as someone who's volunteered nearly full-time work hours to support the project over the last few months, I know I'd personally appreciate someone who's working full time to share the workload with. 

[Leo Lammerink]

"how it was done" is exactly the right wording Michael. Thanks for the link... some more to read and digest

[Nick Savov]

Hi Chris,

Ah, OK, I didn't know notifications weren't working on there until after I
posted (Andrew mentioned it). I guess we could just chat here unless you
have a specific thread for yours (let me know if you do).

Great thoughts!

Having "one active version" can work very well with semantic versioning,
as defined at http://semver.org/. There's nothing within semantic
versioning that prevents that.

For example, let's say you deprecate API's that are outdated, have
replacements, and are being used by less than 2% of your user base. That
requires you to release a new minor version with those deprecations in
place. Then you can continue development releasing minors as they happen.
Then, whenever your community is ready to drop those API's that you
deprecated, you release a new major version (you can also include any new
features that you have and bug fixes). Your old minor version would still
function for the user, but you'd keep going on with the "one active
version" that the community is focusing on development and maintenance.

Sorry, but non-time-based-releases do not have claim to the "when it's
ready" title. Time-based-release can and should, IMO, only include code
"when it's ready" too. We should never include code that's not ready. If
the code's not ready, it can wait for the next train. It's the reviewers'
responsibility to make sure that happens and they shouldn't be rushed
either. I think the timeframes I gave for alpha and beta in my original
post in this thread give the reviewers plenty of breathing space to
properly review the code.

Also, if we wanted to compromise in this area, we could say that we can
release new features at least once every 6 months (if there's something
even submitted). So let's say for example, a new minor version is
released, but one key feature doesn't get in, but is almost there. We
could then do a release 2 months later to give the developer(s) enough
time, hope, and encouragement.

For another perspective on time-based releases, check out:
https://wiki.ubuntu.com/TimeBasedReleases

Two potentially big issues with feature-based releases are scope creep and
volunteer availability, especially as more time elapses. Drupal's going
through that now with Drupal 8 and the projections keep being pushed back.
Looks like it might end up taking them 4+ years.

Joomla suffered it back with 1.5 to 1.6:
http://community.joomla.org/blogs/leadership/1055-joomla-projecting.html
http://community.joomla.org/blogs/leadership/1395-the-path-forward-migration-and-the-future.html
http://community.joomla.org/blogs/leadership/1407-why-developers-should-upgrade-sooner-than-later.html

My 2 cents.

Kind regards,
Nick

>
> Thanks Nick,
>
> I also responded in the gist, but as notifications on gists are not
> working
> at the moment here is a copy...
> ====
>
> Thanks Nick,
>
> I do have some ideas around migration/upgrade strategies - and may write
> another gist to explain my thoughts - but that is for another day.
>
> As for the one version idea, I don't like it for a couple of reasons
>

> - does not conform to semver
> - sticks to a time-based release schedule

>
> Semver is a great way to manage versions, with well defined rules
> regarding

> *major.minor.patch*, which go a long way to helping developers, site

[Nick Savov]

The issue is that "upgradability" shouldn't be an after-thought once the
"features" are done or about done. It should be the cornerstone. Google
Chrome got it right with their auto-updater and making it the cornerstone
from the start.

As for API, I'd also say the closest thing we have to it is our libraries,
but it doesn't matter much unless it's declared and clearly communicated
by the project. That's where
https://groups.google.com/forum/#!topic/joomla-dev-cms/YhPvWTJS4go
hopefully comes into play.

Kind regards,
Nick

[Michael Babker]

On Sat, Dec 14, 2013 at 9:46 PM, Nick Savov <ni...@iowawebcompany.com> wrote:

Sorry, but non-time-based-releases do not have claim to the "when it's
ready" title.  Time-based-release can and should, IMO, only include code
"when it's ready" too. We should never include code that's not ready. If
the code's not ready, it can wait for the next train. It's the reviewers'
responsibility to make sure that happens and they shouldn't be rushed
either. I think the timeframes I gave for alpha and beta in my original
post in this thread give the reviewers plenty of breathing space to
properly review the code.

As one of the main people reviewing code and preparing 3.2 at the leadership level, my opinion is that time based releases are becoming harder to sustain because of the strict schedule it is in theory supposed to lead to.  We delayed 3.2 nearly two months because of an earlier promise that 3.2 would be the last feature release of the 3.x series, leading to a very large influx of proposals, tied in with a very active summer (which was partially influenced by the feature announcement previously) and doing what we could to accommodate the GSoC students.  In reality, 3.2 would not have been half of what it was if we went by our original timeline or your proposal and feature froze in July.  Which 3.2 features were actually ready at that point?  The RAD sprint hadn't occurred, the JUX sprint was the week before that cutoff, the extension installer wasn't written, content history was on the tracker but not in the state it was when finally merged, 2FA wasn't written, etc.  How many of those features would have been written anyways knowing they wouldn't even be merged for a year (the current plan assuming zero changes being that 4.0 is released September 2014)?

I think the reviews for 3.2 code were some of the highest quality this cycle that we've had in my time with the project.  There was no rushing.  The Nashville Code Sprint was aimed at code review and starting work on 3.2 and the weeks that followed continued with a lot of testing and feedback on all of the features.  Even the GSoC reviews this year feel like they were performed better than those from last year.  So, in my mind, if there is any flaw in our timeline it isn't us rushing, it's that we are more willing to delay to ensure everything is done right than what should be allowed on a strict time based schedule.

Lastly on this topic, your timeframes are good in practice, but they also lead to assumptions.  We have *NEVER* started development on a release far enough out to have an alpha release three months before the stable release.  In fact, I seem to recall that we didn't even establish an official git branch for 3.0 until somewhere in the 6-8 week window.  This absolutely must change, and that's where our announcement from September (https://groups.google.com/d/topic/joomla-dev-cms/KrmEVG5Rhvo/discussion) comes into play.  By starting branches for the next release much sooner, we should be able to avoid this last minute rush and start pushing releases much sooner.  It should also help those working on features to test them with other code merged in versus in isolation from the other activity.  There have been a few occasions where features conflict with one another and gets dealt with after the features are merged.

Also, if we wanted to compromise in this area, we could say that we can
release new features at least once every 6 months (if there's something
even submitted). So let's say for example, a new minor version is
released, but one key feature doesn't get in, but is almost there. We
could then do a release 2 months later to give the developer(s) enough
time, hope, and encouragement.

I think we should stop committing ourselves to a timeline for minor releases and just release as practical.  If that's 2 months, so be it.  If it's 12 months, so be it.  What we know works well for us is maintenance releases 6-8 weeks apart, and it sounds like there's a desire for about 4 years lifespan on a release series.  Though the longevity in a constantly evolving internet can potentially be an issue, it surely can be worked with.

Two potentially big issues with feature-based releases are scope creep and
volunteer availability, especially as more time elapses. Drupal's going
through that now with Drupal 8 and the projections keep being pushed back.
Looks like it might end up taking them 4+ years.

To me, part of what will make feature-based releases successful is defining project goals and roadmaps for the products.  We have been very hesitant to make a roadmap for the CMS, and the one that exists now is about half listed with stuff that is expected to happen anyways.  We need to have a vision; this is what we want to see in 4.x, and point contributors to the teams working on those items.  Our current system of people working on what they want, though great for innovation, is equally as painful to monitor.  Until Elin submitted the tracker item for tags, I had no clue she was working on that.  Same for Mark and the content history stuff.  Both great features, but both things that should have been better publicized that they were works in progress.  And if they were and I missed it, then revert to the argument about there being too many communication channels.

A successful feature-based strategy is going to require a high level of code discipline and maintainer teams (emphasis on the plural teams, it cannot continue to be the same singular team always focusing on maintaining 2.5 and 3.x while trying to enable folks to work on 4.x).  The reviewers and PLT will have to be firm on the timelines that are decided and the code decisions that are made.  Note I also mentioned timelines here; feature driven releases aren't explicitly locked into one, but if we dictate our timelines and stand firm to them, we can make the releases work too.  Say we decide 4.0 will be released March 2015 and the feature freeze will be October 2014 for the initial release of that series.  That means that if deprecations aren't removed by then, they live through the series.  If B/C breaking refactoring isn't complete, then if the refactoring occurs during the series, it has to be B/C within the full series.  If features aren't complete, developers keep working on them and a 4.1 branch is established to merge that work into when it is ready.  Assess where we're at and make a call on a timeline.  Is there enough nearly ready to merge that we forecast a new minor release 2-3 months after 4.0?  Or do we put the pin on the calendar for 6-7 months out?  Can we release sooner if we think we're ready or do we have to stay with that initial timeline?

I think some of us are getting hung up in overall concepts.  If you break things down to the level of detail I have here and you see all the moving parts that have to come together to make the proposals work, what I think we will find is that while there will be some things that change, there will also be things that stay the same.

[Michael Babker]

On Sat, Dec 14, 2013 at 10:16 PM, Nick Savov <ni...@iowawebcompany.com> wrote:

The issue is that "upgradability" shouldn't be an after-thought once the
"features" are done or about done. It should be the cornerstone. Google
Chrome got it right with their auto-updater and making it the cornerstone
from the start.

Upgradability isn't an after-thought.  Which do you plan first, the changes you want to make and what you have to do to accomplish that, or the upgrade plan and how you're going to get users to a random point on the map with no details about what they're looking for?  This is almost a chicken versus the egg debate.  If you work on the upgrade path while you're developing your feature or working on your architecture changes, IMO, you'll address all of the concerns during the development phase and not when you consider your work feature complete.

As for API, I'd also say the closest thing we have to it is our libraries,
but it doesn't matter much unless it's declared and clearly communicated
by the project. That's where
https://groups.google.com/forum/#!topic/joomla-dev-cms/YhPvWTJS4go
hopefully comes into play.

What's your definition of clearly communicated?  I think having our API published on a site like api.joomla.org should be a pretty clear sign what we consider the API to be.  Or do we get into details and consider something like InstallerHelper::addSubmenu() part of the API?  By Andrew's definition, it is as it's a public method.  By my definition, it isn't as it's not in the libraries (and in all honesty, that method really is only practical in com_installer anyways the way it's written).

[Joomla Forms]

Hi @all,

I don't know what the final outcome of this conversation is so far, but Nick made a good point on the distribution of the different Joomla! versions.

A good part of the people still use J! 1.5 just as legacy CMS and already moved to other CMS for new websites, so the number of real active users might be a lot smaller than the absolute number of Joomla! installations. That does make Joomla! even less popular than everybody is thinking. 

So where this is coming from? From the fact that one can rebuild his site entirely on major updates, and this is work that regular users - for understandable reasons - would like to prevent.

There is no such thing like compatibility layers since 1.5 any longer -- until then, everything went fine, after the layer went away Joomla!'s popularity decreased constantly.

This can only be solved over time, if the current LTS, hammered-in-stone-release-dates and breaking versions get trashed and replaced by a human like approach (which we are still addressing, right?).

"Human" to me would be something simple like this:

 - Releasing new versions as required

 - Making compatibility layers a core part of the project

If there will be a complete turn to such an approach, the PLT / Nick can do a promise for a "robust upgrade path", similar to the one we had with 1.0 => 1.5:

 - The core devs have enough time stabilize the J! 3.x line and and developing the framework without being scared too much about backwards compatibility.

 - PLT and core devs have new room to harmonize the different requirements and speak with one voice 

 - Extension developers will welcome this, because if it keeps going like now, everybody will start to carry its own layer with his extensions or the J! api won't ever be used at all

 - Every user, especially the ones from 1.5 times, associate that approach positively as it gives them back the safety that they had at that time, while new users will experience the same over time. 

All in all there will be a lot of pressure removed, vertically through all levels of the joomla community, so I would really welcome if a decision that goes into a direction like this would be made soon, for example as nice Christmas present to the community ;-)

I am positive you guys will come up with something clever and I'll wish everybody a Merry Christmas!

Regards,

Markus

[Bakual]

If there will be a complete turn to such an approach, the PLT / Nick can do a promise for a "robust upgrade path", similar to the one we had with 1.0 => 1.5:

Actually, I like the approach we used between 2.5 and 3.0 better. For those wo don't know:

• 1.0 => 1.5 used a "legacy" plugin which made it possible to use 1.0 components in Joomla 1.5.
  
• 2.5 => 3.0 backported the important changes from 3.0 to 2.5.5 which made it possible to use 3.0 components in 2.5.

For the user, both will work in the end. The difference is that 1.5 had to drag along compatibility with 1.0, but on the other side users were able to update early. The other approach we used for 3.0 has the advantage that 3.0 doesn't have to include that much legacy code, but it means that users have to wait till the extensions are updated.

What we surely don't want to repeat is the migration we have between 1.5 and 1.6/2.5. That one was just aweful and is the real reason why users leave Joomla. We will have to make sure upgrades between major versions go as painless as possible, including 3rd party extensions and maybe even templates in the plan.

[Joomla Forms]

Hi Bakual,

of course that would do the trick as well. The advantage with the compatibility layer has some advantages when it comes to 3rd party extensions which is important to users as well (especially with templates).

Regards,

Markus

[Bakual]

As a 3rd party extension developer I prefer the variant we used for 3.0. It allows me to go forward and support 3.0 and still offer the exact same version for 2.5 users. I only had to raise the minimum version to 2.5.5.

Of course the legacy compatibility plugin we had in 1.5 plays better with 3rd party developers who don't update their extensions as soon as the new Joomla version is out.

[Joomla Forms]

Hi Bakual,

"Of course the legacy compatibility plugin we had in 1.5 plays better with 3rd party developers who don't update their extensions as soon as the new Joomla version is out."

That's exactly what I had in mind with a compat. layer. If such a layer would only cover backwards compatibility to the previous version, the responsibility would shift clearly for everybody to the 3rd party developer on the next version. In the JED we had these badges once that stated if the legacy layer is in-use or not, so it would be clear for everybody that in that cases not joomla but that particular extension dev is responsible.

However, I think both strategies can work well if one of these make it into the development agenda.