Re: [jcms] Extend com_content with a new trigger onBeforeQueryContent

[Niels Braczek]

Am 08.04.2013 15:13, schrieb Pierre-André Vullioud - inetis.ch:

> I added a trigger before the query to be able to manipulate it.
> With this trigger, you can add new table, add new field etc..
>
> Here is the code on Github : https://github.com/pvullioud/joomla-cms/commit/47f8f3264a1bbd42cba3fff9627ae6db1c8a2dff
>
> Do you think that it could be a way to improve/extend com_content ?

Generally a valuable approach, but the wrong place.

A generic onBeforeQuery(JDatabaseQuery $query, string $context) in
JDatabase will do the same, plus offer much more opportunities.

Regards,
Niels

--
| http://barcamp-wk.de · 3. Barcamp Westküste · Frühjahr 2014 |
| http://www.bsds.de · BSDS Braczek Software- und DatenSysteme |
| Webdesign · Webhosting · e-Commerce · Joomla! Content Management |
------------------------------------------------------------------

[Yves-K]

On Monday, 8 April 2013 15:13:07 UTC+2, Pierre-André Vullioud - inetis.ch wrote:
> Hi,
>
> I had a discussion at the French J!Day with Yves Koekkoek : https://twitter.com/YvesKoekkoek
>
>
>
> He had an idea about how to extend easily the com_content component.

>
>
>
> I added a trigger before the query to be able to manipulate it.
>
> With this trigger, you can add new table, add new field etc..
>
>
>
> Here is the code on Github : https://github.com/pvullioud/joomla-cms/commit/47f8f3264a1bbd42cba3fff9627ae6db1c8a2dff
>
>
>
> Do you think that it could be a way to improve/extend com_content ?
>
>
>

> Thanks for your feedbacks
>
>
>
> Pierre-André Vullioud
>
>
>
> -----
>
> www.inetis.ch

You can also use it to increase query speed removing unused columns !

The more index and data records can fit into a single block of memory the faster the query will be ;-)

[Matt Thomas]

+1 This is an awesome feature in K2, and would be great to do this kind of thing in core.

Best,

Matt Thomas Founder betweenbrain™ Lead Developer Construct Template Development Framework Phone: 203.632.9322 Twitter: @betweenbrain Github: https://github.com/betweenbrain

--
You received this message because you are subscribed to the Google Groups "Joomla! CMS Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-cm...@googlegroups.com.
To post to this group, send an email to joomla-...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-cms?hl=en-GB.
For more options, visit https://groups.google.com/groups/opt_out.

[Niels Braczek]

Am 08.04.2013 20:05, schrieb Yves-K:

> On Monday, 8 April 2013 19:38:49 UTC+2, Niels Braczek wrote:

>> A generic onBeforeQuery(JDatabaseQuery $query, string $context) in
>> JDatabase will do the same, plus offer much more opportunities.

> Its a "global" idea but each module on the page will trigger this!
> Could create some performance problems ?

No. Each and every content plugin is working this way. The $context
tells an actual plugin, if it should jump in or not.

> The main idea is to manipulate actual component query and nothing else.

But why restrict it to com_content? It is a good extension to any
component, eg. com_users.

> Do you think it gives a new potential for any view (com_content in this case) ?

Yes, I think so. Having this in a central place (JDatabase) allows to
extend any, even 3PD, extensions that access the database.

[piotr_cz]

+1,
This would make extensions (components) more modular: Adding fields
for items in core components.

On Apr 8, 11:27 pm, Matt Thomas <m...@betweenbrain.com> wrote:
> +1 This is an awesome feature in K2, and would be great to do this kind of
> thing in core.
>
> Best,
>
> Matt Thomas Founder betweenbrain™ Lead Developer Construct Template
> Development Framework Phone: 203.632.9322 Twitter: @betweenbrain Github:https://github.com/betweenbrain
> On Apr 8, 2013 2:31 PM, "Yves-K" <yk2...@gmail.com> wrote:
>
>
>
>
>
>
>
> > On Monday, 8 April 2013 15:13:07 UTC+2, Pierre-André Vullioud - inetis.ch wrote:
> > > Hi,
>
> > > I had a discussion at the French J!Day with Yves Koekkoek :
> >https://twitter.com/YvesKoekkoek
>
> > > He had an idea about how to extend easily the com_content component.
>
> > > I added a trigger before the query to be able to manipulate it.
>
> > > With this trigger, you can add new table, add new field etc..
>
> > > Here is the code on Github :

> >https://github.com/pvullioud/joomla-cms/commit/47f8f3264a1bbd42cba3ff...

[Fedir]

sounds great, 

but maybe also will need also something like onAfterQuery, or onQueryResult ? for make it comleted

Понеділок, 8 квітня 2013 р. 20:38:49 UTC+3 користувач Niels Braczek написав:

[George Wilson]

+1 for the idea

Just thinking ahead though - if we do go in and edit JDatabase somewhere - it's probably a good idea to keep a record of it somewhere so that when the day arrives that we start moving over to the Joomla Framework from the platform that we have a log of the changes we're going to need to make to it!!

Kind Regards,

George

[Pierre-André Vullioud - inetis.ch]

Thanks for the good feedback.

I was thinking first adding a onBeforeQuery on JDatabase but this could affect performances and can create may be side effects ?

Any thougth ?

This solution could be great especially with category view. We can extend the data into the initial query.This will be much more efficient than running a trigger for each row.

For the next step, I'll continue to add the other triggers onBeforeQueryContent : category, archives etc... to have a complete overview of the modifications.

[Nils Rückmann]

+1 for context driven events
Just add an additional property "context", which can be delivered on creating a new query. So it's even possible to bypass the events by just setting no context (Maybe there are queries which aren't supposed to be hookable).

[Niels Braczek]

Am 09.04.2013 17:53, schrieb Pierre-André Vullioud - inetis.ch:

> I was thinking first adding a onBeforeQuery on JDatabase but this could
> affect performances and can create may be side effects ?
> Any thougth ?

I answered this already 15 hours ago:

>> > Its a "global" idea but each module on the page will trigger this!
>> > Could create some performance problems ?
>
> No. Each and every content plugin is working this way. The $context
> tells an actual plugin, if it should jump in or not.

The performance impact will be measured in microseconds, so it does not
matter.

[Mark Dexter]

Should we consider any other database-level events, such as an event
after a successful database insert or update? Mark

[Nils Rückmann]

Could be done with context. E.g. "content.insert.article"

[Niels Braczek]

Am 09.04.2013 18:19, schrieb Mark Dexter:

> Should we consider any other database-level events, such as an event
> after a successful database insert or update? Mark

I'd like to see two more events:

- onQueryError(JDatabaseQuery $query, int $errno, string $context):
This event will enable a plugin to react on problems and fx. to
deliver alternative data.

- onAfterQuery(Array $resultSet, string $context):
triggered on a successful query. At the first sight for symmetry
reasons

[Mark Dexter]

Makes good sense to me. What do others think? Mark

[Matt Thomas]

+1

Best,

Matt Thomas

Founder betweenbrain™

Lead Developer Construct Template Development Framework

Phone: 203.632.9322
Twitter: @betweenbrain

Github: https://github.com/betweenbrain

[Andrew Eddie]

On Tuesday, 9 April 2013 03:38:49 UTC+10, Niels Braczek wrote:

Generally a valuable approach, but the wrong place.

Agreed. It prevents you from extending getItems.

 

A generic onBeforeQuery(JDatabaseQuery $query, string $context) in
JDatabase will do the same, plus offer much more opportunities.

JDatabase is a not a good place to tightly couple to an event system. Having to inspect each query would be a very nasty performance hit.

You'd be better off designing a com_content2 from scratch to solve this type of problem. I'll have some new mappers available in the next few months that might be able to help in this regard but I can tell you now, there's no easy solution and I've already spent many months working on this problem (so have K2, Zoo et al).

Regards,

Andrew Eddie

[Yves-K]

Designing a com_content2 (or whatever!) is a fork in Joomla!'s heart.

Just adding onBeforeQuery in com_content's (or user or contact...) parts might be the good deal

Could increase performance (removing unused data from large queries) 

[Nils Rückmann]

Short summary of my idea for contect driven events:

1) Extend JDatabaseQuery with a property "context"
2) Extend JDatabase::getQuery() and JDatabaseQuery::_construct() to deliver the context
3) Triggering events on JDatabase::execute() if a context is set.

That will give us the most flexibility, but with the ability to skip the events (if no context is present).

 And if we really don't want to hook to JDatabase, let's use JTable and JModel, as they should be the only
classes which are dealing with queries.

[Michael Babker]

Actually, there's a lot of places that handle database queries without being instances of JModel or JTable.  Just off the top of my head, the Smart Search indexer (the plugin adapter is a JPlugin instance, everything else is top level classes), various JHtml helpers, some JFormField instances, and many components and modules have helper classes that perform queries.

[Nils Rückmann]

Like i said they SHOULD be the only classes ;) Of course, today that's not always possible, but we could improve it with the new MVC. Like the topic "getting rid of helpers".

[George Wilson]

@Andrew what kind of performance hit do you think adding these into JDatabase would give?

I saw microseconds mentioned earlier. You presumably think longer?

Kind Regards,

George

[Fedir]

what if combine onQueryError and onAfterQuery events in to the on, something like:

onAfterQuery(Array $resultSet, string $context, $isError): 

or
onAfterQuery(object $db, string $context): 

or something similar, with possibility to check whether was the error...

then we can simple check whether was error, and decide whether need to continue the data processing...

and it should be more flexible

Вівторок, 9 квітня 2013 р. 20:41:33 UTC+3 користувач Niels Braczek написав:

[Niels Braczek]

Am 10.04.2013 15:12, schrieb Fedir:

> what if combine onQueryError and onAfterQuery events in to the on,
> something like:
> onAfterQuery(Array $resultSet, string $context, $isError):
> or
> onAfterQuery(object $db, string $context):
> or something similar, with possibility to check whether was the error...

It is easier to handle them separately. If the two events were combined
into one handler, you had to add avoidable complexity¹ to it, thus
having negative impact on maintainability and testabilty.

[1] http://en.wikipedia.org/wiki/Software_complexity

> then we can simple check whether was error, and decide whether need to
> continue the data processing...
> and it should be more flexible

I don't see any benefit regarding flexibility by combining two events
into one.

[Andrew Eddie]

On 10 April 2013 20:50, George Wilson <georgeja...@googlemail.com> wrote:

@Andrew what kind of performance hit do you think adding these into JDatabase would give?

I saw microseconds mentioned earlier. You presumably think longer?

Well, you are solving the wrong problem in the wrong place for a start. But now you'll have every query (when you are probably only looking for one or two), through the plugin event system. We already know that the more articles you have on the page, the slower the page load gets because each article needs to be run through several plugins. Now you are wanting to do that for every query?

The approach is just wrong on so many architectural levels. You should not be letting developers mess with queries in this way. You need to give them an API to use to register ways to change how the data is retrieved in the models. Basically you want to create a mini-ORM to handle it.

Regards,

Andrew Eddie

[Andrew Eddie]

On 10 April 2013 17:32, Yves-K <yk2...@gmail.com> wrote:

Designing a com_content2 (or whatever!) is a fork in Joomla!'s heart.

Not really. It's just code and Joomla is the summation of the contributions it receives from the community. The point is that you should solve the problem in com_content, or a successor rather than penalising every query. It's bad design.

 

Just adding onBeforeQuery in com_content's (or user or contact...) parts might be the good deal

You will just ending patching and patching and patching and it will all end in a mess. You are better off thinking about how you would design a system from scratch, and *then* work out if there are ways to massage com_content to get you there. This is what the UCM item on the official roadmap is talking about. Build the schema in the code and then fire the query that is required.

Besides, in order to attach a query plugin, you now have to provide a lot of documentation about every query that could be affected. You also have to make sure all the queries for retrieving a particular object or lists of objects are used consistently, not to mention adding the unit tests to demonstrate it all works safely (and how are you going to reduce the risk that developer injects bad SQL into the query). So, this is not a two-line fix to load a dispatcher and fire an event.

 

Could increase performance (removing unused data from large queries) 

I doubt it. If the queries are slow now then we should fix the query, not let a plugin do it.

The bottom line is though, that the database driver should execute a prepared query. It is not the job of that database driver to call into the air "hello, does anyone want to modify this query". That's wrong.

Regards,

Andrew Eddie 

[Nils Rückmann]

Well, it's not the best architecture, but there shouldn't be any perfomance issues, especially if we have an ability to trigger those events on desire.
I ran 50 common but unique queries each through 30 plugins (with 2 events "onBeforeQuery" and "onAfterQuery") and getting an average of 6 milliseconds by 1000 runs (on a working station).

But let's say we don't want more ugly architectures in Joomla ;) So why not using JModel and JTable for the events ? It would perfectly fit in the "Implementing new MVC"-project (if the project will be started). In this case we should maybe also discuss the behavior/use of JTable, as it is half MVC half DB.

[Andrew Eddie]

On 11 April 2013 08:33, Nils Rückmann <syb...@gmail.com> wrote:

But let's say we don't want more ugly architectures in Joomla ;) So why not using JModel and JTable for the events ?

I wouldn't use events. But I would use a combination of JModel and JTable (acting as a mapper) to support a flexible way for developers to attach relationships - and when you do that, you've got the basic principles for UCM anyway. What I'd do is review how people like FlexiContent, Zoo and K2 (probably others) are solving this problem, and then try and work out if there is a common 'core' that we can build so that all of these extensions are using the same content and same base API, and hopefully the same content.

Yes, it will take more time to do it that way, but that is going to give you the best final result. 

Regards,

Andrew Eddie

[Yves-K]

The risk of bad query architecture or bad query is the risk of any event, trigger or trap you create !

If it does not give good results, i just have to disable or remove the plugin.

Dont you think the real question is not "what will people do ?" but "do we open a door or don't we ?"

I agree when you say we'll get query problems, more than we can imagine but Joomla is strong and solid abnd thoose bad queries will be removed.

But, look on the other side...

Do you think some devlopers will be good enough to create NEW GOOD THINGS with such a door opened.

I do.

Very best regards

yves-k

[Fedir]

I just change my mind :)

now I not think that it good idea, because it just allow to make the more tricky extensions ...

and almost each query of the almost any component already can be overwritten in other places (not so simple, but possible), as someone already wrote ;)

Четвер, 11 квітня 2013 р. 16:23:40 UTC+3 користувач Yves-K написав:

[Matt Thomas]

Hi everyone,

I was wondering if anyone has given any additional thought to this idea? Given some of the recent discussions, I do wonder if having the ability to capture/manipulate queries would be helpful for fine tuning Joomla for different use cases.

Best,

Matt Thomas
@betweenbrain
http://matt-thomas.me/
http://betweenbrain.com/
https://github.com/betweenbrain

Sent from mobile. Please pardon any typos or brevity.

--

[Fedir]

just forget about extend the com_content with query hacking,

I think that the future in UCM + UPM ;)

Субота, 11 січня 2014 р. 17:10:28 UTC+2 користувач Matt Thomas написав:

[Hannes Papenberg]

Hi Matt,
I would propose to use a Decorator Pattern instead of using plugin
events to extend queries. I would assume that you want to do more than
just extend the query when you want to modify a model and thus a
Decorator Pattern would have less overhead from my POV.

If you are interested, I have some code that I'm playing around with
here: https://github.com/Hackwar/joomla-cms/tree/slicker

Feel free to contact me in private to discuss this further and/or
improve on that rough outline of code.

Hannes

Am 11.01.2014 16:10, schrieb Matt Thomas:
>
> Hi everyone,
>
> I was wondering if anyone has given any additional thought to this
> idea? Given some of the recent discussions, I do wonder if having the
> ability to capture/manipulate queries would be helpful for fine tuning
> Joomla for different use cases.
>
> Best,
>
> Matt Thomas
> @betweenbrain
> http://matt-thomas.me/
> http://betweenbrain.com/
> https://github.com/betweenbrain
>
> Sent from mobile. Please pardon any typos or brevity.
>
> On Apr 12, 2013 7:12 AM, "Fedir" <fed...@gmail.com

> <mailto:fed...@gmail.com>> wrote:
>
> I just change my mind :)
> now I not think that it good idea, because it just allow to make
> the more tricky extensions ...
> and almost each query of the almost any component already can be
> overwritten in other places (not so simple, but possible), as
> someone already wrote ;)
>

> О©ҐО©ҐО©ҐО©ҐО©ҐО©Ґ, 11 О©ҐвіО©ҐО©ҐО©Ґ 2013 О©Ґ. 16:23:40 UTC+3 О©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©Ґ Yves-K О©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©Ґ:

>
> The risk of bad query architecture or bad query is the risk of
> any event, trigger or trap you create !
> If it does not give good results, i just have to disable or
> remove the plugin.
> Dont you think the real question is not "what will people do
> ?" but "do we open a door or don't we ?"
> I agree when you say we'll get query problems, more than we
> can imagine but Joomla is strong and solid abnd thoose bad
> queries will be removed.
> But, look on the other side...
> Do you think some devlopers will be good enough to create NEW
> GOOD THINGS with such a door opened.
>
> I do.
>
> Very best regards
> yves-k
>
> --
> You received this message because you are subscribed to the Google
> Groups "Joomla! CMS Development" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to joomla-dev-cm...@googlegroups.com

> <mailto:joomla-dev-cms%2Bunsu...@googlegroups.com>.

> To post to this group, send an email to
> joomla-...@googlegroups.com

> <mailto:joomla-...@googlegroups.com>.

> Visit this group at
> http://groups.google.com/group/joomla-dev-cms?hl=en-GB.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Joomla! CMS Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to joomla-dev-cm...@googlegroups.com.
> To post to this group, send an email to joomla-...@googlegroups.com.

> Visit this group at http://groups.google.com/group/joomla-dev-cms.

[Beat]

Actually, with Joomla 3.2, you can use Observer pattern implemented on JTable for that purpose. Similar with how i refactored tags to do it, and to how the version history does it now too. So the "trigger" on the com_content queries is there already :-)

Best Regards,
Beat
http://www.joomlapolis.com/

[Hannes Papenberg]

Am I missing something? I thought this was about list queries in models
and not about single items in JTable?

Hannes

> > https://github.com/betweenbrain <https://github.com/betweenbrain>

> >
> > Sent from mobile. Please pardon any typos or brevity.
> >

> > On Apr 12, 2013 7:12 AM, "Fedir" <fed...@gmail.com <javascript:>

> <javascript:>
> > <mailto:joomla-dev-cms%2Bunsu...@googlegroups.com
> <javascript:>>.

> > To post to this group, send an email to

> > joomla-...@googlegroups.com <javascript:>
> > <mailto:joomla-...@googlegroups.com <javascript:>>.

> > Visit this group at
> > http://groups.google.com/group/joomla-dev-cms?hl=en-GB

> <http://groups.google.com/group/joomla-dev-cms?hl=en-GB>.

> > For more options, visit
> https://groups.google.com/groups/opt_out

> <https://groups.google.com/groups/opt_out>.

> >
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Joomla! CMS Development" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send

> > an email to joomla-dev-cm...@googlegroups.com <javascript:>.

> > To post to this group, send an email to

> joomla-...@googlegroups.com <javascript:>.

> > Visit this group at
> http://groups.google.com/group/joomla-dev-cms

> <http://groups.google.com/group/joomla-dev-cms>.

> > For more options, visit https://groups.google.com/groups/opt_out

> <https://groups.google.com/groups/opt_out>.
>

[Beat]

On Sunday, January 12, 2014 2:57:10 PM UTC+1, Hannes Papenberg wrote:

Am I missing something? I thought this was about list queries in models
and not about single items in JTable?

Hannes

Hi Hannes,

I don't think that it's a list query: It's a single item:

https://github.com/pvullioud/joomla-cms/commit/47f8f3264a1bbd42cba3fff9627ae6db1c8a2dff#diff-d04a51c7396acc2384a68b7a2b7a3741R167

However your are right in the sense that unfortunately, the model seems to not be using JTable to load the single item, but a query directly, which could be considered as an architecture bug of the already existing code.

Thus, same extensibility effect could be reached by fixing that query loading a StdClass object, that this query could use JTable to load it ? Or did I miss something ?

[Hannes Papenberg]

AFIAK, we wont be able to use JTable in the frontend, because JTable is
a kind of active record implementation. That prevents us from having
nested data like that initial proposal wants to have. But more
importantly for me, the data not only has to be there in the article
model, but also in the articleS model and that means that we have to
modify it there, too, and not only for one row of data, but several.
That would prevent JTable from working there. As I said, I'd like to see
us to look at Decorator Patterns for our MVC classes, since I hope to be
able to then simply state the features that I want to have in my model
class and then decorators take over the actual implementation, while
being specialized on that one task. You could further then implement
plugin events or observers via a decorator instead of hard coding it in
there, which means that every extension has to re-implement that part.

Hannes

[Matt Thomas]

Thank you for the helpful responses and examples. I will study that code. My current thinking is starting to expand beyond com_content, and it seems like using an Observer or Decorator pattern might be viable solutions, although I don't know enough about both, yet, to make a more informed comment.

Really, the basic idea is to provide a way to tap into any queries being run. As Neils described it "A generic onBeforeQuery(JDatabaseQuery $query, string $context) in JDatabase" is the core idea.

For example, I've been doing a lot of work with K2, which does support this idea with a plugin event, and have been making great use of a multiple category plugin for it. That plugin, for example, just modifies the query to add in multiple category support.

I feel that providing developers with the ability to tap into the queries could create some great opportunities.

Best,

Matt Thomas
@betweenbrain
http://matt-thomas.me/
http://betweenbrain.com/
https://github.com/betweenbrain

Sent from mobile. Please pardon any typos or brevity.

[Matt Thomas]

Hannes,

Maybe I better start a new thread to define the context more.

Best,

Matt Thomas
@betweenbrain
http://matt-thomas.me/
http://betweenbrain.com/
https://github.com/betweenbrain

Sent from mobile. Please pardon any typos or brevity.

To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-cm...@googlegroups.com.
To post to this group, send an email to joomla-...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-cms.
For more options, visit https://groups.google.com/groups/opt_out.

[Johan Janssens]

Hi Matt,

Following is an attempt to try and answer your question based on architecture best practices including a conceptual proposal for a solution. Your question :

“Is having the ability to capture and/or manipulate database queries useful and is so how is this best implemented ?”

Two key requirements here, one the ability to capture queries and second the ability to manipulate queries.

I will first try to outline some of the concepts that adhere to this question, before moving on to discussing a potential solution.

Concepts

There is nothing wrong with implementing an Inversion of Control mechanism (1) to make Joomla’s database layer more extendible. Inversion of Control is a key part of what makes a framework different from a library.

Inversion of Control is often referred to as the Hollywood principle (1) : Don't call us, we will call you and is a useful paradigm that assists in the development of code with high cohesion and low coupling that is easier to debug, maintain and test.

Inversion of Control can be implemented using various different patterns (note 1). In Joomla inversion of control is implemented primarily through the plugin system which is event driven and implements a topic based publisher/subscriber pattern. (note 2).

The concept of using a event driven plugin system as implemented in Joomla is also described by Fowler as Event Collaboration (6)

“Multiple components work together by communicating with each other by sending events when their internal state changes.”

Joomla limits event collaboration to collaboration between a component and one or more plugins, where the plugin can acted as a broker or proxy between different components or can work standalone.

Examples

An event driven architecture is not the only way to implement inversion of control. Many different paradigms exist that are implemented in various frameworks to allow for Inversion of Control, to name a few :

• Mix-ins

• Traits

• Dependency injection

• Aspect oriented programming

• Event driven programming

• Intercepting Filters

• Signal Slots

If we take a quick look at modern PHP frameworks we can see that in PHP, Event Driven Architectures are dominant core architectures for most frameworks. :

• Symfony : Event Dispatcher

• Zend : Event Manager

• Typo3/Flow : Aspect oriented approach

Both Symfony and Zend implement an event driven architecture using a topic based, prioritized publisher/subscriber pattern which is similar to what Joomla does, be it that Zend and Symfony event queues are also prioritised and allow for the implementation of an Intercepting Filter (7), which Joomla does not.

Proposal

Events are a great mechanism to extend a core architecture and allow to capture changes and react on them with high cohesion and low coupling. There is one thing they are not good at or intended for : manipulation of the underlying system. (note 3)

A better and more suited pattern to implement a solution that can manipulate the underlying data flow is using Commands and a Chain of Command (CoC) (note 4 and 5).

Following is a conceptual implementation of a Database::update() method that implements a Chain of Command using a prioritised, topic driven, command chain, together with a modifiable command that shows how a command can be used to set the modification date and user id.

  

public function update(KDatabaseQueryUpdate $query)
{
    $context = $this->getContext();
    $context->query     = $query;

    if ($this->getCommandChain()->run('before.update', $context, CONDITION_FALSE) !== false)
    {
        if (!empty($context->query->values))
        {
            //Execute the query
            $context->result = $this->execute($context->query);
            $context->affected = $this->_affected_rows;
 
            $this->getCommandChain()->run('after.update', $context);
        }
        else $context->affected = false;
    }

    return $context->affected;
}

In the above code the actual execution method is being wrapped inside a command chain call, before the execute method is ran a ‘before.update’ command is run, after the execute method has been completed an ‘after.update’ command is run passing the number of affected rows through in the context.

class DatabaseCommandModifiable extends DatabaseCommand
{
    protected function _beforeUpdate(KDatabaseContextInterface $context)
    {
        $query = $context->query;

        if(isset($query->data->modified_by)) {
           $query->data->modified_by = (int) $this->getUser()->getId();
        }

        if(isset($query->data->modified_on)) {
           $query->data->modified_on = gmdate('Y-m-d H:i:s');
        }
    }
}

In the above code a modifiable command invoker will handle the before.update command setting the modified_on and modified_by database columns.

Details

At first sight the above approach is very similar to an event dispatching implementation but the devil is in the details. The difference lies in the fact that the command chain is a prioritised chain of command invokers.

Each invoker is being called by it’s priority and the context is being passed from one to the next allowing each invoker to not only capture the command, but also modify the context before passing it on.

The chain also has a breakpoint condition. In the example the condition is CONDITION_FALSE. If one of the command invokers would return false the chain execution would be halted and the execute method would not be called.

Above example shows how one can implement a inversion of control mechanism that allows to capture, modify and interrupt core application data flow.

Benefits

Joomla can benefit from a command chain approach in each of it’s MVC layers, and also in it’s database layer. This would give developers a uniform approach to be able to extend, modify, interrupt data flow in models, views, controllers and on a database level.

Potential uses can be :

- logging

- caching

- authorisation

- validation

- ...

If a uniform approach is used  documentation can be kept very limited. Developers would be able to entrust that the same implementation works for any extension build on the Joomla core which would further improve code re-use and interoperabulity between extensions.

Criticisms

1. Joomla has been steadily adding hard coded events to it’s core in past versions. There is a danger of introducing the ‘eventitish’ syndrome, eg using hard-coded events as the ‘go to’ solution for implementing Inversion of Control.

Both Drupal and Wordpress suffer from this and have many hundreds if not thousands hooks (a hook is a precedural form of an event mechniasm) in their core. Wordpress 3.6 counts no less then 1556 documented hooks.

A preferred approach to an event architecture would be a system that can generate events on various levels using an uniform event name syntax.

Example : on[Before/After][Component][Name][Type][Action]

• Before/After - All actions have a before/after event

• Component - The name of the component the event belongs to

• Name - The name of the "entity" the event belongs to (e.g. the controller/table)

• Type - The type of the "entity", e.g. Controller,

• Action - The name of the action being run.

For example an event for adding an user would like like this : onAfterUsersUserControllerAdd()

Using this syntax, events can be generated dynamically and registered, without needing to have further documentation.

2. The Joomla’s event system is not always used only for notification purposes, and contains inconsistencies which add to the complexity of implementing events. Some of the events like onContentPrepare are used as data filters, others like onContentBeforeSave are used to be able interrupt the save operation.

Developers should be aware that this is not the intended use of an event architecture and not consider it the default. Other design patterns  exist to solve this problem in a better way as discussed in the proposal above.

3. New features implemented in recent release and features being proposed have a high focus on extending com_content. While com_content indeed makes up a core part of Joomla, it’s only a single component build on top of the framework.

Developers should take care to implement design patterns in the core framework MVC architecture to maintain consistency. Failure to do so, will lead to an increase in code bloat, inconsistencies and further diverging of the codebases and implementations between major extensions which will lead to less interoperability between extensions.

Notes

1. There is some confusion these days over the meaning of inversion of control due to the rise of IoC containers; some people confuse the general principle with the specific styles of inversion of control (such as dependency injection) that these containers use.

2. There is some confusion in among Joomla developers that the Joomla event system uses an Observer pattern (3). This is not correct, Joomla uses a topic based Publish/Subscribe pattern, which is a sibling of a message queue paradigm (5).

- With an observer pattern an object, called the subject, maintains a list of its dependents, called observers, and notifies them automatically of any state changes passing the object itself to the observer. An observer pattern is not message based.

- With a publish–subscribe pattern senders of messages, called publishers, send a message to a dispatcher which will dispatch the message to the interested subscribers.

The message itself is usually a string in a predefined format eg onBeforeQeury, and in most implementation additional data is also passed together with the message.

3. Zend and Symfony allow for manipulation of the event data and underlying system by implementing the intercepting filters pattern(7). Intercepting filters are key to be able to not only capture but also manipulate an event and the targeted object.

At present the Joomla CMS core does not have support for intercepting filters, and only allows to capture, not to manipulate. This is done by design and was a deliberate choice in Joomla 1.5 as the primarily goal of events is to notify, not to modify.

4. There are subtle differences between commands and events that might not seem obvious. Events clearly share most of the properties of commands - the ability to have their own lifetime, to be queued and executed, event reversal is the same as command undo.

Events, however, just communicate that something happened - with an event you let a system know that Y has happened. Another difference is that you think of broadcasting events to everyone who may be interested but sending commands only the a specific receiver. When it comes to actual behavior it doesn't really matter. The polymorphic reaction of a system to an X command need be no different to a Y event. Yet I think the naming does affect how people think about events and what kind of events they create.

Command is a classic pattern described in [Gang of Four]. You should also look at [hohpe-woolf] for the contrast between Command Message and Event Message.

5. The Chain of Command differs slightly from a Chain of Responsability pattern. In a Chain of Command the chain itself is reposible to move the pointer from one to the next command invoker when the chain is executed. In a chain of responsibility, the command invoker itself would call the next command invoker.

6. The Cocoa and Cocoa Touch frameworks, used for OS X and iOS applications respectively, actively use the chain-of-responsibility pattern for handling events.

Conclusion

Joomla can certainly benefit from adding more mechanism to allow for Inversion of Control, the ability to modify and capture queries is just one example where this can be useful. Inversion of Control will lead to increased flexibility and boost future innovation of Joomla as a web platform.

With the introduction of Inversion of Control come risks. The core application is effectively giving away control to extension developers, and with greater control comes greater responsibility. The question how much control you want to allow for is very relevant and should not be dismissed when making a choice on how to implement Inversion of Control in any architecture.

Different design patterns exist to implement Inversion of Control. A solid understanding of the differences, pro and con’s will be vital to be able to create a solid Inversion of Control driven architecture.

Finally there is a lot more that can be said about Inversion of Control, I have tried to capture the essence without going into to much detail.

I hope this helps, happy coding!

Johan

References

1. Inversion of Control : http://martinfowler.com/bliki/InversionOfControl.html

2. Hollywood principle : http://en.wikipedia.org/wiki/Hollywood_principle

3. Observer pattern : http://en.wikipedia.org/wiki/Observer_pattern

4. Publish/Subscribe : http://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern

5. Message queue : http://en.wikipedia.org/wiki/Message_queue

6. Event collaboration : http://martinfowler.com/eaaDev/EventCollaboration.html

7. Intercepting Filters : http://msdn.microsoft.com/en-us/library/ff647251.aspx

Note : This reply can also be found in the following public Google Doc https://docs.google.com/document/d/1MsFW9GLbRv0juyqdbkD5bMBUmkk6ATnwcBklEaETQqE/edit Comments are welcome.

[Matt Thomas]

Hello Johan,

Thank you! This is extremely informative and thought provoking. I appreciate the time that you took to carefully outline this response, not to mention refactoring my question to sound more intelligent.

I can certainly understand the value, and potential risks, of introducing Inversion of Control. It is something that would certainly need to be weighed, especially the aspect of allowing extension developers to modify core behavior. I have personally witnessed the benefits of this in K2, and am very impressed and encouraged by the concept.

I've been aware of the 'eventish' syndrome in Drupal (https://api.drupal.org/api/drupal/7/search/hook_) and hope that we can avoid that, while continuing the evolve Joomla. It was, in fact, one hesitation that I had about the general idea of using one or more plugin event.

I do need to spend some time studying this, and your references, but would like to ask if you wouldn't mind sharing the reasoning behind the design choice in Joomla 1.5 as the primarily goal of events to notify, not to modify. Do you feel that the reasoning behind that decision has since changed and that we, and or, Joomla is ready for Inversion of Control?

Thanks again.

Best,

Matt Thomas

@betweenbrain
http://matt-thomas.me/

http://betweenbrain.com/

https://github.com/betweenbrain

Visit this group at http://groups.google.com/group/joomla-dev-cms.

[Johan Janssens]

Hi Matt,

To answer your question : "What is the reasoning behind the design choice in Joomla 1.5 as the primarily goal of events to notify, not to modify. Do you feel that the reasoning behind that decision has since changed and that we, and or, Joomla is ready for Inversion of Control?"

Separation before Inversion

Back in the days of the big refactoring of Joomla 1.5 the topic of Inversion of Control was not high on the agenda. The focus was almost completely on Separation of Concerns (ref 1) and creating a proper modularised framework (Note 1) together with the introducing of the MVC pattern to help improve and standardise component development best practices.

Inversion of Control is less about Separation of Concerns and more about offering the ability to implement common functionality that spans layers and tiers. This paradigm is also called Cross Cutting Concerns (ref 2) and typically supports operations such as logging, authorisation, caching, error handling, security, validation, internationalization and localization, business logic and workflows, …

Currently Joomla’s framework architecture does not offer a solid solution for developers to implement cross cutting concerns in their extensions. Some cross cutting concerns are being inverted through the use of plugins, like authorisation, authentication, search, editors, … but a standardised framework inversion of control to allow developers to deal more properly with cross cutting concerns is still missing.

From mambots to plugins

The event and plugin system implementation originally found in Joomla 1.5 is a mix and match of a little bit of everything. We never made a deliberate choice to go either way with the system, mostly due to the fact that we didn’t really know how to do it and our hands where tied because we had to offer 100% compatibility with the legacy Mambo mambots implementation. (Note 2). The choices we made for Joomla 1.5 were :

• Modularise the mosMambotHandler into two packages, improving the separation of concerns and making both more SOLID (ref 3)

• Make the procedural mambots implementation the legacy approach and move to an OO approach for the new plugins, a plugin now extends from JPlugin.

• Allow a plugin to handle multiple events.

• Offer 100% compatibility with the legacy mambot system. Mambots should still be installable and work out of the box with legacy mode enabled.

The mambot code was refactored into following two packages :

1. A JEvent package : https://github.com/joomla/joomla-cms/tree/staging/libraries/joomla/event

2. A JPlugin package : https://github.com/joomla/joomla-cms/tree/staging/libraries/cms/plugin

The event package was originally based on a pure observer pattern and extended from there to implement a topic based publisher/subscriber pattern. (Note 3)

Conclusion

The refactored plugin system found in Joomla 1.5 is not pure notification based event system due to the fact that mambots were not pure event listeners either, they did a lot more. A design choice to make the event system a pure notification system was never and could never be made. Most mambots worked as Interception Filters, they received different arguments by reference and allowed to modify these arguments before passing them back.

Joomla already uses and has always used Inversion of Control even back to the Mambo days, without this concept Joomla wouldn't be extendable at all. Joomla's power and focus since it's birth has been to grow an extendable web application platform which has resulted in an eco-system of almost 8000 extensions.

The question is not if we are ready for Inversion of Control, the question to answer next is in what areas will more Inversion of Control be introduced, why, and most importantly how. 

Using the current event system for this is possible, but it would be better to make a clear distinction between using events for notification purposes and additionally implementing a Chain of Command pattern to offer prioritised Interception Filters, and through this offer further Inversion of Control opportunities for extension developers.  

Hope that helps. Happy coding!

Johan

http://about.me/johanjanssens

Notes

1. Mambo didn’t have a framework at all. All the code was part of one big file called mambo.php all the core objects could be found in this file, our first job for Joomla 1.5 was to split this file up in a modularised framework.

2. Mambo had a so called mambot system. Mambots are what we now call plugins and already used a topic based publisher/subscriber pattern. Implementation was a little rough on the edges but it worked.

You can find the code for the mambots system in the old mambo svn branch on joomlacode.  Search for 'mosMambotHandler' to find the actual implementation for the mambots. Mambots where completely procedural, and had a lot of similarities with 'hooks' in Drupal and Wordpress.

3. Since Joomla Platform 12.3 the event package no longer extends from JObserver and JObserverable. This is more correct as the event system is using a a topic based publish subscribe pattern, and not an observer pattern.

References

1. Seperation of concerns : http://en.wikipedia.org/wiki/Separation_of_concerns

2. Cross cutting concerns : http://en.wikipedia.org/wiki/Cross-cutting_concern

3. SOLID : http://en.wikipedia.org/wiki/SOLID_(object-oriented_design)

Note : I have updated the Google Doc acoordingly with the information found in this reply. You can find it here : https://docs.google.com/document/d/1MsFW9GLbRv0juyqdbkD5bMBUmkk6ATnwcBklEaETQqE

[Andrew Eddie]

On Sunday, 12 January 2014, Matt Thomas wrote:

Really, the basic idea is to provide a way to tap into any queries being run. As Neils described it "A generic onBeforeQuery(JDatabaseQuery $query, string $context) in JDatabase" is the core idea.

My opinion remains that events on queries is a bad idea mainly for performance reasons (what's the default install running at with no sample data? 50 queries?). The reality is you are trying to alter one query amongst many dozens that are firing per page load. But more than than, ad hoc altering of a query is a dangerous practice to move forward with (tesability, introducing slow queries, bad joins, changing physical behaviour of the original query, collisions on the same query from two developers, etc and so on). I clearly remember spending hours debugging a crazy problem which turned out to be Joomfish modifying a query in a way it shouldn't. It also raises the question whether queries would have to be documented. The idea of coupling the database package with the dispatcher doesn't sit right with me.

We really need to define "why" developers are wanting to hack the queries, and I'll wager most of that will be due to a lack of a flexible ORM interface. Just throwing a plugin event in a high-use area because we can is folly. My guess is with the current architecture is you'd either add an adapter or a decorator for content loading, or add an event to change the list of content loaded from the standard content model. As mentioned, the long term solution is a UCM approach and I still stand by my earlier comments that it's probably easier to work on a replacement for com_content than try to bandage the existing component.

Regards

Andrew Eddie

--

Regards,
Andrew Eddie
http://learn.theartofjoomla.com - free tutorials and videos on Joomla development