Patchwork: mod_article_category

[Hannes Papenberg]

Hi folks,
our next task is a bit more difficult:

Your job, if you accept, is to create a module that shows the articles
in a category in a list similar to the mod_weblinks module. Please make
sure that you use JQuery and that you have an option to show articles
from child-categories of the chosen category.

If you are interested in doing this task, please post a short reply to
this thread and if you are done, open a tracker item and attach your
patch as a file to it in our Joomla Feature Patch Tracker:
http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemBrowse&tracker_id=8549

Hannes

[Omar Ramos]

I'd like to work on this one...is it ok to add additional features, or is a barebone "Articles in Category Listing" module desired?

--
You received this message because you are subscribed to the Google Groups "Joomla! CMS Development" group.
To post to this group, send an email to joomla-...@googlegroups.com.
To unsubscribe from this group, send email to joomla-dev-cm...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/joomla-dev-cms?hl=en-GB.

[jonnathan]

Little observation: use the contentModelArticles, see:
mod_articles_popular

> patch as a file to it in our Joomla Feature Patch Tracker:http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemBr...
>
> Hannes

[Hannes Papenberg]

At the moment, a barebone module would be better. The faster we get all
this done, the sooner we can start working on plans for 1.7. :-)

Hannes

Am 01.02.2010 17:46, schrieb Omar Ramos:
> I'd like to work on this one...is it ok to add additional features, or
> is a barebone "Articles in Category Listing" module desired?
>
> On Mon, Feb 1, 2010 at 8:19 AM, Hannes Papenberg
> <hack...@googlemail.com <mailto:hack...@googlemail.com>> wrote:
>
> Hi folks,
> our next task is a bit more difficult:
>
> Your job, if you accept, is to create a module that shows the articles
> in a category in a list similar to the mod_weblinks module. Please
> make
> sure that you use JQuery and that you have an option to show articles
> from child-categories of the chosen category.
>
> If you are interested in doing this task, please post a short reply to
> this thread and if you are done, open a tracker item and attach your
> patch as a file to it in our Joomla Feature Patch Tracker:
> http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemBrowse&tracker_id=8549
> <http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemBrowse&tracker_id=8549>
>
> Hannes
>
> --
> You received this message because you are subscribed to the Google
> Groups "Joomla! CMS Development" group.
> To post to this group, send an email to
> joomla-...@googlegroups.com

> <mailto:joomla-...@googlegroups.com>.

> To unsubscribe from this group, send email to
> joomla-dev-cm...@googlegroups.com

> <mailto:joomla-dev-cms%2Bunsu...@googlegroups.com>.

[Mark Dexter]

Hi Omar. What "additional features" did you have in mind?

I respectfully disagree with Hannes on this. I think all of the
article list modules should have some simple, consistent params,
including the following: one or more categories, one or more authors,
one or more author alias values, and from-through date, where date
could be created, modified, or published. Also, we should be able to
exclude articles that are on the front page. We should probably have
consistent ordering options as well.

If you can do this and still have the change done in a short
timeframe, I would say "go for it!". If would take a lot longer, than
I would agree with Hannes and keep the functionality the same as 1.5.
That's my opinion. Mark

On Feb 1, 8:56 am, Hannes Papenberg <hackwa...@googlemail.com> wrote:
> At the moment, a barebone module would be better. The faster we get all
> this done, the sooner we can start working on plans for 1.7. :-)
>
> Hannes
>
> Am 01.02.2010 17:46, schrieb Omar Ramos:
>
> > I'd like to work on this one...is it ok to add additional features, or
> > is a barebone "Articles in Category Listing" module desired?
>
> > On Mon, Feb 1, 2010 at 8:19 AM, Hannes Papenberg
> > <hackwa...@googlemail.com <mailto:hackwa...@googlemail.com>> wrote:
>
> >     Hi folks,
> >     our next task is a bit more difficult:
>
> >     Your job, if you accept, is to create a module that shows the articles
> >     in a category in a list similar to the mod_weblinks module. Please
> >     make
> >     sure that you use JQuery and that you have an option to show articles
> >     from child-categories of the chosen category.
>
> >     If you are interested in doing this task, please post a short reply to
> >     this thread and if you are done, open a tracker item and attach your
> >     patch as a file to it in our Joomla Feature Patch Tracker:

> >    http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemBr...
> >     <http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemBr...>

[Omar Ramos]

I've begun working on this module, but it has brought up a few of its own questions.

1. Looking at the module names for com_content, I seem to remember some discussions about having more consistent naming conventions for these (using the component in the module name)? If this was the case then quite a few modules would need to be renamed:
1. mod_articles_category => mod_content_articles
2. mod_articles_categories => mod_content_categories
   
3. mod_articles_archive => mod_content_archive
4. mod_articles_latest => mod_content_latest
5. mod_articles_popular => mod_content_popular
6. mod_related_items => mod_related_articles
   
7. mod_weblinks => mod_weblinks_links
8. mod_whosonline => mod_users_online
9. mod_banners?
2. How do I get a parameter that allows multiple fields to be selected? I know the capability is in there, I'm just not sure how to make it work via the XML file.
   

I'll be working on the module more tomorrow, and it shouldn't take to put together.

-Omar

[Hannes Papenberg]

Hi Omar,
com_content will be removed in 1.7 and most likely replaced by a
com_articles, so the modules have been renamed accordingly to that
already. The rest will follow in 1.7.

For a multi-select option, just add the attribute multiple="multiple"
into the XML definition. :-)

Hannes

Am 03.02.2010 07:04, schrieb Omar Ramos:
> I've begun working on this module, but it has brought up a few of its
> own questions.
>

> 1. Looking at the module names for com_content, I seem to remember

> some discussions about having more consistent naming conventions
> for these (using the component in the module name)? If this was
> the case then quite a few modules would need to be renamed:

> 1. mod_articles_category => mod_content_articles
> 2. mod_articles_categories => mod_content_categories
> 3. mod_articles_archive => mod_content_archive
> 4. mod_articles_latest => mod_content_latest
> 5. mod_articles_popular => mod_content_popular
> 6. mod_related_items => mod_related_articles
> 7. mod_weblinks => mod_weblinks_links
> 8. mod_whosonline => mod_users_online
> 9. mod_banners?
> 2. How do I get a parameter that allows multiple fields to be

> selected? I know the capability is in there, I'm just not sure
> how to make it work via the XML file.
>
> I'll be working on the module more tomorrow, and it shouldn't take to
> put together.
>
> -Omar
>
> On Mon, Feb 1, 2010 at 5:10 PM, Mark Dexter <dexter...@gmail.com
> <mailto:dexter...@gmail.com>> wrote:
>
> Hi Omar. What "additional features" did you have in mind?
>
> I respectfully disagree with Hannes on this. I think all of the
> article list modules should have some simple, consistent params,
> including the following: one or more categories, one or more authors,
> one or more author alias values, and from-through date, where date
> could be created, modified, or published. Also, we should be able to
> exclude articles that are on the front page. We should probably have
> consistent ordering options as well.
>
> If you can do this and still have the change done in a short
> timeframe, I would say "go for it!". If would take a lot longer, than
> I would agree with Hannes and keep the functionality the same as 1.5.
> That's my opinion. Mark
>
> On Feb 1, 8:56 am, Hannes Papenberg <hackwa...@googlemail.com

> <mailto:hackwa...@googlemail.com>> wrote:
> > At the moment, a barebone module would be better. The faster we
> get all
> > this done, the sooner we can start working on plans for 1.7. :-)
> >
> > Hannes
> >
> > Am 01.02.2010 17:46, schrieb Omar Ramos:
> >
> > > I'd like to work on this one...is it ok to add additional
> features, or
> > > is a barebone "Articles in Category Listing" module desired?
> >
> > > On Mon, Feb 1, 2010 at 8:19 AM, Hannes Papenberg
> > > <hackwa...@googlemail.com <mailto:hackwa...@googlemail.com>

> <mailto:hackwa...@googlemail.com

> > > <mailto:joomla-...@googlegroups.com

> <mailto:joomla-...@googlegroups.com>>.
> > > To unsubscribe from this group, send email to
> > > joomla-dev-cm...@googlegroups.com
> <mailto:joomla-dev-cms%2Bunsu...@googlegroups.com>

> > > <mailto:joomla-dev-cms%2Bunsu...@googlegroups.com
> <mailto:joomla-dev-cms%252Buns...@googlegroups.com>>.

[Omar Ramos]

Hmmm...I had added the multiple="multiple" as the first thing to try and there didn't appear to be any change for me last night. I'm guessing the same goes for the size? (size="6" or whatever I wish to use).

I'll check it out again later on, and thank you for the clarification about the module names.

-Omar

[Omar Ramos]

OK, I was able to get a basic version of this working (just a very basic version that would display articles from one or more categories), but I wanted to use the library in order to fulfill your requirement of "have an option to show articles
from child-categories of the chosen category" so I tried using the example from ContentHelperRoute, which has some code that looks like this:
jimport('joomla.application.categories');
$categoryTree = JCategories::getInstance('com_content');
$category = $categoryTree->get($catid);

But category in this case doesn't get anything populated for its _children variable so I get nothing out using the above code.

I know it had been mentioned the other week that JCategories may not be complete yet, but what should I use to build up the category tree in that case?

-Omar

[Omar Ramos]

It looks like I was going the wrong way about that...I had forgotten about the Categories Content Model...it has an option for getting children that I'm pretty sure I can use.

-Omar

[Omar Ramos]

OK, that seems to be working pretty nicely...I'll begin cleaning it up/polishing tomorrow morning.

Quick question: I seem to remember you mentioning something about dependent parameters at some point? Like if a group of parameters could be disabled depending on the value of another parameter?

If that's available could you let me know how I can use it? Otherwise I'll just go ahead and group things the best I can...which brings up another good idea for a formfield/parameter type: header (as is being used by the JoomlaWorks folks in K2...it's sort of between a spacer and a separate field group as it acts like a spacer, but it allows you to provide a textual heading to visually separate parameters within the same group).

Thank you!

-Omar

[Hannes Papenberg]

This dependency feature hasn't been coded yet. Since I'm a Javascript
noob (I understand JS code, but can't code it myself) I was hoping that
someone else takes this up and changes JForm to support this. :-)

Hannes

> > <mailto:dexter...@gmail.com

> <mailto:joomla-dev-cms%2Bunsu...@googlegroups.com
> <mailto:joomla-dev-cms%252Buns...@googlegroups.com>
> >
> <mailto:joomla-dev-cms%252Buns...@googlegroups.com <mailto:joomla-dev-cms%25252Bun...@googlegroups.com>>>.

[Omar Ramos]

I've pretty much finished this module (I'm just going to test it here locally with a fresh export from SVN and then create my patch for testing).

The module itself required some additional filters to be added to the articles model and some additions to the _getStoreId() methods in the articles and categories models so you can't really test the module on its own (even if I make an installable zip).

It looks I would also need to add an entry within the installation sql file for the #__extensions table as well.

For some of the functionality, available in the module I think it would be a great idea if you could allow me to also integrate two much needed additions for modules in an additional patch:

• Hide if empty parameter (this would be displayed on the Module Edit screen and would default to No...it would be a backwards compatible change and allows the user to hide a module's output completely if they need that functionality for a particular module). This is available within Peter's Advanced Module Manager (http://www.nonumber.nl/extensions/advancedmodulemanager), among other features.
  
• Implement this fix for countMenuChildren that Johan submitted in 2008 which is still applicable today: http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=12858 which corrects the issue of empty modules still counting (this is used a lot by template developers...the issue noticed in this case is that if an empty module is the only module in let's say, the left column, then the template will still see a count of 1, even though the module is empty, and then display an empty left-hand column, when you would really want the template to see a count of 0 so that the template can collapse that left-hand column automatically).
  

The fixes are needed to fix some longstanding issues I've experienced with displaying dynamic modules and would pretty certainly not affect existing installations in a negative way.

[Omar Ramos]

I've attached a mod_articles_category patch here:
http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=19678

NOTE: This patch is only for the module itself, and does not address the two module issues I just mentioned (the Hide If Empty addition and the countMenuChildren Issue).

-Omar

[Omar Ramos]

Darn...should have tried testing again locally some of the functions instead of just seeing that it installed properly here at work.

Looks like there's an issue so if you were thinking of testing this wait a few minutes so I can troubleshoot it and update the patch :-).

-Omar

[Omar Ramos]

Patch has been updated:
http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=19678

There was an issue when the article's catid = 0 and a call to ContentHelperRoute::getCategoryRoute() that needed to be suppressed in that situation, everything else was actually working fine :-).

-Omar

[Mark Dexter]

Hi Omar. I would say go for it on the two additional items you
mentioned: hide if empty and the count issue. Mark

[Omar Ramos]

I'm about to work on those two improvements I mentioned for modules.

Any idea when the mod_articles_category patch might get applied? I think I might have to rely on the additional filters made to the com_content articles/categories models in order to begin working on the separate mod_articles_categories patchwork item. It would also make it easier for others to test out the new module and offer suggestions/tweaks (for example, the language file will probably need to be tweaked, as well as the names for parameters).

-Omar

[Hannes Papenberg]

Hi Omar,
I've looked at your patch and it looks good so far, except for a tiny,
little issue.... I set it up to show all articles from the "Extensions"
category in the sample data and without any limits whatsoever else. This
resulted in a pretty long list and a whooping 121 queries for this
module alone... Can you take a look at this again and fix this up? I
mean, we should be able to have this down to a single query for the
whole module...

This might be a general problem in the content model...

Hannes

> * Hide if empty parameter (this would be displayed on

> the Module Edit screen and would default to No...it
> would be a backwards compatible change and allows
> the user to hide a module's output completely if
> they need that functionality for a particular
> module). This is available within Peter's Advanced
> Module Manager
> (http://www.nonumber.nl/extensions/advancedmodulemanager),
> among other features.

> * Implement this fix for countMenuChildren that Johan

> submitted in 2008 which is still applicable today:
> http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=12858
> <http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=12858>
> which corrects the issue of empty modules still
> counting (this is used a lot by template
> developers...the issue noticed in this case is that
> if an empty module is the only module in let's say,
> the left column, then the template will still see a
> count of 1, even though the module is empty, and
> then display an empty left-hand column, when you
> would really want the template to see a count of 0
> so that the template can collapse that left-hand
> column automatically).
>
> The fixes are needed to fix some longstanding issues I've
> experienced with displaying dynamic modules and would
> pretty certainly not affect existing installations in a
> negative way.
>
>
> On Thu, Feb 4, 2010 at 2:26 AM, Hannes Papenberg
> <hack...@googlemail.com

> > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>>

> wrote:
> >
> > It looks like I was going the wrong way about
> that...I had
> > forgotten about the Categories Content
> Model...it has an option
> > for getting children that I'm pretty sure I can use.
> >
> > -Omar
> >
> >
> > On Wed, Feb 3, 2010 at 9:02 PM, Omar Ramos
> <orw...@gmail.com <mailto:orw...@gmail.com>

> > <mailto:orw...@gmail.com

> > <mailto:orw...@gmail.com

> <mailto:orw...@gmail.com>>> wrote:
> >
> > Hmmm...I had added the
> multiple="multiple" as the first
> > thing to try and there didn't appear to
> be any change for
> > me last night. I'm guessing the same
> goes for the size?
> > (size="6" or whatever I wish to use).
> >
> > I'll check it out again later on, and
> thank you for the
> > clarification about the module names.
> >
> > -Omar
> >
> >
> > On Tue, Feb 2, 2010 at 11:58 PM, Hannes
> Papenberg
> > <hack...@googlemail.com
> <mailto:hack...@googlemail.com>

> > <mailto:hack...@googlemail.com

> <mailto:joomla-dev-cms%252Buns...@googlegroups.com <mailto:joomla-dev-cms%25252Bun...@googlegroups.com>
> <mailto:joomla-dev-cms%25252Bun...@googlegroups.com
> <mailto:joomla-dev-cms%2525252Bu...@googlegroups.com>>>>.

[Omar Ramos]

Let me take a look at it right now...I have an inkling it might be due to the content model (which I tried to use as is), but let me check and get back to you in a few minutes.

-Omar

[Omar Ramos]

I'm counting that this query is being used about about 120 times:
SELECT c.*,CASE WHEN CHAR_LENGTH(c.alias) THEN CONCAT_WS(":", c.id, c.alias) ELSE c.id END as slug,COUNT(i.`id`) AS numitems
  FROM r14742_categories as c
  LEFT JOIN r14742_categories AS s
  ON (s.lft = c.lft
  AND s.rgt = c.rgt) OR (s.parent_id = c.id) OR (c.parent_id = s.id) OR (s.parent_id = c.parent_id) OR (s.lft <= c.lft
  AND s.rgt >= c.rgt) OR (s.lft >= c.lft
  AND s.rgt <= c.rgt)
  LEFT JOIN `r14742_content` AS i
  ON i.`catid` = c.id
  WHERE (c.extension='com_content' OR c.extension='system')
  AND c.access IN (1)
  AND s.id=39
  GROUP BY c.id
  ORDER BY c.lft

(In my test I came out with 181 queries...I switched the category to Extensions as you mentioned, turned the levels from 1  to 10 and then turned the show child category articles option on).

That seems to be forming the bulk of them, though there are some others:

Called ~20 times:
SELECT guest, usertype, client_id
  FROM r14742_session
  WHERE client_id = 0

Called ~20 times:
SELECT a.username, a.time, a.userid, a.usertype, a.client_id
  FROM r14742_session AS a
  WHERE a.userid != 0
  GROUP BY a.userid

[Omar Ramos]

The first query is getting called with JCategories' _load() method which would be related to mod_articles_category,

The other two queries both appear to be called from mod_whosonline.

[Omar Ramos]

We can cut down the number of queries significantly (from 149 queries to 69 on my end after I have disabled the mod_whosonline module) by modifying the mod_articles_category code around line 183 so that the ContentHelperRoute::getCategoryRoute() method only gets run if show_category is enabled:
             if ($item->catid) {
                if ($show_category) {
                    $item->displayCategoryLink = JRoute::_(ContentHelperRoute::getCategoryRoute($item->catid));
                    $item->displayCategoryTitle = $show_category ? '('.'<a href="'.$item->displayCategoryLink.'">'.$item->category_title.'</a>'.')' : '';
                } else {
                    $item->displayCategoryLink = '';
                    $item->displayCategoryTitle = '';
                }
            }

However, I still feel like that call to getCategoryRoute() in general is kind of expensive and could probably use some optimization because we should be able to call it freely without worrying too much about its performance cost I think.

-Omar

[Hannes Papenberg]

Ok, ContentHelperRoute is currently broken. We still have to decide how
this is supposed to work, but so far the *HelperRoute classes are all
broken/obsolete at the moment. The correct ones to use are ContentRoute,
etc.

This reduces the number of queries to 40... That's still way to much for
my taste...

Hannes

Am 10.02.2010 23:12, schrieb Omar Ramos:
> The first query is getting called with JCategories' _load() method
> which would be related to mod_articles_category,
>
> The other two queries both appear to be called from mod_whosonline.
>
> On Wed, Feb 10, 2010 at 2:08 PM, Omar Ramos <orw...@gmail.com
> <mailto:orw...@gmail.com>> wrote:
>
> I'm counting that this query is being used about about 120 times:
> SELECT c.*,CASE WHEN CHAR_LENGTH(c.alias) THEN CONCAT_WS(":", c.id

> <http://c.id>, c.alias) ELSE c.id <http://c.id> END as

> slug,COUNT(i.`id`) AS numitems
> FROM r14742_categories as c
> LEFT JOIN r14742_categories AS s
> ON (s.lft = c.lft

> AND s.rgt = c.rgt) OR (s.parent_id = c.id <http://c.id>) OR
> (c.parent_id = s.id <http://s.id>) OR (s.parent_id = c.parent_id)

> OR (s.lft <= c.lft
> AND s.rgt >= c.rgt) OR (s.lft >= c.lft
> AND s.rgt <= c.rgt)
> LEFT JOIN `r14742_content` AS i

> ON i.`catid` = c.id <http://c.id>

> WHERE (c.extension='com_content' OR c.extension='system')
> AND c.access IN (1)

> AND s.id <http://s.id>=39
> GROUP BY c.id <http://c.id>

> ORDER BY c.lft
>
> (In my test I came out with 181 queries...I switched the category
> to Extensions as you mentioned, turned the levels from 1 to 10
> and then turned the show child category articles option on).
>
> That seems to be forming the bulk of them, though there are some
> others:
>
> Called ~20 times:
> SELECT guest, usertype, client_id
> FROM r14742_session
> WHERE client_id = 0
>
> Called ~20 times:
> SELECT a.username, a.time, a.userid, a.usertype, a.client_id
> FROM r14742_session AS a
> WHERE a.userid != 0
> GROUP BY a.userid
>
>
> On Wed, Feb 10, 2010 at 1:57 PM, Omar Ramos <orw...@gmail.com
> <mailto:orw...@gmail.com>> wrote:
>
> Let me take a look at it right now...I have an inkling it
> might be due to the content model (which I tried to use as
> is), but let me check and get back to you in a few minutes.
>
> -Omar
>
>
> On Wed, Feb 10, 2010 at 1:54 PM, Hannes Papenberg

> <hack...@googlemail.com <mailto:hack...@googlemail.com>>
> wrote:
>
> Hi Omar,

> > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>> wrote:
> >
> > Darn...should have tried testing again locally some
> of the
> > functions instead of just seeing that it installed
> properly here
> > at work.
> >
> > Looks like there's an issue so if you were thinking
> of testing
> > this wait a few minutes so I can troubleshoot it and
> update the
> > patch :-).
> >
> > -Omar
> >
> >
> > On Mon, Feb 8, 2010 at 12:44 PM, Omar Ramos
> <orw...@gmail.com <mailto:orw...@gmail.com>

> > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>>
> wrote:
> >
> > I've attached a mod_articles_category patch here:
> >
> http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=19678
> <http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=19678>
> >
> <http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=19678
> <http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=19678>>
> >
> > NOTE: This patch is only for the module itself,
> and does not
> > address the two module issues I just mentioned
> (the Hide If
> > Empty addition and the countMenuChildren Issue).
> >
> > -Omar
> >
> >
> > On Mon, Feb 8, 2010 at 12:05 PM, Omar Ramos
> <orw...@gmail.com <mailto:orw...@gmail.com>

> <mailto:joomla-dev-cms%25252Bun...@googlegroups.com
> <mailto:joomla-dev-cms%2525252Bu...@googlegroups.com>
> >
> <mailto:joomla-dev-cms%2525252Bu...@googlegroups.com <mailto:joomla-dev-cms%252525252B...@googlegroups.com>>>>>.

[Omar Ramos]

So the ContentHelperRoute calls should instead go to ContentRoute in the router.php file's article() and category() methods.

That reduced the number of queries from 69 to 29 in my case and then changing back the other code that calls getCategoryRoute() didn't change the number of queries on my end either.

Would you like me to update the patch with the calls using ContentRoute?

-Omar

[Omar Ramos]

Too late, already updated in the tracker :-).
http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=19678

[Omar Ramos]

Seems to consistently be showing 29 queries now, even after moving the category up to the root to include even more articles.

Thanks for pointing this out because even if I had tried to optimize the number of queries beforehand I wouldn't have thought of using ContentRoute since I was already familiar with ContentHelperRoute (from Joomla 1.5) and the router.php file only used to include the ContentBuildRoute, ContentParseRoute functions.

Does this help make the patch more worthy now :-).

-Omar

[Hannes Papenberg]

I've discovered why it creates 67 queries on my system when the
mod_whosonline is enabled... There are 19 articles about modules, which
have the loadposition plugin triggered in there, which again have the
position "syndicate" as their position. The content model responsible
for loading the list for the mod_articles_category module executes the
loadposition plugin on each article and that's why we have this enormous
amount of queries. That is not nice... I'm not sure what to do about this...

About the router thing: That is the reason why I'm still not happy with
the routing.

I'll look at the patch again tomorrow, when I have had a bit more sleep. :-)

Hannes

> <hack...@googlemail.com <mailto:hack...@googlemail.com>>

> wrote:
>
> Ok, ContentHelperRoute is currently broken. We still have
> to decide how
> this is supposed to work, but so far the *HelperRoute
> classes are all
> broken/obsolete at the moment. The correct ones to use are
> ContentRoute,
> etc.
>
> This reduces the number of queries to 40... That's still
> way to much for
> my taste...
>
> Hannes
>
> Am 10.02.2010 23:12, schrieb Omar Ramos:
> > The first query is getting called with JCategories'
> _load() method
> > which would be related to mod_articles_category,
> >
> > The other two queries both appear to be called from
> mod_whosonline.
> >
> > On Wed, Feb 10, 2010 at 2:08 PM, Omar Ramos
> <orw...@gmail.com <mailto:orw...@gmail.com>

> > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>> wrote:
> >
> > I'm counting that this query is being used about
> about 120 times:
> > SELECT c.*,CASE WHEN CHAR_LENGTH(c.alias) THEN
> CONCAT_WS(":", c.id <http://c.id>
> > <http://c.id>, c.alias) ELSE c.id <http://c.id>
> <http://c.id> END as
> > slug,COUNT(i.`id`) AS numitems
> > FROM r14742_categories as c
> > LEFT JOIN r14742_categories AS s
> > ON (s.lft = c.lft
> > AND s.rgt = c.rgt) OR (s.parent_id = c.id

> <http://c.id> <http://c.id>) OR
> > (c.parent_id = s.id <http://s.id> <http://s.id>) OR

> (s.parent_id = c.parent_id)
> > OR (s.lft <= c.lft
> > AND s.rgt >= c.rgt) OR (s.lft >= c.lft
> > AND s.rgt <= c.rgt)
> > LEFT JOIN `r14742_content` AS i

> > ON i.`catid` = c.id <http://c.id> <http://c.id>

> > WHERE (c.extension='com_content' OR
> c.extension='system')
> > AND c.access IN (1)

> > AND s.id <http://s.id> <http://s.id>=39
> > GROUP BY c.id <http://c.id> <http://c.id>

> > ORDER BY c.lft
> >
> > (In my test I came out with 181 queries...I switched
> the category
> > to Extensions as you mentioned, turned the levels
> from 1 to 10
> > and then turned the show child category articles
> option on).
> >
> > That seems to be forming the bulk of them, though
> there are some
> > others:
> >
> > Called ~20 times:
> > SELECT guest, usertype, client_id
> > FROM r14742_session
> > WHERE client_id = 0
> >
> > Called ~20 times:
> > SELECT a.username, a.time, a.userid, a.usertype,
> a.client_id
> > FROM r14742_session AS a
> > WHERE a.userid != 0
> > GROUP BY a.userid
> >
> >
> > On Wed, Feb 10, 2010 at 1:57 PM, Omar Ramos
> <orw...@gmail.com <mailto:orw...@gmail.com>

> > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>>
> wrote:
> >
> > Let me take a look at it right now...I have an
> inkling it
> > might be due to the content model (which I tried
> to use as
> > is), but let me check and get back to you in a
> few minutes.
> >
> > -Omar
> >
> >
> > On Wed, Feb 10, 2010 at 1:54 PM, Hannes Papenberg
> > <hack...@googlemail.com
> <mailto:hack...@googlemail.com>

> <mailto:hack...@googlemail.com
> <mailto:hack...@googlemail.com>>>
> > wrote:
> >
> > Hi Omar,

> > > <mailto:orw...@gmail.com
> <mailto:orw...@gmail.com> <mailto:orw...@gmail.com
> <mailto:orw...@gmail.com>>>> wrote:
> > >
> > > Darn...should have tried testing again
> locally some
> > of the
> > > functions instead of just seeing that
> it installed
> > properly here
> > > at work.
> > >
> > > Looks like there's an issue so if you
> were thinking
> > of testing
> > > this wait a few minutes so I can
> troubleshoot it and
> > update the
> > > patch :-).
> > >
> > > -Omar
> > >
> > >
> > > On Mon, Feb 8, 2010 at 12:44 PM, Omar
> Ramos
> > <orw...@gmail.com <mailto:orw...@gmail.com>
> <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>

> <mailto:joomla-dev-cms%2525252Bu...@googlegroups.com <mailto:joomla-dev-cms%252525252B...@googlegroups.com>
> <mailto:joomla-dev-cms%252525252B...@googlegroups.com
> <mailto:joomla-dev-cms%25252525252...@googlegroups.com>>>>>>.

> --
> You received this message because you are subscribed to the Google
> Groups "Joomla! CMS Development" group.

> To post to this group, send an email to joomla-...@googlegroups.com.

> To unsubscribe from this group, send email to

> joomla-dev-cm...@googlegroups.com.

[Omar Ramos]

Sounds like a plan ;-) (the sleep part I mean).

I'll continue to investigate a bit using the new information you've given me as I know I'm running JHtml::_('content.prepare') on the article introtext so maybe I can optimize that part to prevent those additional queries mostly.

-Omar

[Omar Ramos]

Hi Hannes,

I know I've already updated things in my patch to use new ContentRoute class, but I was thinking about the issue with the JCategories::_load() method today (which kept on getting called, causing all of the additional queries you noticed) and it seems a lot of the duplicate calls were actually duplicate queries as well.

That is, they were calls for the same Category ID that should have already been loaded (and had been).

So, after kind of realizing this I changed the beginning of the JCategories::_load() method to use the node($catid) existence check and only proceed if a null value was returned:
protected function _load($id,$options)
{
    if (is_null($this->node($id)))
    {
        // Rest of _load() code here
    }
}

This immediately reduced the number of queries using that method from 213 on a page I was on to 26 queries (from a base query count of 17 without the module) with 5 of those queries being from calls to JCategories::_load() (even though the articles actually came from 15 different categories...so that ratio seems to be lower than what I expected (1/3), which is a good thing I think...because before the change the 64 articles showing up in the list were causing 3 (192/64 = 3) additional queries).

I was testing all of this on a slightly older version of mod_articles_category (from before our discussion) so the results are even more skewed in the wrong direction...(the article and category routes are getting computed in each of these cases even though only the article route is necessary...so out of those additional, and extraneous, 3 queries, 1 is coming from the article route and 2 from the category route calls in ContentHelperRoute).

Even so, the 213 queries would be considered a worst case scenario when using ContentHelperRoute so it's a good figure to compare against.

I don't know how much this would go towards making JCategories more usable, but it definitely made a difference in this scenario so I wanted to bring the above to your attention and see what thoughts you had on it.

-Omar

[Hannes Papenberg]

Hi Omar,
take a look at the routing branch. I've rewritten JCategories, the
router and the *HelperRoute classes, which reduced queries a lot and
actually made execution a lot faster.

Hannes

> > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>> wrote:
> >
> > I'm counting that this query is being used about about 120
> times:
> > SELECT c.*,CASE WHEN CHAR_LENGTH(c.alias) THEN
> CONCAT_WS(":", c.id <http://c.id>
> > <http://c.id>, c.alias) ELSE c.id <http://c.id>
> <http://c.id> END as
> > slug,COUNT(i.`id`) AS numitems
> > FROM r14742_categories as c
> > LEFT JOIN r14742_categories AS s
> > ON (s.lft = c.lft
> > AND s.rgt = c.rgt) OR (s.parent_id = c.id <http://c.id>
> <http://c.id>) OR

> > (c.parent_id = s.id <http://s.id> <http://s.id>) OR

> (s.parent_id = c.parent_id)
> > OR (s.lft <= c.lft
> > AND s.rgt >= c.rgt) OR (s.lft >= c.lft
> > AND s.rgt <= c.rgt)
> > LEFT JOIN `r14742_content` AS i

> > ON i.`catid` = c.id <http://c.id> <http://c.id>

> > WHERE (c.extension='com_content' OR c.extension='system')
> > AND c.access IN (1)

> > AND s.id <http://s.id> <http://s.id>=39
> > GROUP BY c.id <http://c.id> <http://c.id>

> > ORDER BY c.lft
> >
> > (In my test I came out with 181 queries...I switched the
> category
> > to Extensions as you mentioned, turned the levels from 1 to 10
> > and then turned the show child category articles option on).
> >
> > That seems to be forming the bulk of them, though there are some
> > others:
> >
> > Called ~20 times:
> > SELECT guest, usertype, client_id
> > FROM r14742_session
> > WHERE client_id = 0
> >
> > Called ~20 times:
> > SELECT a.username, a.time, a.userid, a.usertype, a.client_id
> > FROM r14742_session AS a
> > WHERE a.userid != 0
> > GROUP BY a.userid
> >
> >
> > On Wed, Feb 10, 2010 at 1:57 PM, Omar Ramos
> <orw...@gmail.com <mailto:orw...@gmail.com>

> > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>> wrote:
> >
> > Let me take a look at it right now...I have an inkling it
> > might be due to the content model (which I tried to use as
> > is), but let me check and get back to you in a few minutes.
> >
> > -Omar
> >
> >
> > On Wed, Feb 10, 2010 at 1:54 PM, Hannes Papenberg
> > <hack...@googlemail.com

> <mailto:hack...@googlemail.com> <mailto:hack...@googlemail.com
> <mailto:hack...@googlemail.com>>>
> > wrote:
> >
> > Hi Omar,

> > > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>
> <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>>> wrote:
> > >
> > > Darn...should have tried testing again locally
> some
> > of the
> > > functions instead of just seeing that it installed
> > properly here
> > > at work.
> > >
> > > Looks like there's an issue so if you were
> thinking
> > of testing
> > > this wait a few minutes so I can troubleshoot
> it and
> > update the
> > > patch :-).
> > >
> > > -Omar
> > >
> > >
> > > On Mon, Feb 8, 2010 at 12:44 PM, Omar Ramos
> > <orw...@gmail.com <mailto:orw...@gmail.com>
> <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>

> > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>>>>

> <mailto:joomla-dev-cms%2525252Bu...@googlegroups.com
> <mailto:joomla-dev-cms%252525252B...@googlegroups.com>
> <mailto:joomla-dev-cms%252525252B...@googlegroups.com

> <mailto:joomla-dev-cms%25252525252...@googlegroups.com>>>>>>.

[Omar Ramos]

Sounds like some of these changes may go towards the topic you brought up the other day about category URL paths?

I took a look at the changes, (there's quite a few of them), however when I tried to install the routing branch I ran into some trouble so things must be a little unstable at the moment (the configuration.php file wasn't created nor the default user loaded, but even after making those additions I still ran into trouble logging into the backend).

I noticed in the mod_articles_category helper.php file though that at the top it is still referencing router.php, while down in the code itself it seems to be referencing the ContentHelperRoute methods which might cause an error if the code were actually run:
$item->link = JRoute::_(ContentHelperRoute::getArticleRoute($item->slug, $item->catslug));


With these improvements will it mean we will still continue to use the ContentRoute classes, or will we revert back to using ContentHelperRoute?

[Hannes Papenberg]

Hi Omar,
yes, the two are related.

On the installing: I actually didn't really change anything there, so
that makes me rather concerned that the current trunk is not installable
either... I'll have a look at that later in the week...

The code that I've been creating in the routing branch is the direction
that I would go with the whole issue, but this is something that needs
as many brains on this as possible. That is why I would love to hear
more opinions on this, good or bad. In current trunk, we have a
performance issue with the categories view of com_content. Creating that
site takes just way to long, 1.1s on my system. With my rewrite with
JCategories, its just half the time. So there is work to be done in that
area. Then we have the URLs, which are just plain broken in the trunk at
the moment. They might work in a lot of places, but they don't look the
way we expect them to look. That is why there is some work necessary
there, too. The stuff that I did in the branch is one way of solving
these issues, but I'm not claiming that it is the only way nor the best
way.

I was just sitting here yesterday and basking in my own genius, thinking
how great I am. So now I need someone sane to look at this, tell me that
it is crap and that he can do it better. (And then of course show me
better code or at least a concept. ;-) )

So, if we decide on this solution, we will be going back to the old
ContentHelperRoute, so as to not introduce more backwards compatibility
issues.

Hannes

Am 28.02.2010 19:26, schrieb Omar Ramos:
> Sounds like some of these changes may go towards the topic you brought
> up the other day about category URL paths?
>
> I took a look at the changes, (there's quite a few of them), however
> when I tried to install the routing branch I ran into some trouble so
> things must be a little unstable at the moment (the configuration.php
> file wasn't created nor the default user loaded, but even after making
> those additions I still ran into trouble logging into the backend).
>
> I noticed in the mod_articles_category helper.php file though that at
> the top it is still referencing router.php, while down in the code
> itself it seems to be referencing the ContentHelperRoute methods which
> might cause an error if the code were actually run:
> $item->link =
> JRoute::_(ContentHelperRoute::getArticleRoute($item->slug,
> $item->catslug));
>
>
> With these improvements will it mean we will still continue to use the
> ContentRoute classes, or will we revert back to using ContentHelperRoute?
>
> On Sun, Feb 28, 2010 at 3:04 AM, Hannes Papenberg
> <hack...@googlemail.com <mailto:hack...@googlemail.com>> wrote:
>
> Hi Omar,

> <mailto:hack...@googlemail.com
> <mailto:hack...@googlemail.com>>> wrote:
> >
> > Ok, ContentHelperRoute is currently broken. We still have to
> > decide how
> > this is supposed to work, but so far the *HelperRoute
> classes are all
> > broken/obsolete at the moment. The correct ones to use are
> > ContentRoute,
> > etc.
> >
> > This reduces the number of queries to 40... That's still way to
> > much for
> > my taste...
> >
> > Hannes
> >
> > Am 10.02.2010 23:12, schrieb Omar Ramos:
> > > The first query is getting called with JCategories'
> _load() method
> > > which would be related to mod_articles_category,
> > >
> > > The other two queries both appear to be called from
> mod_whosonline.
> > >
> > > On Wed, Feb 10, 2010 at 2:08 PM, Omar Ramos
> <orw...@gmail.com <mailto:orw...@gmail.com>
> > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>

> > > AND s.id <http://s.id> <http://s.id> <http://s.id>=39
> > > GROUP BY c.id <http://c.id> <http://c.id> <http://c.id>

> > > ORDER BY c.lft
> > >
> > > (In my test I came out with 181 queries...I switched the
> > category
> > > to Extensions as you mentioned, turned the levels from
> 1 to 10
> > > and then turned the show child category articles
> option on).
> > >
> > > That seems to be forming the bulk of them, though
> there are some
> > > others:
> > >
> > > Called ~20 times:
> > > SELECT guest, usertype, client_id
> > > FROM r14742_session
> > > WHERE client_id = 0
> > >
> > > Called ~20 times:
> > > SELECT a.username, a.time, a.userid, a.usertype,
> a.client_id
> > > FROM r14742_session AS a
> > > WHERE a.userid != 0
> > > GROUP BY a.userid
> > >
> > >
> > > On Wed, Feb 10, 2010 at 1:57 PM, Omar Ramos
> > <orw...@gmail.com <mailto:orw...@gmail.com>
> <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>

> > > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>
> <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>>> wrote:
> > >
> > > Let me take a look at it right now...I have an
> inkling it
> > > might be due to the content model (which I tried
> to use as
> > > is), but let me check and get back to you in a few
> minutes.
> > >
> > > -Omar
> > >
> > >
> > > On Wed, Feb 10, 2010 at 1:54 PM, Hannes Papenberg
> > > <hack...@googlemail.com
> <mailto:hack...@googlemail.com>
> > <mailto:hack...@googlemail.com
> <mailto:hack...@googlemail.com>>

> <mailto:hack...@googlemail.com <mailto:hack...@googlemail.com>
> > <mailto:hack...@googlemail.com
> <mailto:hack...@googlemail.com>>>>
> > > wrote:
> > >
> > > Hi Omar,

> <mailto:orw...@gmail.com <mailto:orw...@gmail.com>
> > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>
> > > <mailto:orw...@gmail.com
> <mailto:orw...@gmail.com> <mailto:orw...@gmail.com
> <mailto:orw...@gmail.com>>>
> > > > <mailto:orw...@gmail.com
> <mailto:orw...@gmail.com>
> > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>>
> > > <mailto:orw...@gmail.com <mailto:orw...@gmail.com>

> <mailto:joomla-dev-cms%252525252B...@googlegroups.com
> <mailto:joomla-dev-cms%25252525252...@googlegroups.com>
> >
> <mailto:joomla-dev-cms%25252525252...@googlegroups.com
> <mailto:joomla-dev-cms%252525252525...@googlegroups.com>>>>>>>.