Menu / Module Load

[Matthew]

I'm not quite sure this is a development question or an
'administrative' question, but I think it focuses on the broad scheme
of setting up large scale, enterprise level Joomla sites, so I'll post
it here.

I have a site, around 10,000 articles/pages. The site has around 200
categories, and each category has a menu and menu module. There are
around These categories/menus/modules represent departments. There
around 6000 menu items in total. This is all managed in a single
installation.

In J!1.5, administering this wasn't a big deal with extremely heavy
traffic. The module loaded and saved quickly. Adding menu items was
really quick. Well, after a migration to J!1.6, it either takes
around 2 minutes OR the browser completely crashes when trying to edit
a menu module. IF the page does load and I get to make changes,
saving around 2 minutes OR the browser crashes.

This is a test server with no traffic right now, and I think they have
8 cores and 16gb of ram dedicated to it. It also uses a SCSI SAS. The
server is a powerhorse, so I don't think it's a server issue.

Thoughts?

[elin]

This is when actually editing the module, not the menu items?  I wonder if it is that menu assignment area since you have so many menu items that must be crazy. And are you saying 6000 total on 200 menus so that is about 30 per menu? 

Elin

[Matthew]

Yes, when editing the module. Yes, about 30 per menu.

I run a the entire university web site with one instance of Joomla
1.5. It works REALLY REALLY well. But, 1.6 is painfully SLOW and
really not working too well right now, especially when trying to
administer.

Here's a screen capture (once it loads) of the menu assignment area:
http://www.bornin83.com/share/menus.jpg

Any thoughts or recommendations are appreciated. Is J1.6 just not
meant for a site of this size?

[Mark Dexter]

In theory, version 1.6 should be faster than version 1.5. Having said that, it is likely that there are bugs or performance bottlenecks that we haven't found and fixed yet. So your help in working through this is greatly appreciated.

One thing you could try is to turn on System Debug and then look at the profile information below the page. That may give you an idea of which query or queries is causing the problem. That could provide a useful clue.

Thanks. Mark

--
You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.
To post to this group, send an email to joomla-de...@googlegroups.com.
To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/joomla-dev-general?hl=en-GB.

[Matthew]

Here's something else on the frontend....

This is loading 1 article with "hello world" as the content, with 1
menu/module published with 1 link (to itself).

Profile Information
Application 0.000 seconds (+0.000); 0.89 MB (+0.89) - afterLoad
Application 0.047 seconds (+0.046); 4.51 MB (+3.62) - afterInitialise
Application 0.815 seconds (+0.769); 46.82 MB (+42.31) - afterRoute
-> Application 62.561 seconds (+61.746); 50.59 MB (+3.77) -
afterDispatch
-> Application 62.570 seconds (+0.008); 50.88 MB (+0.29) - afterRender

Memory Usage
50.88 MB (53,347,920 Bytes)
21 Queries Logged
SELECT `data`
FROM `j16_session`
WHERE `session_id` = '4gk34fuato3dbv56a2pkh4vjl6'
DELETE
FROM `j16_session`
WHERE `time` < 1299852443
SELECT `session_id`
FROM `j16_session`
WHERE `session_id` = '4gk34fuato3dbv56a2pkh4vjl6'
LIMIT 0, 1
INSERT INTO `j16_session` (`session_id`, `client_id`, `time`)
VALUES ('4gk34fuato3dbv56a2pkh4vjl6', 0, 1299853343)
SELECT extension_id AS "id", element AS "option", params, enabled
FROM j16_extensions
WHERE `type` = 'component'
AND `element` = 'com_users'
SELECT id, rules
FROM `j16_viewlevels`
SELECT folder AS type, element AS name, params
FROM j16_extensions
WHERE enabled >= 1
AND type ='plugin'
AND state >= 0
AND access IN (1,1)
ORDER BY ordering
SELECT extension_id AS "id", element AS "option", params, enabled
FROM j16_extensions
WHERE `type` = 'component'
AND `element` = 'com_languages'
SELECT m.id, m.menutype, m.title, m.alias, m.path AS route, m.link,
m.type, m.level,m.browserNav, m.access, m.params, m.home, m.img,
m.template_style_id, m.component_id, m.parent_id,m.language,e.element
as component
FROM j16_menu AS m
LEFT JOIN j16_extensions AS e
ON m.component_id = e.extension_id
WHERE m.published = 1
AND m.parent_id > 0
AND m.client_id = 0
AND m.access IN (1,1)
ORDER BY m.lft
SELECT extension_id AS "id", element AS "option", params, enabled
FROM j16_extensions
WHERE `type` = 'component'
AND `element` = 'com_content'
SELECT *
FROM j16_languages
WHERE published=1
SELECT id, home, template, params
FROM j16_template_styles
WHERE client_id = 0
SELECT a.rules
FROM j16_assets AS a
WHERE (a.id = 1)
GROUP BY a.id
SELECT b.rules
FROM j16_assets AS a
LEFT JOIN j16_assets AS b
ON b.lft <= a.lft
AND b.rgt >= a.rgt
WHERE (a.name = 'com_content' OR a.parent_id=0)
GROUP BY b.id
ORDER BY b.lft
SELECT a.id, a.asset_id, a.title, a.alias, a.title_alias, a.introtext,
a.fulltext, CASE WHEN badcats.id is null THEN a.state ELSE 0 END AS
state, a.mask, a.catid, a.created, a.created_by, a.created_by_alias,
a.modified, a.modified_by, a.checked_out, a.checked_out_time,
a.publish_up, a.publish_down, a.images, a.urls, a.attribs, a.version,
a.parentid, a.ordering, a.metakey, a.metadesc, a.access, a.hits,
a.metadata, a.featured, a.language, a.xreference,c.title AS
category_title, c.alias AS category_alias, c.access AS
category_access,u.name AS author,contact.id as contactid,parent.title
as parent_title, parent.id as parent_id, parent.path as parent_route,
parent.alias as parent_alias,ROUND( v.rating_sum / v.rating_count ) AS
rating, v.rating_count as rating_count
FROM j16_content AS a
LEFT JOIN j16_categories AS c
on c.id = a.catid
LEFT JOIN j16_users AS u
on u.id = a.created_by
LEFT JOIN j16_contact_details AS contact
on contact.user_id = a.created_by
LEFT JOIN j16_categories as parent
ON parent.id = c.parent_id
LEFT JOIN j16_content_rating AS v
ON a.id = v.content_id
LEFT
OUTER JOIN (SELECT cat.id as id
FROM j16_categories AS cat JOIN j16_categories AS parent
ON cat.lft BETWEEN parent.lft
AND parent.rgt
WHERE parent.extension = 'com_content'
AND parent.published <= 0
GROUP BY cat.id) AS badcats
ON badcats.id = c.id
WHERE a.id = 9448
AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <=
'2011-03-11 14:22:56')
AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >=
'2011-03-11 14:22:56')
AND (a.state = 1 OR a.state =2)
SELECT c.*,CASE WHEN CHAR_LENGTH(c.alias) THEN CONCAT_WS(":", c.id,
c.alias) ELSE c.id END as slug
FROM j16_categories as c
LEFT JOIN j16_categories AS s
ON (s.lft <= c.lft
AND s.rgt >= c.rgt) OR (s.lft > c.lft
AND s.rgt < c.rgt)
LEFT JOIN (SELECT cat.id as id
FROM j16_categories AS cat JOIN j16_categories AS parent
ON cat.lft BETWEEN parent.lft
AND parent.rgt
WHERE parent.extension = 'com_content'
AND parent.published != 1
GROUP BY cat.id) AS badcats
ON badcats.id = c.id
WHERE (c.extension='com_content' OR c.extension='system')
AND c.access IN (1,1)
AND c.published = 1
AND s.id=7
AND badcats.id is null
GROUP BY c.id
ORDER BY c.lft
SELECT b.rules
FROM j16_assets AS a
LEFT JOIN j16_assets AS b
ON b.lft <= a.lft
AND b.rgt >= a.rgt
WHERE (a.name = 'com_content.article.9448' OR a.parent_id=0)
GROUP BY b.id
ORDER BY b.lft
SELECT a.id, CASE WHEN CHAR_LENGTH(a.alias) THEN CONCAT_WS(":", a.id,
a.alias) ELSE a.id END as slug, CASE WHEN CHAR_LENGTH(cc.alias) THEN
CONCAT_WS(":", cc.id, cc.alias) ELSE cc.id END as catslug
FROM j16_content AS a
LEFT JOIN j16_categories AS cc
ON cc.id = a.catid
WHERE a.catid = 7
AND a.state = 1
AND a.access = 1
AND (a.state = 1 OR a.state = -1)
AND (publish_up = '0000-00-00 00:00:00' OR publish_up <= '2011-03-11
14:22:56')
AND (publish_down = '0000-00-00 00:00:00' OR publish_down >=
'2011-03-11 14:22:56')
ORDER BY a.created DESC
UPDATE j16_content
SET hits = hits + 1
WHERE id = 9448
SELECT id, title, module, position, content, showtitle, params,
mm.menuid
FROM j16_modules AS m
LEFT JOIN j16_modules_menu AS mm
ON mm.moduleid = m.id
WHERE m.published = 1
AND (m.publish_up = '0000-00-00 00:00:00' OR m.publish_up <=
'2011-03-11 14:23:26')
AND (m.publish_down = '0000-00-00 00:00:00' OR m.publish_down >=
'2011-03-11 14:23:26')
AND m.access IN (1,1)
AND m.client_id = 0
AND (mm.menuid = 21 OR mm.menuid <= 0)
ORDER BY position, ordering
UPDATE `j16_session`
SET `data` = '__default|a:8:{s:15:\"session.counter\";i:1;s:
19:\"session.timer.start\";i:1299853343;s:18:\"session.timer.last\";i:
1299853343;s:17:\"session.timer.now\";i:1299853343;s:
22:\"session.client.browser\";s:128:\"Mozilla/5.0 (Macintosh; U; Intel
Mac OS X 10_6_6; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/
9.0.597.107 Safari/534.13\";s:8:\"registry\";O:9:\"JRegistry\":1:{s:
7:\"\0*\0data\";O:8:\"stdClass\":0:{}}s:4:\"user\";O:5:\"JUser\":23:{s:
9:\"\0*\0isRoot\";b:0;s:2:\"id\";i:0;s:4:\"name\";N;s:8:\"username
\";N;s:5:\"email\";N;s:8:\"password\";N;s:14:\"password_clear\";s:
0:\"\";s:8:\"usertype\";N;s:5:\"block\";N;s:9:\"sendEmail\";i:0;s:
12:\"registerDate\";N;s:13:\"lastvisitDate\";N;s:10:\"activation\";N;s:
6:\"params\";N;s:6:\"groups\";a:0:{}s:5:\"guest\";i:1;s:10:\"\0*
\0_params\";O:9:\"JRegistry\":1:{s:7:\"\0*\0data\";O:8:\"stdClass\":0:
{}}s:14:\"\0*\0_authGroups\";a:1:{i:0;i:1;}s:14:\"\0*\0_authLevels\";a:
2:{i:0;i:1;i:1;i:1;}s:15:\"\0*\0_authActions\";N;s:12:\"\0*\0_errorMsg
\";N;s:10:\"\0*\0_errors\";a:0:{}s:3:\"aid\";i:0;}s:
16:\"com_mailto.links\";a:1:{s:
40:\"4c7d36fdb10451017fe5ba4e73705f1422d4af1f\";O:8:\"stdClass\":2:{s:
4:\"link\";s:32:\"http://cms2.tntech.edu/jupgrade/\";s:6:\"expiry\";i:
1299853406;}}}', `time` = 1299853406
WHERE `session_id` = '4gk34fuato3dbv56a2pkh4vjl6'
18 Query Types Logged, Sorted by Occurrences.
SELECT Tables:
3 × SELECT extension_id AS "id", element AS "option", params, enabled
FROM j16_extensions
2 × SELECT b.rules FROM j16_assets AS a LEFT JOIN j16_assets AS b ON
b.lft <= a.lft AND b.rgt >= a.rgt
1 × SELECT a.id, a.asset_id, a.title, a.alias, a.title_alias,
a.introtext, a.fulltext, CASE WHEN badcats.id is null THEN a.state
ELSE 0 END AS state, a.mask, a.catid, a.created, a.created_by,
a.created_by_alias, a.modified, a.modified_by, a.checked_out,
a.checked_out_time, a.publish_up, a.publish_down, a.images, a.urls,
a.attribs, a.version, a.parentid, a.ordering, a.metakey, a.metadesc,
a.access, a.hits, a.metadata, a.featured, a.language,
a.xreference,c.title AS category_title, c.alias AS category_alias,
c.access AS category_access,u.name AS author,contact.id as
contactid,parent.title as parent_title, parent.id as parent_id,
parent.path as parent_route, parent.alias as
parent_alias,ROUND( v.rating_sum / v.rating_count ) AS rating,
v.rating_count as rating_count FROM j16_content AS a LEFT JOIN
j16_categories AS c on c.id = a.catid LEFT JOIN j16_users AS u on u.id
= a.created_by LEFT JOIN j16_contact_details AS contact on
contact.user_id = a.created_by LEFT JOIN j16_categories as parent ON
parent.id = c.parent_id LEFT JOIN j16_content_rating AS v ON a.id =
v.content_id LEFT OUTER JOIN (SELECT cat.id as id FROM j16_categories
AS cat JOIN j16_categories AS parent ON cat.lft BETWEEN parent.lft AND
parent.rgt
1 × SELECT c.*,CASE WHEN CHAR_LENGTH(c.alias) THEN CONCAT_WS(":",
c.id, c.alias) ELSE c.id END as slug FROM j16_categories as c LEFT
JOIN j16_categories AS s ON (s.lft <= c.lft AND s.rgt >= c.rgt) OR
(s.lft > c.lft AND s.rgt < c.rgt) LEFT JOIN (SELECT cat.id as id FROM
j16_categories AS cat JOIN j16_categories AS parent ON cat.lft BETWEEN
parent.lft AND parent.rgt
1 × SELECT a.id, CASE WHEN CHAR_LENGTH(a.alias) THEN CONCAT_WS(":",
a.id, a.alias) ELSE a.id END as slug, CASE WHEN CHAR_LENGTH(cc.alias)
THEN CONCAT_WS(":", cc.id, cc.alias) ELSE cc.id END as catslug FROM
j16_content AS a LEFT JOIN j16_categories AS cc ON cc.id = a.catid
1 × SELECT id, title, module, position, content, showtitle, params,
mm.menuid FROM j16_modules AS m LEFT JOIN j16_modules_menu AS mm ON
mm.moduleid = m.id
1 × SELECT a.rules FROM j16_assets AS a
1 × SELECT * FROM j16_languages
1 × SELECT id, rules FROM `j16_viewlevels
1 × SELECT `session_id` FROM `j16_session`
1 × SELECT folder AS type, element AS name, params FROM j16_extensions
1 × SELECT m.id, m.menutype, m.title, m.alias, m.path AS route,
m.link, m.type, m.level,m.browserNav, m.access, m.params, m.home,
m.img, m.template_style_id, m.component_id,
m.parent_id,m.language,e.element as component FROM j16_menu AS m LEFT
JOIN j16_extensions AS e ON m.component_id = e.extension_id
1 × SELECT `data` FROM `j16_session`
1 × SELECT id, home, template, params FROM j16_template_styles
OTHER Tables:
1 × UPDATE `j16_session` SET `data` = '__default|a:8:{s:
15:\"session.counter\";i:1;s:19:\"session.timer.start\";i:1299853343;s:
18:\"session.timer.last\";i:1299853343;s:17:\"session.timer.now\";i:
1299853343;s:22:\"session.client.browser\";s:128:\"Mozilla/5.0
(Macintosh; U; Intel Mac OS X 10_6_6; en-US) AppleWebKit/534.13
(KHTML, like Gecko) Chrome/9.0.597.107 Safari/534.13\";s:8:\"registry
\";O:9:\"JRegistry\":1:{s:7:\"\0*\0data\";O:8:\"stdClass\":0:{}}s:
4:\"user\";O:5:\"JUser\":23:{s:9:\"\0*\0isRoot\";b:0;s:2:\"id\";i:0;s:
4:\"name\";N;s:8:\"username\";N;s:5:\"email\";N;s:8:\"password\";N;s:
14:\"password_clear\";s:0:\"\";s:8:\"usertype\";N;s:5:\"block\";N;s:
9:\"sendEmail\";i:0;s:12:\"registerDate\";N;s:13:\"lastvisitDate\";N;s:
10:\"activation\";N;s:6:\"params\";N;s:6:\"groups\";a:0:{}s:5:\"guest
\";i:1;s:10:\"\0*\0_params\";O:9:\"JRegistry\":1:{s:7:\"\0*\0data\";O:
8:\"stdClass\":0:{}}s:14:\"\0*\0_authGroups\";a:1:{i:0;i:1;}s:14:\"\0*
\0_authLevels\";a:2:{i:0;i:1;i:1;i:1;}s:15:\"\0*\0_authActions\";N;s:
12:\"\0*\0_errorMsg\";N;s:10:\"\0*\0_errors\";a:0:{}s:3:\"aid\";i:0;}s:
16:\"com_mailto.links\";a:1:{s:
40:\"4c7d36fdb10451017fe5ba4e73705f1422d4af1f\";O:8:\"stdClass\":2:{s:
4:\"link\";s:32:\"http://cms2.tntech.edu/jupgrade/\";s:6:\"expiry\";i:
1299853406;}}}', `time` = 1299853406
1 × UPDATE j16_content SET hits = hits + 1
1 × INSERT INTO `j16_session` (`session_id`, `client_id`, `time`)
VALUES ('4gk34fuato3dbv56a2pkh4vjl6', 0, 1299853343
1 × DELETE FROM `j16_session`

[Mark Dexter]

You have something interesting going on, maybe in your template, or your server? Not sure. Here is my debug from a localhost install with sample database, loading the home page:

Profile Information

Application 0.001 seconds (+0.001); 0.58 MB (+0.58) - afterLoad
Application 0.052 seconds (+0.050); 4.17 MB (+3.59) - afterInitialise
Application 0.096 seconds (+0.044); 5.54 MB (+1.37) - afterRoute
Application 0.311 seconds (+0.215); 9.33 MB (+3.79) - afterDispatch
Application 0.439 seconds (+0.128); 9.94 MB (+0.61) - afterRender

Memory Usage

9.94 MB (10,424,120 Bytes)

19 Queries Logged

Maybe you could try a vanilla site with sample data and compare it to your test? Both your time (62+ seconds) and memory (50mb) seem a lot higher that expected.

Good luck. Mark

[Ian MacLennan]

He has 6000 menu items.

Is there any chance of getting the data so we can test and profile locally?  Otherwise, I can probably generate a data set, but it will take a little longer.

Ian

[Matthew]

Hey Guys,
I've used the basic template that came with 1.6. But, it doesn't
really address the admin area lag.

Ian, I'm sending you an e-mail about the data.

Thanks!
-mg

On Mar 11, 11:17 pm, Ian MacLennan <ian.maclen...@joomla.org> wrote:
> He has 6000 menu items.
>
> Is there any chance of getting the data so we can test and profile locally?
>  Otherwise, I can probably generate a data set, but it will take a little
> longer.
>
> Ian
>
>
>
>
>
>
>
> On Fri, Mar 11, 2011 at 5:47 PM, Mark Dexter <dextercow...@gmail.com> wrote:
> > You have something interesting going on, maybe in your template, or your
> > server? Not sure. Here is my debug from a localhost install with sample
> > database, loading the home page:
>
> > Profile Information
>
> > Application 0.001 seconds (+0.001); 0.58 MB (+0.58) - afterLoad
> > Application 0.052 seconds (+0.050); 4.17 MB (+3.59) - afterInitialise
> > Application 0.096 seconds (+0.044); 5.54 MB (+1.37) - afterRoute
> > Application 0.311 seconds (+0.215); 9.33 MB (+3.79) - afterDispatch
> > Application 0.439 seconds (+0.128); 9.94 MB (+0.61) - afterRender
>
> > Memory Usage
>
> > 9.94 MB (10,424,120 Bytes)
>
> > 19 Queries Logged
>
> > Maybe you could try a vanilla site with sample data and compare it to your
> > test? Both your time (62+ seconds) and memory (50mb) seem a lot higher that
> > expected.
>
> > Good luck. Mark
>

> ...
>
> read more »