Large Intranet - Slow editing

[aram balakjian]

Hi guys I have recently been working on building a new intranet for my old employers using SilverStripe. It's a large site, currently with upwards of 1500 pages. The site is running on a dual quad-core box with 8GB of ram and served over a 100mb newtwork. However navigating the tree with an admin login is quite slow, say 5 seconds to expand a node with 20 sub-nodes and pages are certainly not snappy to load. But the real problems comes when I start to implement groups and permissions. With a non admin login expanding the same node will take upwards of 30 seconds! Another major speed problem is the Files and Images section. This is literally unusable whether with an admin login or not. Granted it has hundreds of files stored in it (accross 10's of folders), but it will take upwards of 30seconds to a minute just to load the section then another 20 seconds to open the Uploads folder. I have tried getting the systems department to install X-cache on the server but it seemed to make little difference. Has anyone got any experience with this size of site or tips on how to improve performance? Thanks Aram

[Morven Lewis-Everley]

We built a site not to long ago that was about 3 - 400 pages in size. I definatly started to notice the beginnings of what you mention. Slow access to site tree and files and images. Also it could take quite a while to load.

I found modifying the response headers helped a little, try installing the YSLOW plugin for firefox, and following its advice.

I also noticed a lot of difference using different browsers. IE6 was almost unusable, firefox 3 was faster but Safari and Chrome were by far the fastest (By a good margin).

Hope that helps a little.

Mo

[Pete Bacon Darwin]

Has this ticket been implemented yet?  If not it may give some help here?

http://open.silverstripe.com/ticket/826

Pete

Checked by AVG - www.avg.com
Version: 8.5.375 / Virus Database: 270.13.5/2220 - Release Date: 07/08/09 21:51:00

[aram balakjian]

Thanks Morven, Ill give the YSLOW plugin a go.. Unfortunately because it's an internal intranet which runs a load of other web apps written for firefox thats all we use and the main install for all the linux boxes is still on V2, I would imaging V3.5 would be as quick as chrome, so perhaps I will try to convince the systems guys to get that installed. I will let you know what happens. Thanks for your help Aram --- On Thu, 9/7/09, Morven Lewis-Everley <m.lewis...@googlemail.com> wrote:

From: Morven Lewis-Everley <m.lewis...@googlemail.com> Subject: [silverstripe-dev] Re: Large Intranet - Slow editing To: silverst...@googlegroups.com Date: Thursday, 9 July, 2009, 1:12 PM

We built a site not to long ago that was about 3 - 400 pages in size. I definatly started to notice the beginnings of what you mention. Slow access to site tree and files and images. Also it could take quite a while to load. I found modifying the response headers helped a little, try installing the YSLOW plugin for firefox, and following its advice. I also noticed a lot of difference using different browsers. IE6 was almost unusable, firefox 3 was faster but Safari and Chrome were by far the fastest (By a good margin). Hope that helps a little. Mo

On Thu, Jul 9, 2009 at 11:39 AM, aram balakjian <aramba...@yahoo.co.uk> wrote:

[Liam Whittle]

This gets brought up every so often as it is a major issue for large sites. There is a forum topic with a hack to at least limit the amount of pages shown in the admin section per node.

http://silverstripe.org/archive/show/15416?start=24#post76093

I'm unaware of anything for the files/images section though.

I just launched a site recently that will soon run into this problem as a lot of content is added monthly. Hopefully a real solution is in place as it really makes using SS for large sites almost impossible.

[Pete Bacon Darwin]

It would be interesting to see a profile, Aram, from the page that takes 30 secs to load up when not an administrator.

 

I haven’t checked this but I guess that one reason this using non-admin users is even more slow is that canView is being called on every page in the hierarchy.  I noticed that the SiteTree:canView is rather complex and also calls up to the parent page’s canView object.  This means that a parent page’s canView is being called at least as many times as it has children.  It would seem that some kind of caching might be helpful here.

 

For instance in SiteTree you could have an instance variable $canViewCache which would be an array that uses member id for its keys and the member’s canView status for this page for its values.  Then the start of the SiteTree::canView function would look like:

 

       public function canView($member = null) {

              if(!$member || !(is_a($member, 'Member')) || is_numeric($member)) $member = Member::currentUser();

              if ( isset($this->canViewCache[$member->id]) {

                     return $this->canViewCache[$member->id];

              }

              ...  the rest of the function.

       }

 

Any thoughts?

 

Pete

 

From: silverst...@googlegroups.com [mailto:silverst...@googlegroups.com] On Behalf Of aram balakjian

Sent: 09 July 2009 11:39
To: silverst...@googlegroups.com

[Pete Bacon Darwin]

Also the Permission system in general could be given a workout to make it a bit leaner and meaner!  There are potentially a lot of small DB calls going on.

Pete

 

From: silverst...@googlegroups.com [mailto:silverst...@googlegroups.com] On Behalf Of aram balakjian
Sent: 09 July 2009 11:39
To: silverst...@googlegroups.com
Subject: [silverstripe-dev] Large Intranet - Slow editing

 

Hi guys

[Sigurd Magnusson]

I've certainly been impressed with Firefox 3.5 speed-ups; not that
I've had time to actually benchmark them and compare it to the
millisecond...

On 10/07/2009, at 2:22 AM, aram balakjian wrote:

> I would imaging V3.5 would be as quick

Sig

[Nicolaas Thiemen Francken - Sunny Side Up]

Hi

I am not sure how the "serving speed" of a website has much to do with the browser that you are using. I mean, the problem with big sites is that the PHP processing (and related Mysql retrievals).  How a locally installed Firefox install can make a different to that is unclear to me. 

For me, also, it is really important that SS allows the creation of large sites without slowing down. I have found that 4000 pages works ok (slow on the CMS, but reasonably fast on the front), but that 4000 files and images is unworkable.

One thing that I have done to speed things up for a 4000 page website is to cache the menu (saved in the database).  This reduces the need for traversing the sitetree at runtime (see http://sunny.svnrepository.com/svn/sunny-side-up-general/menucache for a pre-alpha release). 

Nicolaas 

2009/7/10 Sigurd Magnusson <sig...@silverstripe.com>

[Sam Minnee]

> For me, also, it is really important that SS allows the creation of large
> sites without slowing down. I have found that 4000 pages works ok (slow on
> the CMS, but reasonably fast on the front), but that 4000 files and images
> is unworkable.

I've got some performance fixes for the Files & Images section on a
project branch. They turning the sync operation (that looks for files
& images uploaded via FTP or something) into a button, and removing
the slow & broken "unused images" tab. There were also some more
performances fixes for the generation of the tree.

I'll make sure that they make their way into trunk shortly. We're
going to do a 2.3.3 release with a few critical bugfixes, but I'm not
sure if these changes are really appropriate for 2.3.3 because they
change the functionality of the Files & Images section.

[Keri Henare]

The changes do change the functionality yes but I don't think that
they would break code.
We have a client who calls the Files & Images section "The Site
Exploder!". They have so many images that PHP requires more than the
Gig of RAM allocated to it.

So we would really appreciate this fix into the 2.3.3 release.

---------------------------------------------------
Keri Henare

[e] ke...@henare.co.nz
[m] 021 874 552
[w] www.kerihenare.com

[Cam Spiers]

+1 for ASAP.

This is a pain point for our clients too.

Cam Spiers

[Nicolaas Thiemen Francken - Sunny Side Up]

+1

[Craig Boxall]

+1000000


On 10/07/2009, at 12:34 PM, Nicolaas Thiemen Francken - Sunny Side Up
wrote:

> +1
>
> >

[Liam Whittle]

Ya just voicing my opinion as well, but I'd really, _really_ like to have this taken care of as soon as possible. I realize there are other things to do, and this has been in here all along, but it really makes running a large site impossible. Cranking up memory usage allowance isn't really a good fix, though it might be a temporary one.

On Thu, Jul 9, 2009 at 8:34 PM, Nicolaas Thiemen Francken - Sunny Side Up <nfra...@gmail.com> wrote:

+1

[Joe Crawford]

This is a major source of pain for a client of mine as well. Big
bummer for a large intranet. They're living with it for now, but it
does not endear them (or me) to my great choice.

Joe
--
Joe Crawford
http://joecrawford.com/
805-857-3951

[Sam Minnee]

Haha, I think the message here is pretty clear. Given the
overwhelming response I think it's probably worth pushing this into
2.3.3.

[Sigurd Magnusson]

Heh - yes looks like it :)

I suggest we have a method you can have in _config.php that lets you
retain automatic syncing of assets file system <-> db schema for those
with smaller sites; it is a nice convenient feature when it works
quickly.

Sigurd.

-------
Sigurd Magnusson
Chief Marketing Officer
SilverStripe

--
SilverStripe is the 2008 most promising open source CMS platform!
http://packtpub.com/article/2008-most-promising-open-source-cms-announced
--

DDI: +64 4 978 7332
Mobile: +64 21 42 12 08
Skype: sigurdmagnusson
twitter.com/SigurdMagnusson

Level 5, 97-99 Courtenay Place
Wellington, New Zealand
www.silverstripe.com

[aram balakjian]

Wow, it seems I'm really not the only one having this problem :) Great to hear that update will make it into 2.3.3, for now I have got around it by removing the SilverStripe TinyMCE manager and installing the ImageManager plugin which provides a pretty good and most importanty very snappy file manager from within the Editor. I have then hidden the Files & images tab from the menu. I got firefox 3.5 installed and indeed from the javascript side of things it did speed things up noticably, but I think this is kind of beyond the point, there are clearly some fundemental issues particularly with the way permissions are checked when expanding the tree. Anyway it's great to see that this is an issue that everyone is aware of. I will give the menu caching a go, thanks for that Nicolaas. Aram --- On Fri, 10/7/09, Sigurd Magnusson <sig...@silverstripe.com> wrote:

[Jamie Neil]

Sam Minnee wrote:
> I've got some performance fixes for the Files & Images section on a
> project branch. They turning the sync operation (that looks for files
> & images uploaded via FTP or something) into a button, and removing
> the slow & broken "unused images" tab. There were also some more
> performances fixes for the generation of the tree.

Just removing the "unused images" code would be a huge improvement -
I've had to comment it out in at least three sites to avoid script timeouts.

Jamie

[DesignCity]

> I suggest we have a method you can have in _config.php that lets you
> retain automatic syncing of assets file system <-> db schema for those
> with smaller sites; it is a nice convenient feature when it works
> quickly.

Sig - why not leave the automatic syncing of the assets file system by
default and add in a method for _config.php for removing it? I believe
the number of installs for smaller sites would far outweigh the larger
ones; it is a simple enough fix for those of us with larger sites to
implement, and I'd dare say we have the experience to be able to make
the change easily enough. Perhaps if there are a number of changes
that can speed up the CMS for large sites, we can either create a
tutorial for the necessary alterations or link them all to a single
method in _config.php?

M :)

[Nicolaas Thiemen Francken - Sunny Side Up]

Ultimately, we should work out why the synching does not work.  I will have a look at this on a site that I am working on where we have 5000 images.

- Nicolaas

[nicolaas]

Many assets crashing your site?

This is what I found: http://open.silverstripe.com/ticket/4106

- Nicolaas

[blueskies79]

So does anyone know of a quick-fix or workaround for this problem ?
We have one site which can hardly be edited because of the amount of
assets (waiting for image or file fields to appear in the CMS takes
minutes).

[Nicolaas Thiemen Francken - Sunny Side Up]

Hey

You may find some solace here: http://open.silverstripe.com/changeset/80782 - just been submitted to trunk a few days ago.

NT

[blueskies79]

Thanks.. It didn't really speed things up though... ;-(

On Aug 7, 4:05 pm, Nicolaas Thiemen Francken - Sunny Side Up
<nfranc...@gmail.com> wrote:
> Hey
> You may find some solace here:http://open.silverstripe.com/changeset/80782-just been submitted to

[Nicolaas Thiemen Francken - Sunny Side Up]

no, i agree there still a long way to go