speed speed speed

214 views
Skip to first unread message

Nicolaas Thiemen Francken - Sunny Side Up

unread,
Jun 15, 2015, 8:15:13 PM6/15/15
to silverstripe-dev
Hi Everyone

I hope it is OK to post this message here.  

Speed seems to be a big topic these days.  We have used the following techniques:
  • fast hosting company ;-)
  • limit use of modules and add-ons in PHP where possible.
  • innoDB tables in RAM (give MYSQL lots of RAM)
  • use IDs rather than classes for CSS + JS references
  • use javascript rather than jQuery where appropriate
  • simplify html
  • ajaxify forms (the last one is also great to avoid spammers) - you can also use this for carts, etc...
  • caching in PHP using private statics for custom methods that may be called more than once (e.g. if self::$foo is empty then fill it otherwise return self::$foo)
  • partial caching in *.ss templates. Our basic rules it to cache every loop. 
  • write CSS without the use of compass and that sort of thing (keep CSS simple). 
  • write considered JS that does not change the DOM too much (e.g. if you have an error message then just add it to DOM with JS only showing it - not adding it). 
  • minify CSS and JS into one file (combine_files)
  • cloudflare or another CDN (we had a bad experience with CF, but of course it makes sense to use it) - especially if you have a global audience. Does it pay to load jQuery via Google CDN (methinks NOT!)
  • static publishing (we have NOT used this much, but we should). 
Now, I have two questions:

(1) any feedback on the list?  What is nonsense, what should we add? 

(2) what caching "engine" do you use.  We tend to use the SS default one, but we have also used APC (with mixed results).  What do you recommend? Does anyone have any idea on how to tune that. 

I would love to hear your thoughts.

THANK YOU.


Nicolaas

Marcus Nyeholt

unread,
Jun 15, 2015, 8:52:45 PM6/15/15
to silverst...@googlegroups.com
For the server side -

* Use XHProf to identify your bottlenecks. Loading lots of objects in loops, or triggering lazyload events, can quickly pile on the amount of DB queries executed
* Apply partial caching from templates where appropriate
* In some cases, use a key/value cache for in-code things where partial-caching isn't feasible or the cached data needs to be used from many sections of a page and across requests
* Eventually, you will get to a point where the bottle neck becomes the actual bootstrapping process of the framework; at this point, you need to start looking at using full page caching







--
You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to silverstripe-d...@googlegroups.com.
To post to this group, send email to silverst...@googlegroups.com.
Visit this group at http://groups.google.com/group/silverstripe-dev.
For more options, visit https://groups.google.com/d/optout.

Nicolaas Thiemen Francken - Sunny Side Up

unread,
Jun 16, 2015, 4:37:35 AM6/16/15
to silverstripe-dev
Thank you for your ideas Marcus.  

Here is an updated list:

network:
  • Cloudflare or another CDN (we had a bad experience with CF, but of course it makes sense to use a CDN) - especially if you have a global audience. 
  • Host in the same region as your audience
  • Set manual expiration times (e.g. one month) for images, css, js
  • Compress response files (gzip your html sort of thing) - pretty standard
  • Reduce requests by minifying / combining CSS and JS into one file (Requirements::combine_files())
  • Browser cache the HTML where possible.

server: 

application:
  • Use XHProf to identify your bottlenecks. Loading lots of objects in loops, or triggering lazyload events, can quickly pile on the amount of DB queries executed (THANK YOU Marcus). 
  • Limit use of modules and add-ons in PHP
  • Caching in PHP using private statics for custom methods that may be called more than once (e.g. if self::$foo is empty then fill it using the method below otherwise return self::$foo without further ado)
  • Partial caching in *.ss templates. Our basic rules it to cache every loop in the templates using <% cached foo %> 
  • Frameworks (e.g. jQuery) via Google / Another CDN
  • Static publishing of pages

client-side:
  • Above the fold inline css
  • Responsive image techniques - use CSS media queries to get the best image for the screen. 
  • Use IDs rather than classes for CSS + JS references (faster to lookup)
  • Use javascript rather than jQuery where appropriate/practicable)
  • Simplify html - reduce the number of nodes.
  • Ajaxify forms - only load forms on request using ajax techniques
  • Write CSS without the use of compass and that sort of thing (keep CSS simple). 
  • Write considered JS that does not change the DOM too much (e.g. form validation - add error messages on first load then just add it to DOM with JS only showing it - not adding it). 

references:

Daniel Hensby

unread,
Jun 16, 2015, 6:09:43 AM6/16/15
to silverstripe-dev
Nice list of practices there.

I would make a few observations:

> Memcached / APC for SS based caching - what is the best one?

Memcached is probably slightly more performant, but APC has the advantage of being a native PHP module; I think they also do slightly different things as APC caches the compiled PHP files, rather than compiling on every request (as well as acting as an cache store). see http://docs.silverstripe.org/en/3.1/developer_guides/performance/caching/ for more info

> Caching in PHP using private statics for custom methods that may be called more than once (e.g. if self::$foo is empty then fill it using the method below otherwise return self::$foo without further ado)

This is good, but you make a memory trade-off. Note that the template engine will do this automatically for you, too.

> Frameworks (e.g. jQuery) via Google / Another CDN

This contradicts an earlier point about minimising HTTP requests. Ultimately, if you have gzip and good cache headers, you may as well include jQuery in your minified JS file. That connection will be opened anyway. Also, using CDNs to deliver jQuery can actually slow your site if they are unavailable (unlikely but possible) or blocked by the ISP/country of the visitor (more likely).

> Use IDs rather than classes for CSS + JS references (faster to lookup)

tl;dr: Although IDs may be fractionally faster, they aren't worth the trade-offs

This is really a trade-off between more maintainable and "modular" CSS/JS and speed, and when we say "speed" we mean micro optimisations here. Using an ID for CSS is so minutely slower, it's not worth the pain of it's specificity. It's best to keep a flat class structure (any kind of nested rules are slower) further reading: http://benfrain.com/css-performance-revisited-selectors-bloat-expensive-styles/. If you're interested in modular CSS that's fast and flexible, I'd recommend reading up on BEM.

The same is true with JS - I'd say it's quite rare that you'd ever have a JS component that can safely hook into an ID (ie: you only need it once on any page) and the cons of tightly coupling your JS to your IDs, again, make this a pain. An interesting article and it's test results actually show class selectors are faster?!

Anyway, these kinds of optimisations for the front-end are really only important if you're running a massive front-end app OR have very large datasets/DOM trees to work with.

> Ajaxify forms - only load forms on request using ajax techniques

I'm not sure if I follow, if you mean lazy-load forms like one could with images this may be quite bad for people using assistive technologies (like screen readers) that expect to be able to find a form on the page when loaded.

To add to server: HTTP 2.0 / turn on Keep-Alive; and Image resampling optimisation.

As with lots of these things (and particularly your front-end ones) the scalability, modularity and maintainability (of your development workflow) are often going to be at odds with some of the things you've suggested.

I strictly work to never using IDs (for selectors) and if we need an extra wrapper div to make a component modular or to add another styling hook, then we do. The trade-offs for front-end performance (for the work we do) is completely unnoticeable and using minification, image optimisation, gzip, caching and faster protocols will all make a far superior speed difference to the browser than replacing your CSS class selectors with IDs.

Last thought: Lots of your server optimisations are about moving things into memory - again, it's about being pragmatic (unless you have a huge quantity of memory available) when determining what can and can't be held in RAM. I probably wouldn't load by 5GB DB into memory on a small VPS (for example) especially if it involves using the page file as you'll just end up thrashing anyway.

Hope that's somewhat informative and useful.

Dan

--

Nicolaas Thiemen Francken - Sunny Side Up

unread,
Jun 16, 2015, 6:59:40 AM6/16/15
to silverstripe-dev
Hey Dan

Great notes below. I agree with all you are saying. A few notes below ...



On 16 June 2015 at 22:09, Daniel Hensby <d...@hens.by> wrote:
Nice list of practices there.

I would make a few observations:

> Memcached / APC for SS based caching - what is the best one?

Memcached is probably slightly more performant, but APC has the advantage of being a native PHP module; I think they also do slightly different things as APC caches the compiled PHP files, rather than compiling on every request (as well as acting as an cache store). see http://docs.silverstripe.org/en/3.1/developer_guides/performance/caching/ for more info


​In terms of APC vs Memcached, different of course, but, as far as I understand it, both can be used for the SS native caching class. I would be keen to hear what experiences people had with cache stores.  

 
> Caching in PHP using private statics for custom methods that may be called more than once (e.g. if self::$foo is empty then fill it using the method below otherwise return self::$foo without further ado)

This is good, but you make a memory trade-off. Note that the template engine will do this automatically for you, too.

> Frameworks (e.g. jQuery) via Google / Another CDN

This contradicts an earlier point about minimising HTTP requests. Ultimately, if you have gzip and good cache headers, you may as well include jQuery in your minified JS file. That connection will be opened anyway. Also, using CDNs to deliver jQuery can actually slow your site if they are unavailable (unlikely but possible) or blocked by the ISP/country of the visitor (more likely).

yeah - totally agree here. ​



> Use IDs rather than classes for CSS + JS references (faster to lookup)

tl;dr: Although IDs may be fractionally faster, they aren't worth the trade-offs

This is really a trade-off between more maintainable and "modular" CSS/JS and speed, and when we say "speed" we mean micro optimisations here. Using an ID for CSS is so minutely slower, it's not worth the pain of it's specificity. It's best to keep a flat class structure (any kind of nested rules are slower) further reading: http://benfrain.com/css-performance-revisited-selectors-bloat-expensive-styles/. If you're interested in modular CSS that's fast and flexible, I'd recommend reading up on BEM.

The same is true with JS - I'd say it's quite rare that you'd ever have a JS component that can safely hook into an ID (ie: you only need it once on any page) and the cons of tightly coupling your JS to your IDs, again, make this a pain. An interesting article and it's test results actually show class selectors are faster?!

Anyway, these kinds of optimisations for the front-end are really only important if you're running a massive front-end app OR have very large datasets/DOM trees to work with.

> Ajaxify forms - only load forms on request using ajax techniques

I'm not sure if I follow, if you mean lazy-load forms like one could with images this may be quite bad for people using assistive technologies (like screen readers) that expect to be able to find a form on the page when loaded.

for example, if
​the design has an​
 enquiry form on every page then why not add a button:

[enquire here]

then when the user clicks the button you ajax load the form from the server.  This means that you can have a more (or completely) static page - and still have a form as well (albeit that it is slightly harder to access). 
 

To add to server: HTTP 2.0 / turn on Keep-Alive; and Image resampling optimisation.

As with lots of these things (and particularly your front-end ones) the scalability, modularity and maintainability (of your development workflow) are often going to be at odds with some of the things you've suggested.

I strictly work to never using IDs (for selectors) and if we need an extra wrapper div to make a component modular or to add another styling hook, then we do. The trade-offs for front-end performance (for the work we do) is completely unnoticeable and using minification, image optimisation, gzip, caching and faster protocols will all make a far superior speed difference to the browser than replacing your CSS class selectors with IDs.

Last thought: Lots of your server optimisations are about moving things into memory - again, it's about being pragmatic (unless you have a huge quantity of memory available) when determining what can and can't be held in RAM. I probably wouldn't load by 5GB DB into memory on a small VPS (for example) especially if it involves using the page file as you'll just end up thrashing anyway.
Hope that's somewhat informative and useful.

​Thank you - very useful.

Olli Tyynelä

unread,
Jun 16, 2015, 8:07:43 AM6/16/15
to silverst...@googlegroups.com
My 5 cents :

Memcached:

by default cache only the zend lang files. At least on our tests > memcached doesn't support tagging and the version of the zend included doesn't have a class that would "fake" that support so it wont store all the data there. if anyone HAS got that work please tell me :)

APC:
As opcache is in php 5.5 directly there is now point running APC so what is the future for apc at the moment. Of course if you are running older PHP version then APC is a good choise. 

PHP:
fpm if you have the change to get most of the opcaching. 

The combination we are running at the moment that we can get a decent performance:

1) This lovely module https://github.com/heyday/silverstripe-cacheinclude . Allows chaching EASILY urls with params in them what is a time saver from having to write them your self. 
2) use redis as the store with that 
3) Our custom implementation on the navigation generation to speed up things if the cache happens to be invalidated . Basically we added a "path" information column simplifies the queries to get the tree nicely out. Something that Id love to see SS generate automatically. 


:Olli 







--

Daniel Hensby

unread,
Jun 16, 2015, 11:28:04 AM6/16/15
to silverst...@googlegroups.com
Of course, something to mention is php-hhvm, which will probably put all of this to bed as the single easiest way to get SS to be fast.

However, there's some work involved in getting it completely compatible, which simon welsh has done.

off...@netwerkstatt.at

unread,
Jun 16, 2015, 2:51:07 PM6/16/15
to silverst...@googlegroups.com

Hi,

 

My 2 cents:

 

Use nginx instead of apache, and a newer version of php (PHP 5.6 is a lot faster than 5.4…)

 

Use staticpublisher where it makes sense (no forms, no dynamic stuff…), or load forms with ajax (with fallback for js disabled browsers, screenreaders etc..)

 

Cheers,

 

Werner

Shea Dawson

unread,
Jun 17, 2015, 9:41:02 PM6/17/15
to silverst...@googlegroups.com, n...@sunnysideup.co.nz
Here's a couple of things I do that I don't think have been mentioned yet:

Optimise images - you can save quite a lot on image sizes without loosing quality. 
https://github.com/heyday/silverstripe-optimisedimage

Lazy loading images

Dynamic Cache - similar to static publisher 

In addition to combining global css/js into one file, I also put css/js that is only required on custom pages into their own files (ie. homepage.css) and use the following custom methods on my Page_Controller to get their contents into the page rather than the usual of adding an additional http request. This reduces the amount of css/js loaded/running on standard pages to only what is necessary/global, without having additional requests on custom pages.

Conrad Dobbs

unread,
Jun 17, 2015, 11:24:30 PM6/17/15
to silverst...@googlegroups.com, n...@sunnysideup.co.nz
If using Apache, turn off htaccess files with AllowOverride None. Can make a considerable difference.

Cam Findlay

unread,
Jun 18, 2015, 10:37:40 PM6/18/15
to silverst...@googlegroups.com, con...@webtorque.co.nz, n...@sunnysideup.co.nz
Anyone interested in putting together a "how to" around performance in the docs based on the suggestions here?

Be good to capture the various approaches used out in the real world to gain some effective speed boosts on SilverStripe.

Nicolaas Thiemen Francken - Sunny Side Up

unread,
Jun 19, 2015, 5:57:13 AM6/19/15
to silverstripe-dev
These are the notes so far.

network:

    • Cloudflare or another CDN (we had a bad experience with CF, but of course it makes sense to use a CDN) - especially if you have a global audience. 
    • Host in the same region as your audience.
    • Set manual expiration times (e.g. one month) for images, css, js.
    • Compress response files (gzip your html sort of thing).
    • Reduce requests by minifying / combining CSS and JS into one file (Requirements::combine_files()).
    • Browser cache the HTML where possible.

      server: 

      application

      • Use XHProf to identify your bottlenecks. Loading lots of objects in loops, or triggering lazyload events, can quickly pile on the amount of DB queries executed (thank you Marcus). 
      • Limit use of modules and add-ons in PHP.
      • Caching in PHP using private statics for custom methods that may be called more than once (e.g. if self::$foo is empty then fill it using the method below otherwise return self::$foo without further ado).
      • Partial caching in *.ss templates. Our basic rules it to cache every loop in the templates using <% cached foo %>.
      • Frameworks (e.g. jQuery) via Google / Another CDN

      client-side

      • Responsive image techniques - use CSS media queries to get the best image for the screen. 
      • Use IDs rather than classes for CSS + JS references (faster to lookup).
      • Use javascript rather than jQuery where appropriate/practicable).
      • Simplify html - reduce the number of nodes.
      • Ajaxify forms - only load forms on request using ajax techniques.
      • Write CSS without the use of compass and that sort of thing (keep CSS simple). 
      • Write considered JS that does not change the DOM too much (e.g. form validation - add error messages on first load then just add it to DOM with JS only showing it - not adding it). 

        references:



          If you think about a small site, with little traffic and a small budget vs a large site with a ton of traffic and a large budget then what tricks would you apply for the former and what tools would you look at as the site grows???  What are the ones above that offer the best gains for the least amount of hassle?

          Any more ideas?

          Nicolaas



          --
          You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to silverstripe-d...@googlegroups.com.
          To post to this group, send email to silverst...@googlegroups.com.
          Visit this group at http://groups.google.com/group/silverstripe-dev.
          For more options, visit https://groups.google.com/d/optout.



          --
          Nicolaas Thiemen Francken
            www.sunnysideup.co.nz
            phone: +64221697577

          Gordon Anderson

          unread,
          Jun 19, 2015, 6:43:15 AM6/19/15
          to silverst...@googlegroups.com, con...@webtorque.co.nz, n...@sunnysideup.co.nz
          hi

          There are some interesting tips here, and a few I did not know about so I guess some reading to do :)  Only fair to return the favor from my experiences.

          I've done a few jobs where I've had to improve the performance of a site and more often than not the issue was way too many database queries.  On one memorable job I was recording over 100,000 database queries just to render the home page, it turned out the programmers did not understand the concept of a join and did the join test by manually looping through SilverStripe DataObjects one by one :)


          1) Ensure there are no 404s.  Each file not found statically will be passed through to SilverStripe as it were potentially a page, resulting in effectively mostly repeat database queries to render the error page, also queries for checking the current Member and loading the SiteConfig.


          2) My goal in the above project was to minimize the number of database queries.  I noted when doing partial caching as per instructions on the SilverStripe site there were two issues:
          i) The LastEdited field is not indexed, meaning that when say rendering the four newest NewsItem objects on your home page, all of the records in that table would need to be checked.
          ii) For each section of a page that requires a partial, a database query is executed.
          I mitigated these factors with the following modules:
          i) https://github.com/gordonbanderson/weboftalent-index-lastedited - indexes LastEdited field for DataObject
          ii) https://github.com/gordonbanderson/weboftalent-cachekey-helper - group all queries for rendering partials into one single SQL statement

          Caveat: I've only recently come across https://github.com/heyday/silverstripe-cacheinclude and may well move to this once I have evaluated it.


          3) Traversing the SiteTree is expensive when rendering menus, with multiple repeated queries, for the gory details see https://github.com/silverstripe/silverstripe-framework/issues/2979 - note a technique to identify repeated SQL queries is included here.

          I think something similar to the old Ruby on Rails module acts_as_tree here would be good, where the depth and lineage of a page are stored.  So for example if we had a SiteTree structure like this:

          - Thailand (4)
              - Beaches (8)
                  - Chonburi (10)
                     - Rayong (12)
                     - Pattaya (14)
                  - Hua Hin (15)
              - Rivers (9)

          then the depth and lineage of the page titled 'Chonburi' would be 3 and likes of 000040000800010.  The reason for padding out the SiteTree ID to N figures (here 5) means that all descendant pages can be found using a single query.  For example to find all of the descendants of the page titled 'Beaches' the database query would be:

          SELECT * from SiteTree WHERE Lineage LIKE '00040008%';

          The only potential problem here is that moving a page in the SiteTree would be more expensive, though it may be possible to do a one shot SQL statement to update all the descendant pages.

          Olli, is the above similar to what you called "path information"?


          4) I don't use the Requirements functionality for CSS and JS in SilverStripe.  I've written a module to remove these files via a YAML config file, and then a separate one (which is a fork of Rails smart_assets) to compress CSS/JS and generate template files to include these.  The Dev templates includes all the CSS/JS separately, the Live ones includes only the packaged equivalents.

          If anybody is interested in this approach, I will tidy up the documentation on https://github.com/gordonbanderson/silverstripe-smart-asset and make the first module public.


          5) Avoid custom inline JavaScript, use unobtrusive JavaScript to initialize variables
          I came across this problem recently when updating the Mappable module, delaying the JavaScript execution as far as possible down the page.  If variables are primed using SilverStripe's custom JavaScript inclusion, then it cannot be included in a partial cache.  For example if the call to draw a map looks like this in YourTemplate.ss

          <% cached 'Map', ID, LastEdited %>
          $Map
          <% end_cached %>

          and the call to $Map method invokes the following PHP code:

          Requirements::customScript(<<<JS
            var latitude1 = 100;
            var longitude1 = 13,2;
          JS
          );

          then the variables above only appear when the $Map is not cached.

          The solution to this problem is to instead attach the coordinates as data attributes on a suitably identifiable div.

          Have a look at the source of http://www.jakayanrides.com/gallery/bangkok/20150503-exploring-klongs-in-tao-poon-area/ and do a search for "data-" for a live example.



          Cheers for all of the other tips on this page

          Regards

          Gordon

          Olli Tyynelä

          unread,
          Jun 19, 2015, 7:12:50 AM6/19/15
          to silverst...@googlegroups.com
          Forgot to link to this book previously and some more thoughts as I've got a lazy day now :)


          It has good tips on how to configure Linux to handle loads of requests and whatnots and how to setup opcache and what each setting actually means :)

          Best way to handle performance issues is to measure and identify where are the bottlenecks and figure out will you throw money on the hardware or on the code to fix it, either by caching or optimizing the code. And where to get information of performance/issues: try Newrelic (or similar).. Only bad thing in that is the pricing.

          Mostly it's about caching and heydeys cache-include module does it pretty well - it invalidates caches when you save a data object. So basically event based invalidation. Also it just is simple to use to cache forms that use query params.. just set the cache context to full :). We went from the dynamic cache module to heydeys as it gives easier granual control.

          And with cache: try to avoid filesystem as the cache backend if you can

          On php 7 and hhvm:
          Both will make codes soar.. But still there will be bits that do loads requests or to deep loops that go back and forth between database and template rendering and whatnots so you should still try to cache them or at least use a backend for the generated renders something else than the file system.

          :olli

          Lähetetty iPadista
          Reply all
          Reply to author
          Forward
          0 new messages