SEO Improvements

[Amy Stephen]

I looked back at my SEO patches from August and offer it for
discussion as to whether or not it makes sense to consider for Joomla!
1.6.

It's fair to say that there is no magic bullet for SEO. Some might
believe software can, by itself, improve SEO and tend to look to the
application, rather than to focus on content and off-site promotion
where real gains can be found. We probably can't fix that. These
proposals, however, address standard, recurring requests, and there
does seem to be support from reputable SEO experts to have this
functionality available. If implemented, there might be minor, not
major, improvements.

Also, it must be recognized that each can be implemented with a
plugin. I have done this with Tamka. The downside is that there are
performance considerations when rewriting the buffer to add or remove
meta, However, none of these features require a core hack.

1. "SEO: HTML Page Titles"

Appends Site Name (or an optional value for Site Name) before or after
that Page Name on each page.

Value: Search results carry page name in the linked text. There is
value to add Site Name to that that. Such is also the case with social
bookmarking, pinging, standard bookmarking, etc. So, instead of only
having a page title, like "We won an award", the linked text could be
"Joomla! - We won an award.") I think this would provide decent
benefit to most sites.

Cons: Plugins are available (Ercan has a very popular plugin for
this). Increased Global Configuration complexity.

1.5 Tracker: http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=12448


2. "SEO: Global Robots and Custom Meta" 12452

a. Allows Global Options for Robots to a) follow or no follow and b)
index or no index - pages for the site. (Currently, it is always
"follow, index")

Pro: Provides choice for site-wide robot follow/index, rather than
assume follow/index. Would fulfill an often requested option. Plugin
solutions require rewrite of the page from the buffer to offer option.
Core hacks are used as solution now.

Cons: Fairly special case to want nofollow or noindex. Would
complicate User interface.

Additional consideration: We might want to consider robots more
broadly. There have also been requests for this functionality at the
menu items and with Categories.

Reference: http://www.robotstxt.org/meta.html

b. Allows entry of Custom Site-wide Meta Named Pairs (for example, one
could add Copyright information to add this type of statement:

<meta name="Copyright" content= "Open Source Matters, Inc." />

Pros: Offers control over site meta not currently available. While
plugin solutions can be made to avoid hack, those solutions would
require rewriting page from buffer. (Of course, I might be missing a
more efficient approach.)

Cons: Fairly special case. Complicates User interface.

Reference: http://www.robotstxt.org/meta.html

1.5 Tracker: http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=12452


3. "SEO: No WWW"

Forces 301 redirect to non-WWW.

Pros: Industry standard. Easier than configuring an .htaccess file

Cons: Complication. Would require SEF URL and .htaccess file used.
(People may not understand that).

Reference: http://no-www.org/
http://www.mattcutts.com/blog/seo-advice-url-canonicalization/
You can instruct Google http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=44231
- but that only takes care of Google - and it doesn't address social
bookmarking or other search services, etc.

1.5 Tracker: http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=12449


Would be good to hear Steve Burge's feedback on this, too. (And other
ideas for improving SEO in 1.6).

Discussion welcome. Thanks.

[Amy Stephen]

I have a plugin that offers several options for Permalinks for content, contacts, web links and newsfeeds for Tamka. I personally prefer this type of URL, see it as important for sharing links, whether it's social bookmarks or pings or "here's our link to our sales catalog." I think they are industry standard now. (Down the road, when you think of getting data ready for mashup, permanent URLs are requisite. Example - Flickr URI.)

With Joomla!'s router overrides, one can create plugins that attach one's own rules for parsing and creating URLs. So, the ability is there to do so without hacking core or creating a set of tables to store the URL, etc.. (This will be very helpful for people wanting to create their own mashable data and API.)

WordPress has true Permalinks - every page will have one and only one URL ever, end of subject. So, if you create a post and assign it to several categories, and you use the "category-slug" in the URL, the first "category" created is "category-slug" used. As with everything else, Permalinks have both pro's and con's. The limitations imposed are what is creating that challenge to really become a CMS. It's a simple architecture. (Elegant, useful, but has real limits.)

Drupal has a nice set of SEF URL options that were patterned after WordPress. I'd certainly call those Pretty URLs, but not necessarily Permalinks. The reason I say that is one can still use Drupal to place content into multiple menus and create multiple URLs for that article. That provides flexibility.

Joomla!'s URL situation is much more like Drupal in that sense, the difference is that Joomla! has ugly URLs with that primary key embedded, and no options for varying the URLs.

"True" Permanent URLs come at a cost to the architecture. Once committed to one Article, one URL, a lot of what's cool flexible about Joomla! becomes rigid. Right now, we can place an Article into several locations on the Web site and, because of the distinct URL, we can treat that Article as a distinct page and "decorate it" with different modules. Only possible because there is a different non-permanent URL.

It's also important to note that you can use Joomla! to create Permanent URLs but you have to know how. (Not saying that's ideal, just saying it's true.)

I've given this a lot of thought as I've worked on this plugin. I don't recommend Permanent URLs for Joomla! 1.6. I am certain we will see plugins emerge when people see what can be done. It will be helpful for the community to see what those restrictions amount to when building sites. For 2.0, I think there is no question that Pretty Permanent URL options will be required as an option in core.

Related to this is ensuring a unique alias value for Articles, Contacts, Newsfeeds and Web links. I. For Tamka, I also have a plugin that fires onBeforeContentSave. It determines which URL option was selected and enforces uniqueness in the alias slug.

A more doable approach might be possible for acheiving Pretty URLs in 1.6. If we could set a goal to get rid of article IDs in the slug for all menu items - and use triggers to ensure the alias for these content types is unique to the category level, I think most people would be satisfied with that step while the broader architecture issues are considered. If that sounds like something we'd like to try, I would love to help with it.

Anyway, those are some of my thoughts on URLs.

Steve, do you see any benefit to the patches I am describing? Are there worthy SEO improvements there?

As always, thanks for your contributions.

On Mon, Feb 2, 2009 at 6:02 AM, Stephen Burge <st...@alledia.com> wrote:

Hi Amy

One idea that has been intriguing me lately is permalinks.

They're an old and familiar part of Wordpress, Blogger, Drupal and other systems. Full list here:
http://en.wikipedia.org/wiki/Permalink#Popular_permalink_formats

I know that this would solve a lot of issues from SEO and duplicate content to Itemid problems.

I wonder what kind of new issues might it create, if any?

Steve

[Rob Schley]

Amy,

Thanks for sharing your thoughts on this topic. Most of these issues are things we've all dealt with in some form or another. I've implemented "no follow" options in some of my extensions to allow for more specific control over search engines and have, on more than one occasion, appended the site name to the HTML title.

My personal preferences would be to implement an option that allowed administrators to configure the site name to be prepended/appended to the document title. I think this could be done with one global configuration option. Alternatively, it could be exposed as a template option but I think I am more inclined to provide a global configuration option. Further, I think we should be able to control search engine instructions (index, follow) on a per menu item level. It would not be hard to add these options to the System Parameters provided in the menu manager (once the menu manager is fixed, that is).

As for no-www, I'm inclined to defer that to htaccess (and the like) rules. Yes, it is slightly harder but it if they are that concerned about SEF issues, they have some familiarity with htaccess and we could easily provide the necessary htaccess rule in the distribution's htaccess.txt file, commented out.

Lastly, for permalinks, while I am fully supportive of the concept and have been a proponent for it in several discussions about 2.0, I don't think it can be solved properly until 2.0. In my mind, 2.0 should merge some of the concepts in Wordpress and Drupal to create truly reliable and unique permanent URLs but until the site's structure has been decoupled from the menu structure at the core level, it is not really practical within the core.

Also, Steve, you should join in on the list discussion. It is going to prove difficult to keep up if you leave it to the mindfulness of others to include you on the reply. =)

Best,

Rob

[Amy Stephen]

On Tue, Feb 3, 2009 at 6:39 PM, Rob Schley <rob.s...@community.joomla.org> wrote:

Amy,

Thanks for sharing your thoughts on this topic. Most of these issues are things we've all dealt with in some form or another. I've implemented "no follow" options in some of my extensions to allow for more specific control over search engines and have, on more than one occasion, appended the site name to the HTML title.

My personal preferences would be to implement an option that allowed administrators to configure the site name to be prepended/appended to the document title. I think this could be done with one global configuration option.

Yes - that's what the patch does.
 

Alternatively, it could be exposed as a template option but I think I am more inclined to provide a global configuration option. Further, I think we should be able to control search engine instructions (index, follow) on a per menu item level. It would not be hard to add these options to the System Parameters provided in the menu manager (once the menu manager is fixed, that is).

As for no-www, I'm inclined to defer that to htaccess (and the like) rules. Yes, it is slightly harder but it if they are that concerned about SEF issues, they have some familiarity with htaccess and we could easily provide the necessary htaccess rule in the distribution's htaccess.txt file, commented out.

K.
 

Lastly, for permalinks, while I am fully supportive of the concept and have been a proponent for it in several discussions about 2.0, I don't think it can be solved properly until 2.0.

Agree.
 

[Rob Schley]

Amy,

I tried taking a look at the HTML page titles patch but it seems a bit outdated and no longer applies directly on the 1.6 trunk. I reviewed it (on a high level) by studying the diff for now.

Was there a specific reason to use a predefined set of separators versus allowing the administrator to enter a separator (like in breadcrumbs, for example)?

A minor issue, the patch will need a general cleanup because we've decided on a new standard for language keys (in that we've decided to actually follow the INI standard) so punctuation and the like will have to removed from the keys and underscores used instead of spaces... and spaces should probably be used instead of just joining everything into difficult to understand keys like "TIPPAGETITLEONLYONDEFAULTPAGE".

A bigger issue, in the HTML document head renderer, the patch assumes Itemid 1 is the default menu item which is not necessarily true. Any Itemid can be the default home page and so the document renderer will have to get the correct Itemid of the default menu item. There is an interesting coupling that comes with that which some might feel is an issue as well. I am not sure how I feel about the option of allowing control of the homepage as a special condition in that context; I'd like to hear other people's thoughts on it though.

Best,

Rob Schley

[Rob Schley]

Minor clarification, "... and spaces should probably be used instead of just joining everything into difficult to understand keys like..." should be "... and underscores should probably be used instead of just joining everything into difficult to understand keys like...".

*slaps himself on the wrist*

Rob

[Amy Stephen]

On Tue, Feb 3, 2009 at 10:12 PM, Rob Schley <rob.s...@community.joomla.org> wrote:

Amy,

I tried taking a look at the HTML page titles patch but it seems a bit outdated and no longer applies directly on the 1.6 trunk. I reviewed it (on a high level) by studying the diff for now.

It is very old - probably six months. Wanted to see what interest there was, if any, before starting over for 1.6.
 

Was there a specific reason to use a predefined set of separators versus allowing the administrator to enter a separator (like in breadcrumbs, for example)?

No.
 

A minor issue, the patch will need a general cleanup because we've decided on a new standard for language keys (in that we've decided to actually follow the INI standard) so punctuation and the like will have to removed from the keys and underscores used instead of spaces... and spaces should probably be used instead of just joining everything into difficult to understand keys like "TIPPAGETITLEONLYONDEFAULTPAGE".

 

Yes.

A bigger issue, in the HTML document head renderer, the patch assumes Itemid 1 is the default menu item which is not necessarily true.

Yea, that's not good. :-P Have no idea what I might have been thinking if I hardcoded Itemid 1.
 

Any Itemid can be the default home page and so the document renderer will have to get the correct Itemid of the default menu item. There is an interesting coupling that comes with that which some might feel is an issue as well. I am not sure how I feel about the option of allowing control of the homepage as a special condition in that context; I'd like to hear other people's thoughts on it though.

The reason for "default page" logic is to avoid "My Web Site - My Web Site" when both the default page and the Site Name have the same values.

It is also possible that the "Site Name" is different than the "default home" Page Name. In such cases, people might want both values.

Hence, the choice.

If we don't offer the choice, I'd recommend not using the "Site Name" for the "default page." (The simplicity would make sense to me.)

Thanks!
 

[Amy Stephen]

On Tue, Feb 3, 2009 at 10:16 PM, Rob Schley <rob.s...@community.joomla.org> wrote:

Minor clarification, "... and spaces should probably be used instead of just joining everything into difficult to understand keys like..." should be "... and underscores should probably be used instead of just joining everything into difficult to understand keys like...".

*slaps himself on the wrist*

Rob

Agreed. I don't use those massive, uppercase abbreviations for language literals anymore. I used to think it was cool. ;-)

[Amy Stephen]

I just realized Steve's response is not in this list. He must have
responded to me and not the list.

My comments about permalinks were in response to his suggestion
permalinks be added to 1.6.

He must have just selected respond to author, accidentally. I wonder
if he got my response?

[Arno Zijlstra]

One thing that would already give more flexibility is to move HEAD
rendering from the framework to truly being a file in the template
directory. I've had this discussion before and one of the reasons said
that it was not possible was the dynamic adding of css and js files. I
can't believe this is something that can't be solved because other
CMSes have these files in the template directories and are doing the
same kind of things.
The head of a file is part of the html and should be in the template
directory so the designer/builder has easy access to make changes for
each specific site.

Arno

[Gergo Erdosi]

What do you mean by dynamically? JDocument has two methods (addScript
and addStyleSheet) for adding js and css files to the header. Aren't
you looking for these methods?

Gergo

[Amy Stephen]

Gergo, Arno is saying that he wants more control of the <head> section. One of the reasons given for the current setup is the need to provide developers ability to add JS and CSS, using the commands you shared.

Arno - other than adding custom meta - what other limitations do you have - what can't you do in the <head> section without a hack to core?

[Gergo Erdosi]

I still don't see clearly what's the problem with this way. If you
want to add a js file for example from a component, then
JDocument::addScript is an easy and good way. A developer should know
how to use the API, so it shouldn't be a problem. Of course for
average users this is not an easy way, but they are not developing
components. And if you want to add it from the template, then you can
simple include it under <jdoc:include type="head" />, this requires
only HTML knowledge, nothing else. However I agree, that a better
control on META tags would be good.

Gergo

On Feb 12, 3:26 pm, Amy Stephen <amystep...@gmail.com> wrote:
> Gergo, Arno is saying that he wants more control of the <head> section. One
> of the reasons given for the current setup is the need to provide developers
> ability to add JS and CSS, using the commands you shared.
>
> Arno - other than adding custom meta - what other limitations do you have -
> what can't you do in the <head> section without a hack to core?
>

[Stian Didriksen]

The code that renders the content that goes in <jdoc:include
type="head" /> is, as you say, a part of the framework.
I agree this should be a part of the template. That is to say, the
template should control this.
If you want to change the order of the content in there, or have more
control over it, you got two options as far as I know.

The first, and isn't recommended, is to hack the core files that
render the head data.
Second, create a system plugin that manipulate the head with the
getHeadData and setHeadData APIs.

I solution I can think of is making the render of the head work
similar to modules. In specific module chrome.
For example we could have a simple file in the html folder called
head.php.

The code of file could be more or less a copy of head.php in the
directory "libraries/joomla/document/html/renderer.

With this method we would get the same kind of flexibility as we get
with layout overrides and module chrome, and is in fact doable.
I dare anyone to prove me wrong ;)

Regards,
Stian Didriksen

[Louis Landry]

On the contrary, I dare you to prove it right :)

- Louis

--
Development Coordinator
Joomla! ... because open source matters.
http://www.joomla.org

[Amy Stephen]

Stian -

I understand Custom Meta is a missing capability - are there other things you have wanted to do within the <head> section, but were unable to do?

Still trying to get a grasp on the problem we are trying to solve.

Thanks!
Amy

[Stian Didriksen]

Considering the number of methods available to change, control or
manipulate the data that goes in the head section there is nearly
nothing we can't do.
For example there are many ways of removing Mootools if you prefer
jQuery.
But by moving it to the template, it cold be done in a cleaner and
easier way.

Although, even though there are few things you would want to change in
the head rendering function, the ability to override it and customize
it are useful.
More freedom and flexibility isn't a bad thing isn't?

There is a common analogy that goes like this, "If something's not
broken, don't fix it". But I believe more in, "if it can be improved,
improve it".
In other words, I don't believe there are any big problems or things
that need to be fixed with this function. But I believe it can be
improved.

So don't grasp the problem, but try seeing if my suggestion would have
enough benefits to be well worth the effort :)

Regards,
Stian Didriksen

[Ian MacLennan]

Hmmm...  I really haven't bothered much with the head element except to add headers for XRDS discovery and stuff like that.  Can you provide a use case?  I'm sure there is room for improvement, but deciding how is best done when one knows what we're trying to accomplish.

Ian

[Hannes Papenberg]

I currently don't see the necessity to change the current system here. I
have not heard one argument that would make me believe, something in the
head is not ok the way it is now. I heard rumours about improved SEO
when using some head tags in a special order, but the people that I've
discussed this with were as unbelieving as me. I also don't see a
situation, where you would need something in the head that could not be
done so by just adding it in the template or by using the correct API.
As long as nobody puts forward a use-case which currently can not be
(nicely) solved, I would oppose to changing this. And if you give us new
arguments, please add some links to support this view.

Hannes

Ian MacLennan schrieb:

[Stian Didriksen]

My suggestion was to prove it is possible to move this part of the
framework to the template directory.
So, as Arno said,

"The head of a file is part of the html and should be in the template
directory so
the designer/builder has easy access to make changes for each specific

site.".

This is not about if the whole function needs to be changed in any
way. As far as my experience goes as a extension developer and
template developer goes, I have never came to the stage where I have
needed to change these files. Even when I have needed to remove
Mootools, I have many other solutions than change the rendering to
solve that.

But I'm no SEO expert, so I can't answer for those parts. This is
about the ability to change the rendering. For those who for whatever
reasons need to do so, without hacking or complicated solutions.
As Arno said, other CMSs have the head rendering function within the
template. I can't imagine this is without good reasons.

Personally I can imagine few reasons for why you would want to change
that function, but that's because of my own limited knowledge. But my
suggestion on how you could add that control to the template is based
on how other parts of the template works. And would give you the
flexibility you would need to do whatever you want to do. Without
hacking the core which I believe where the goal in Arno's scenario.

I agree that if there are no real need to change the rendering of the
head, then time shouldn't be wasted on moving it to the template.
But again, my SEO knowledge is limited, so I don't know how you could
improve the SEO by changing the rendering of meta tags in Joomla from
the default one, and I have nearly no experience with other CMS's than
Joomla(so I don't know which onesdo the rendering of the head in the
template) but if you can prove that wrong I would believe you.

If you can prove that adding a function which would allow overriding
the rendering of the head from the template as in my suggestion would
be a regression I agree with you.
I would never oppose to a good change, no matter the small or
insignificant just because it isn't major enough. As long as the
effort to do the change isn't too large.

Regards,
Stian Didriksen

[Anthony Ferrara]

The reasoning is that presently (1.5 and Trunk) there is no method to get the data from the head after all modules are rendered (you can get components), but before the head is rendered...  So you're left using regex instead on the head buffer in 1.6 (onBeforeRender). 

The the use case?  To be able to do any number of things (minify JS/CSS anyone)?

[Arno Zijlstra]

Thats the typical coder response Hannes, besides whether you want todo
something with it or not for a template, designers like to have
control over as much html as possible from within the template
directory just like you want as much control over code.

A simple example would be building a J site in html4 strict instead of
xhtml:

html4:
<meta name="author" content="Hannes Papenberg">
xhtml
<meta name="author" content="Hannes Papenberg" />

see the missing slash at the end for html4

or look at http://dublincore.org source and notice the difference.

I can see the reason Anthony is giving and if there are real issues
well than it's not happening yet but that doesn't change the fact that
that piece of html that is now in the framework should really be in
the template or at least an override.

Arno

[Ian MacLennan]

Thank you for providing a use case.

This makes sense to me, and I would support making this possible so long as it wasn't too complicated (which I wouldn't think it would be).

Ian

[Gergo Erdosi]

Actually I'm not sure if you could create a HTML 4 site with Joomla,
we use a lot of code in the framework which is designed for XHTML, for
example we have a function which replaces <br> tags with <br /> and
there is no way to change this at template level unless you modify the
core. However I agree according the comments that an optional head.php
file would be good in the template directory to get more control over
the header somehow.

Gergo

On Feb 12, 9:05 pm, Arno Zijlstra <arno.alva...@gmail.com> wrote:
> Thats the typical coder response Hannes, besides whether you want todo
> something with it or not for a template, designers like to have
> control over as much html as possible from within the template
> directory just like you want as much control over code.
>
> A simple example would be building a J site in html4 strict instead of
> xhtml:
>
> html4:
> <meta name="author" content="Hannes Papenberg">
> xhtml
> <meta name="author" content="Hannes Papenberg" />
>
> see the missing slash at the end for html4
>

> or look athttp://dublincore.orgsource and notice the difference.

[Rob Schley]

I'd love to see this part of the system changed. A few reasons:

This is one of the few places in the framework that actually requires you to hack a file if you want to change/remove something basic like, say, the generator tag. 

I strongly dislike how some of the stuff is automatically rendered by the system and some things are static in the template.

Example:

<jdoc:include type="head" />

<link rel="stylesheet" href="<?php echo $this->baseurl ?>/templates/rhuk_milkyway/css/template.css" type="text/css" />

...

If you want to do anything with all of the JavaScript files or with all of the CSS files, you can't unless you run a regex over the output buffer to find them. A great use case for this is automatic compression of CSS and JavaScript when caching is enabled.

Solving this, properly, is probably not as easy as it sounds. There are a few things that come into play: One, there probably needs to be a default way of rendering the head. Two, it should be API based. Three, the API needs to be easy enough for a designer (think JHtml not JDocument).

Best,

Rob

[Louis Landry]

Frankly the concept is a good one, and it would be nice to be able to manipulate hte document head in a similar way to other aspects of the page.  1.6 giving us triggers before document compilation will be a big step in the right direction, but as Arno points out this doesn't get us to a place that lets you do a HTML 4 valid document without hacking.  I am honestly much more concerned with HTML 5 than 4 and would love to have solid support for that as things move forward.

That being said, solving that particular problem could be something as easy as a setting in the template which switches the output behavior from XHTML to HTML.  That doesn't mean its the best solution or the one we will use... but its possible and probably simple.  Working out a way to allow an override of those behaviors in a simple manner strikes me as a somewhat difficult proposition.  I would welcome thoughts on how this might be accomplished and would welcome even more some proof of concept code we could look at based on the 1.6 trunk.

- Louis