Blame & ROBOTS

[Matt Good]

I believe the reasoning for r4540 was to prevent search engines from
trying to index the annotation pages and incurring the performance hit
of building these pages. However, I don't think this is going to help,
since the search engine will actually have to visit the page in order
to get the <meta name="ROBOTS"> tag, so the page is still being
rendered.

Also, I think we should try to avoid too much abuse of the "and" and
"or" logic for flow-control in the templates.

This code:
<py:with vars="flags = (file and file.annotate) and 'NOINDEX, NOFOLLOW'
or dir and 'NOINDEX'">
<meta py:if="flags" name="ROBOTS" content="$flags" />
</py:with>

I think would be cleaner as:
<meta py:if="file and file.annotate" name="ROBOTS" content="NOINDEX,
NOFOLLOW" />
<meta py:if="dir" name="ROBOTS" content="NOINDEX" />

The "and" and "or" logic is very useful, but it can be harmful to
readability when overused in a single statement.

-- Matt Good

[Christian Boos]

Matt Good wrote:
> I believe the reasoning for r4540 was to prevent search engines from
> trying to index the annotation pages and incurring the performance hit
> of building these pages. However, I don't think this is going to help,
> since the search engine will actually have to visit the page in order
> to get the <meta name="ROBOTS"> tag, so the page is still being
> rendered.

The intent was not exactly that. Of course the annotated page will
eventually be loaded by the search engine, but then it won't follow the
many changeset links. This is the main "immediate" benefit of r4540 in
terms of performance hit. The second benefit is that this page is not
indexed, which means it shouldn't show up in search results, hence a
secondary benefit, as people coming from search sites will go to the
normal view of files instead of their annotated one.

First I thought about using a rel="nofollow" attribute in the
"Annotated" link to try to prevent search engines to trigger the build
of annotate pages, but then I learnt that the nofollow is badly named as
its intent is not to prevent following links, but rather to avoid adding
"weight" to those links
(http://microformats.org/wiki/rel-nofollow#open_issues), so it also
doesn't help. We could however use a <meta name="ROBOTS"
content="NOFOLLOW" /> in the file view, but that would also prevent
legitimate links in the file content to be followed.

> Also, I think we should try to avoid too much abuse of the "and" and
> "or" logic for flow-control in the templates.
>
> This code:
> <py:with vars="flags = (file and file.annotate) and 'NOINDEX, NOFOLLOW'
> or dir and 'NOINDEX'">
> <meta py:if="flags" name="ROBOTS" content="$flags" />
> </py:with>
>
> I think would be cleaner as:
> <meta py:if="file and file.annotate" name="ROBOTS" content="NOINDEX,
> NOFOLLOW" />
> <meta py:if="dir" name="ROBOTS" content="NOINDEX" />

Agreed, the above is clearer.

> The "and" and "or" logic is very useful, but it can be harmful to
> readability when overused in a single statement.

Well, there are places it's overused even more... those places could
perhaps benefit from the <?py ?> blocks once they're available.

-- Christian