Improve SEO for automated links

67 views
Skip to first unread message

Guillaume Rossolini

unread,
Jan 8, 2016, 9:00:19 AM1/8/16
to mod-pagespeed-discuss
Hi,

Currently, mod_pagespeed includes "noscript" links in the source of optimized pages:
Please click <a href="https://www.example.com/?PageSpeed=noscript">here</a> if you are not redirected within a few seconds.</div>

Since they are <a> tags, crawlers can index these links, which can lead to SEO problems (duplicate content, for one). I have seen canonical metadata have no effect on Sitelinks, and the robots file might not be the best answer.
Is it possible to automatically add rel="noindex" on these links?

Thanks,

Jeff Kaufman

unread,
Jan 8, 2016, 9:39:08 AM1/8/16
to mod-pagespeed-discuss
Do you mean something like <a
href="https://www.example.com/?PageSpeed=noscript" rel="nofollow"> ?
Or that pages loaded with ?PageSpeed=noscript should get <meta
name="robots" content="noindex"> in their HEAD?
> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mod-pagespeed-discuss/c94fd98c-fe33-411a-adff-343aefb529ae%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Guillaume Rossolini

unread,
Jan 8, 2016, 9:53:50 AM1/8/16
to mod-pagespeed-discuss
I don't know about adding any meta information on the target page (HTML HEAD tag or HTTP header) since some people may be using these pages for their SEO, although I see no reason to do this.
But yes, either always add rel=noindex on these links, or add a configuration option to enable this.

I wouldn't use "nofollow" though but "noindex", or maybe both if the hostname is different. Or you could make that configurable too, with a sensible default value.
Also, I initially talked about PageSpeed=noscript links, but maybe it would be worth applying on all links?

What do you think? I'm not sure about the scope, there are so many possible strategies.

Regards,


On Friday, 8 January 2016 15:39:08 UTC+1, Jeff Kaufman wrote:
Do you mean something like <a
href="https://www.example.com/?PageSpeed=noscript" rel="nofollow"> ?
Or that pages loaded with ?PageSpeed=noscript should get <meta
name="robots" content="noindex"> in their HEAD?

On Fri, Jan 8, 2016 at 9:00 AM, Guillaume Rossolini
<guillaume...@instantluxe.com> wrote:
> Hi,
>
> Currently, mod_pagespeed includes "noscript" links in the source of
> optimized pages:
> Please click <a href="https://www.example.com/?PageSpeed=noscript">here</a>
> if you are not redirected within a few seconds.</div>
>
> Since they are <a> tags, crawlers can index these links, which can lead to
> SEO problems (duplicate content, for one). I have seen canonical metadata
> have no effect on Sitelinks, and the robots file might not be the best
> answer.
> Is it possible to automatically add rel="noindex" on these links?
>
> Thanks,
>
> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an

Jeff Kaufman

unread,
Jan 8, 2016, 1:03:17 PM1/8/16
to mod-pagespeed-discuss
I don't know very much about seo, but is rel=noindex on links a thing?
Looking now I'm not seeing that. Where are you seeing it
recommended?

On Fri, Jan 8, 2016 at 9:53 AM, Guillaume Rossolini
>> > email to mod-pagespeed-di...@googlegroups.com.
> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mod-pagespeed-discuss/62501fef-5c51-4928-ab88-6d3d0aaae726%40googlegroups.com.

Guillaume Rossolini

unread,
Jan 8, 2016, 1:46:55 PM1/8/16
to mod-pagespeed-discuss
Actually, you're right: rel=noindex is not a thing on links.

That means it would be better to have this as a meta information on the target page, which is safer in any case. Also, it should save some bandwidth, most notably as an HTTP header over HTTP/2.

In fact, it seems rel=noindex can be added as meta information on the target page in case any crawler has previously followed the URL, while rel=nofollow can be added to <a> tags to save a few server hits.

Regards,
>> > To view this discussion on the web visit
>> >
>> > https://groups.google.com/d/msgid/mod-pagespeed-discuss/c94fd98c-fe33-411a-adff-343aefb529ae%40googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an

Jeff Kaufman

unread,
Jan 11, 2016, 11:05:50 AM1/11/16
to mod-pagespeed-discuss
I opened https://github.com/pagespeed/mod_pagespeed/issues/1242 for
this, and then closed it after I saw that we were already inserting a
rel=canonical link.

What makes you think rel=canonical isn't enough? Are you seeing
?PageSpeed=noscript urls showing up in search results somewhere?
>> >> > email to mod-pagespeed-di...@googlegroups.com.
>> >> > To view this discussion on the web visit
>> >> >
>> >> >
>> >> > https://groups.google.com/d/msgid/mod-pagespeed-discuss/c94fd98c-fe33-411a-adff-343aefb529ae%40googlegroups.com.
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "mod-pagespeed-discuss" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an
>> > email to mod-pagespeed-di...@googlegroups.com.
> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mod-pagespeed-discuss/b0f4bd00-a7e1-47c4-97ef-2d338be62326%40googlegroups.com.

Guillaume Rossolini

unread,
Jan 11, 2016, 12:28:21 PM1/11/16
to mod-pagesp...@googlegroups.com
Hi Jeff,

As I said before, I have seen cases where rel=canonical was not enough to avoid indexing these pages. I'm not saying Google would index "?PageSpeed=noscript" pages, but these URL are part of the source code and therefore they can be shared, which means they *will* eventually be shared.

However, I didn't realize there was a rel=canonical already, because our app adds this meta in any case.

Thanks,

--
Guillaume ROSSOLINI
----------------------------
CTO - Directeur Technique

INSTANTLUXE.com
5 Rue Daunou | 75002 Paris | France
-----------------------------
Download "InstantLuxe" for free on the App Store.

You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/fsTpPYSs8g0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAMJ6YUsYrYXWVKhBHnfArpGs7weejjqy66kXqFHYMDYjKEwUeQ%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages