Googlebot dissing my GWT app

197 views
Skip to first unread message

rhodebump

unread,
Oct 10, 2012, 2:02:05 PM10/10/12
to google-we...@googlegroups.com
Hi,
I have been trying for weeks, but can't get google to update it's index for my GWT application.   If i do a google search of my site (site:www.spotmouth.com), only my first page comes up.   I have read (many many times) the docs for https://developers.google.com/webmasters/ajax-crawling/

To summarize, I followed the steps:
1) Updated my site to use the ajax crawlable link format (i.e. http://www.spotmouth.com/#menu  ->  http://www.spotmouth.com/#!menu)
2) Implemented htmunit on serverside so that server responds with static copy:  http://www.spotmouth.com/?_escaped_fragment_=menu  
3) updated http://www.spotmouth.com/sitemap.xml to use ajax crawable links. (ie http://www.spotmouth.com/#!menu)

But if i do a google search of my site (site:www.spotmouth.com), only my first page comes up.  Nothing else.
I will literally ship a case of beer to whoever helps me fix this (if your state allows it, and I can get it delivered).

If I use the "Fetch as googlebot" tool in webmaster tools for " http://www.spotmouth.com/#!menu", it looks good, that is, a static copy of the content is returned.


Thanks,
Phillip

Chris Lercher

unread,
Oct 12, 2012, 6:18:23 AM10/12/12
to google-we...@googlegroups.com
A Google search for "site:www.spot...com" [replace the ellipsis] just returns "www.spot...com".

But you don't have that page in your sitemap, and you're not using step 3 "Handle pages without hash fragments" of the Step-by-step guide:

In order to make pages without hash fragments crawlable, you include a special meta tag in the head of the HTML of your page. The meta tag takes the following form:
<meta name="fragment" content="!">

Your main page is a dynamic JS-page. Googlebot can't see your links (like #driver_reports), as they don't appear in the static content of your main page (it doesn't know about "www.spot...com/#!home" yet, only about "www.spot...com".

Sitemaps just complement a discovery crawl (so that's probably not enough). Only with the meta tag, googlebot will go to "www.spot...com?_escaped_fragment_="

rhodebump

unread,
Oct 12, 2012, 8:52:32 AM10/12/12
to google-we...@googlegroups.com
I will give that a shot (adding the meta tag).  I did not think that was nessary since all my links on that page did use the hash bang syntax.  Thanks very much.  If this works, I literally owe you a case of beer.

rhodebump

unread,
Oct 15, 2012, 8:09:24 AM10/15/12
to google-we...@googlegroups.com
That meta tag did the trick.  Thanks very much!!!!
Reply all
Reply to author
Forward
0 new messages