> Long-time site in Google and no previous problems. Only started paying
> attention to Google's webmaster tools as they've improved them this
> year (thank you!).

> Just noticed that Google hasn't had a successful crawl on my site
> since Sept. 5. The error message for the main page is:

> "We can't currently access your home page because of a timeout."

> The site is online and functional 24/7 with significant traffic (even
> from Google's Adsensebot crawl servers, no problemo). But Googlebot
> has nearly stopped completely, down from hundreds of pages crawled per
> day to just a dozen or two since 9/5.

> So without any further information, I decided I'd better upload a
> sitemap and see if that would give me further info. Well, it did:

> "URL timeout: DNS lookup timeout"

> The problem is, DNS lookups are fine for my domain. We use a third-
> party enterprise DNS hosting service that lists 5 geographically-
> diverse DNS nameservers for our domain. Naturally, these are all up
> and running fine. DNStools reports no problems with either our domain
> or its DNS.

> Looking in the log files for Googlebot instances shows that the few
> times it is crawling these days, it's downloading the files fine
> (e.g., byte count and result code is what is expected and what they've
> always been).

> So a few suggestions come to mind from this experience:

> 1. Google should add more detail to the default error message that is
> reported on the dashboard's main page for a URL about the specific
> timeout issue. I only discovered additional detailed information about
> the error from uploading a sitemap.

> 2. Error messages are fairly useless to a webmaster unless they
> suggest a troubleshooting procedure. Luckily, I know the
> troubleshooting procedure for DNS timeouts and now I've done
> everything I can do on my side of things... What's next? How do I
> contact Google about this problem? (I assume this is it.)

> 3. A button that says, "Okay, I've fixed the error, please clear your
> cache and crawl again next time you're scheduled to do so" would be
> great. I assume it's not there because Google is *always* crawling and
> automatically rechecks these things on its own. That's cool, but it
> appears my domain is stuck in some sort of error cache thing or
> something on Google's search crawl side (while Google Adsensebot
> continues to churn along just fine). I hope/assume this will fix
> itself, since there doesn't appear to be anything else I can do on my
> end.

> 4. Google's internal systems and bots should be able to talk to one
> another. Google Adsensebot continues to function fine, while Googlebot
> thinks my domain's DNS is broken. If Googlebot could talk to Google
> Adsensebot, they'd say, "Hey, I'm getting through just fine. Try
> hitting it from a different IP address or something and clear your
> error cache."

> John

> Long-time site in Google and no previous problems. Only started paying
> attention to Google's webmaster tools as they've improved them this
> year (thank you!).

> Just noticed that Google hasn't had a successful crawl on my site
> since Sept. 5. The error message for the main page is:

> "We can't currently access your home page because of a timeout."

> The site is online and functional 24/7 with significant traffic (even
> from Google's Adsensebot crawl servers, no problemo). But Googlebot
> has nearly stopped completely, down from hundreds of pages crawled per
> day to just a dozen or two since 9/5.

> So without any further information, I decided I'd better upload a
> sitemap and see if that would give me further info. Well, it did:

> "URL timeout: DNS lookup timeout"

> The problem is, DNS lookups are fine for my domain. We use a third-
> party enterprise DNS hosting service that lists 5 geographically-
> diverse DNS nameservers for our domain. Naturally, these are all up
> and running fine. DNStools reports no problems with either our domain
> or its DNS.

> Looking in the log files for Googlebot instances shows that the few
> times it is crawling these days, it's downloading the files fine
> (e.g., byte count and result code is what is expected and what they've
> always been).

> So a few suggestions come to mind from this experience:

> 1. Google should add more detail to the default error message that is
> reported on the dashboard's main page for a URL about the specific
> timeout issue. I only discovered additional detailed information about
> the error from uploading a sitemap.

> 2. Error messages are fairly useless to a webmaster unless they
> suggest a troubleshooting procedure. Luckily, I know the
> troubleshooting procedure for DNS timeouts and now I've done
> everything I can do on my side of things... What's next? How do I
> contact Google about this problem? (I assume this is it.)

> 3. A button that says, "Okay, I've fixed the error, please clear your
> cache and crawl again next time you're scheduled to do so" would be
> great. I assume it's not there because Google is *always* crawling and
> automatically rechecks these things on its own. That's cool, but it
> appears my domain is stuck in some sort of error cache thing or
> something on Google's search crawl side (while Google Adsensebot
> continues to churn along just fine). I hope/assume this will fix
> itself, since there doesn't appear to be anything else I can do on my
> end.

> 4. Google's internal systems and bots should be able to talk to one
> another. Google Adsensebot continues to function fine, while Googlebot
> thinks my domain's DNS is broken. If Googlebot could talk to Google
> Adsensebot, they'd say, "Hey, I'm getting through just fine. Try
> hitting it from a different IP address or something and clear your
> error cache."

> John

> Same here!!

> Suddenly Google is giving us an ERROR message when we try to update
> our sitemap.xml files on Google Webmaster Tools.

> Following is my online chat with Adam, our hosting company's support
> rep.

> Question: Google seems to be having a problem accessing our files.
> Each of our sites is getting a "URL timeout." We know for certain that
> all our files are right. Bluehost support rep the other day told me
> that it was probably an issue that our non-Bluehost domain registrars
> need to fix, but in fact that is very unlikely. Most likely is that we
> need Bluehost to adjust folder permissions or firewall settings to
> allow Google access.

> Adam: Hello! Welcome to our live chat. Please allow me a moment to
> read over your question. Keep in mind that we take multiple chats at
> one time so my response to your questions may take a few moments.

> Lawrence: Google seems to be having a problem accessing our files.
> Each of our sites is getting a "URL timeout." We know for certain that
> all our files are right. Bluehost support rep the other day told me
> that it was probably an issue that our non-Bluehost domain registrars
> need to fix, but in fact that is very unlikely. Most likely is that we
> need Bluehost to adjust folder permissions or firewall settings to
> allow Google access.

> Adam: As long as you don't have a robots.txt file blocking google you
> should be fine. We wouldn't block google's bot for any reason.

> Lawrence: Hmmm... However, it's not fine. :-(
> Here's the error message:
> "URL timeout: DNS lookup timeout We encountered an error while trying
> to access your Sitemap. Please ensure your Sitemap follows our
> guidelines and can be accessed at the location you provided and then
> resubmit"
> Are you familiar with Google sitemaps.xml files?

> Adam: Yes.

> Lawrence: We have four different websites hosted on Bluehost.
> I setup sitemap.xml files on each last year to help Google.
> They were able to accept them without problem until recently.
> Nothing changed on our side.
> As you know, it's nearly impossible to open a support ticket with
> Google. They refer users to Google Groups instead.
> This issue was discussed by the Google Webmaster Help group

> Adam: Yeah, I've noticed that.

> Lawrence: each solution named something to do with hosting
> Have no other Bluehost clients encountered the issue?

> Adam: Not that I am aware of, no.

> Lawrence: You know, it's always a mystery how Google works, and we're
> all doing guesswork all the time.
> Some have suggested that since DNS is mentioned it may have to do with
> the domains' registrars
> But we have multiple registrars for different sites and I find it
> unlikely that all would become problematic at the same time. Know what
> I mean?
> It seems more likely that there would be a permissions or firewall
> snafu on Bluehost (where all these sites are hosted)
> Or, the problem is with Google, in which case I'll have to throw up my
> hands and just give up having a business. (and die penniless in the
> gutter somewhere)

> Adam: I'm afraid we really wouldn't gain anything from blocking
> googlebot :(

> Lawrence: Now, googlebot indexes web pages, right?

> Adam: Yes, that is the part of google that does that.

> Lawrence: but when we request Google to come and download our latest
> updated sitemap.xml do they index it the same way? same IP as their
> normal googlebot?
> what i mean is,

> Adam: Yes, although I've never seen it come from the same IP address.
> That's why the robots.txt file is important if you want to block it.

> Lawrence: i don't actually know whether they're indexing our web pages
> okay this week. maybe or maybe not, i don't know. But what i do know
> is that they are unable to access our sitemap.xml files
> we don't use robots.txt

> Adam: Are they in the same location you stated when you submitted the
> first time?

> Lawrence: yes
> in fact, 3 out of 4 of them haven't been changed at all since the last
> successful grabbing by Google
> so, i can only conclude that it's something changed at Google or
> something changed at Bluehost, (or some new and unfortunate
> interaction effect between the two)
> one day they're all fine, then suddenly all ERROR
> here's a blogger who encountered the same problemhttp://blog.mcnicholl.com/2007/02/15/google-sitemaps-url-timeout-dns-...

> Adam: Although I can't completely rule us out, I would suggest trying
> to get an answer out of google.

> Lawrence: hmmm... okay
> will i get a transcript of this chat after i close the window?

> Adam: I'll continue asking around, but I'm not aware of any change
> that we have made that would cause this.
> Yes, after you close the chat it will ask you if you want it to be
> emailed to you.

> Lawrence: Adam, thanks for your help so far... I'll see what response
> I get from Google
> bye for now

> Adam: Anytime, let us know how it goes.

> Naively, I tried sending my question to [email address]

> What I got back was:

> "Thank you for writing to Google. We'd like to assist you, but we only
> respond to messages submitted through our online contact form. Please
> visithttp://www.google.com/support/to submit your message, and we'll
> get
> back to you soon. We apologize for any inconvenience, and we look
> forward
> to hearing from you.

> Regards,
> The Google Team"

> So, when you go to google.com/support looking for the "online contact
> form" to "submit your message," guess what you find? A never-ending
> network of FAQs and no contact form anywhere where you can submit your
> message.

> Hmmmmmm.... We're doomed....

> On Sep 10, 3:12 pm, LawrenceJT wrote:

> > Same here!!

> > Suddenly Google is giving us an ERROR message when we try to update
> > our sitemap.xml files on Google Webmaster Tools.

> > Following is my online chat with Adam, our hosting company's support
> > rep.

> > Question: Google seems to be having a problem accessing our files.
> > Each of our sites is getting a "URL timeout." We know for certain that
> > all our files are right. Bluehost support rep the other day told me
> > that it was probably an issue that our non-Bluehost domain registrars
> > need to fix, but in fact that is very unlikely. Most likely is that we
> > need Bluehost to adjust folder permissions or firewall settings to
> > allow Google access.

> > Adam: Hello! Welcome to our live chat. Please allow me a moment to
> > read over your question. Keep in mind that we take multiple chats at
> > one time so my response to your questions may take a few moments.

> > Lawrence: Google seems to be having a problem accessing our files.
> > Each of our sites is getting a "URL timeout." We know for certain that
> > all our files are right. Bluehost support rep the other day told me
> > that it was probably an issue that our non-Bluehost domain registrars
> > need to fix, but in fact that is very unlikely. Most likely is that we
> > need Bluehost to adjust folder permissions or firewall settings to
> > allow Google access.

> > Adam: As long as you don't have a robots.txt file blocking google you
> > should be fine. We wouldn't block google's bot for any reason.

> > Lawrence: Hmmm... However, it's not fine. :-(
> > Here's the error message:
> > "URL timeout: DNS lookup timeout We encountered an error while trying
> > to access your Sitemap. Please ensure your Sitemap follows our
> > guidelines and can be accessed at the location you provided and then
> > resubmit"
> > Are you familiar with Google sitemaps.xml files?

> > Adam: Yes.

> > Lawrence: We have four different websites hosted on Bluehost.
> > I setup sitemap.xml files on each last year to help Google.
> > They were able to accept them without problem until recently.
> > Nothing changed on our side.
> > As you know, it's nearly impossible to open a support ticket with
> > Google. They refer users to Google Groups instead.
> > This issue was discussed by the Google Webmaster Help group

> > Adam: Yeah, I've noticed that.

> > Lawrence: each solution named something to do with hosting
> > Have no other Bluehost clients encountered the issue?

> > Adam: Not that I am aware of, no.

> > Lawrence: You know, it's always a mystery how Google works, and we're
> > all doing guesswork all the time.
> > Some have suggested that since DNS is mentioned it may have to do with
> > the domains' registrars
> > But we have multiple registrars for different sites and I find it
> > unlikely that all would become problematic at the same time. Know what
> > I mean?
> > It seems more likely that there would be a permissions or firewall
> > snafu on Bluehost (where all these sites are hosted)
> > Or, the problem is with Google, in which case I'll have to throw up my
> > hands and just give up having a business. (and die penniless in the
> > gutter somewhere)

> > Adam: I'm afraid we really wouldn't gain anything from blocking
> > googlebot :(

> > Lawrence: Now, googlebot indexes web pages, right?

> > Adam: Yes, that is the part of google that does that.

> > Lawrence: but when we request Google to come and download our latest
> > updated sitemap.xml do they index it the same way? same IP as their
> > normal googlebot?
> > what i mean is,

> > Adam: Yes, although I've never seen it come from the same IP address.
> > That's why the robots.txt file is important if you want to block it.

> > Lawrence: i don't actually know whether they're indexing our web pages
> > okay this week. maybe or maybe not, i don't know. But what i do know
> > is that they are unable to access our sitemap.xml files
> > we don't use robots.txt

> > Adam: Are they in the same location you stated when you submitted the
> > first time?

> > Lawrence: yes
> > in fact, 3 out of 4 of them haven't been changed at all since the last
> > successful grabbing by Google
> > so, i can only conclude that it's something changed at Google or
> > something changed at Bluehost, (or some new and unfortunate
> > interaction effect between the two)
> > one day they're all fine, then suddenly all ERROR
> > here's a blogger who encountered the same problemhttp://blog.mcnicholl.com/2007/02/15/google-sitemaps-url-timeout-dns-...

> > Adam: Although I can't completely rule us out, I would suggest trying
> > to get an answer out of google.

> > Lawrence: hmmm... okay
> > will i get a transcript of this chat after i close the window?

> > Adam: I'll continue asking around, but I'm not aware of any change
> > that we have made that would cause this.
> > Yes, after you close the chat it will ask you if you want it to be
> > emailed to you.

> > Lawrence: Adam, thanks for your help so far... I'll see what response
> > I get from Google
> > bye for now

> > Adam: Anytime, let us know how it goes.

> This is pretty horrible.

> My site is hosted with ThePlanet... so this is not confined to one
> host. The issue seems to either be on Google's end, OR they started
> somehow strictly enforcing some rules that in the past they were lax
> with (E.g., making sure your DNS record is perfect).

> Let's keep bumping this up... I spent about an hour trying to find the
> right contact form and came up with *nothing*... I hope Google staff
> monitors this forum-- this is important and we really need your help
> to get this straightened out!

> On Sep 9, 11:31 pm, LawrenceJT wrote:

> > Naively, I tried sending my question to [email address]

> > What I got back was:

> > "Thank you for writing to Google. We'd like to assist you, but we only
> > respond to messages submitted through our online contact form. Please
> > visithttp://www.google.com/support/tosubmit your message, and we'll
> > get
> > back to you soon. We apologize for any inconvenience, and we look
> > forward
> > to hearing from you.

> > Regards,
> > The Google Team"

> > So, when you go to google.com/support looking for the "online contact
> > form" to "submit your message," guess what you find? A never-ending
> > network of FAQs and no contact form anywhere where you can submit your
> > message.

> > Hmmmmmm.... We're doomed....

> > On Sep 10, 3:12 pm, LawrenceJT wrote:

> > > Same here!!

> > > Suddenly Google is giving us an ERROR message when we try to update
> > > our sitemap.xml files on Google Webmaster Tools.

> > > Following is my online chat with Adam, our hosting company's support
> > > rep.

> > > Question: Google seems to be having a problem accessing our files.
> > > Each of our sites is getting a "URL timeout." We know for certain that
> > > all our files are right. Bluehost support rep the other day told me
> > > that it was probably an issue that our non-Bluehost domain registrars
> > > need to fix, but in fact that is very unlikely. Most likely is that we
> > > need Bluehost to adjust folder permissions or firewall settings to
> > > allow Google access.

> > > Adam: Hello! Welcome to our live chat. Please allow me a moment to
> > > read over your question. Keep in mind that we take multiple chats at
> > > one time so my response to your questions may take a few moments.

> > > Lawrence: Google seems to be having a problem accessing our files.
> > > Each of our sites is getting a "URL timeout." We know for certain that
> > > all our files are right. Bluehost support rep the other day told me
> > > that it was probably an issue that our non-Bluehost domain registrars
> > > need to fix, but in fact that is very unlikely. Most likely is that we
> > > need Bluehost to adjust folder permissions or firewall settings to
> > > allow Google access.

> > > Adam: As long as you don't have a robots.txt file blocking google you
> > > should be fine. We wouldn't block google's bot for any reason.

> > > Lawrence: Hmmm... However, it's not fine. :-(
> > > Here's the error message:
> > > "URL timeout: DNS lookup timeout We encountered an error while trying
> > > to access your Sitemap. Please ensure your Sitemap follows our
> > > guidelines and can be accessed at the location you provided and then
> > > resubmit"
> > > Are you familiar with Google sitemaps.xml files?

> > > Adam: Yes.

> > > Lawrence: We have four different websites hosted on Bluehost.
> > > I setup sitemap.xml files on each last year to help Google.
> > > They were able to accept them without problem until recently.
> > > Nothing changed on our side.
> > > As you know, it's nearly impossible to open a support ticket with
> > > Google. They refer users to Google Groups instead.
> > > This issue was discussed by the Google Webmaster Help group

> > > Adam: Yeah, I've noticed that.

> > > Lawrence: each solution named something to do with hosting
> > > Have no other Bluehost clients encountered the issue?

> > > Adam: Not that I am aware of, no.

> > > Lawrence: You know, it's always a mystery how Google works, and we're
> > > all doing guesswork all the time.
> > > Some have suggested that since DNS is mentioned it may have to do with
> > > the domains' registrars
> > > But we have multiple registrars for different sites and I find it
> > > unlikely that all would become problematic at the same time. Know what
> > > I mean?
> > > It seems more likely that there would be a permissions or firewall
> > > snafu on Bluehost (where all these sites are hosted)
> > > Or, the problem is with Google, in which case I'll have to throw up my
> > > hands and just give up having a business. (and die penniless in the
> > > gutter somewhere)

> > > Adam: I'm afraid we really wouldn't gain anything from blocking
> > > googlebot :(

> > > Lawrence: Now, googlebot indexes web pages, right?

> > > Adam: Yes, that is the part of google that does that.

> > > Lawrence: but when we request Google to come and download our latest
> > > updated sitemap.xml do they index it the same way? same IP as their
> > > normal googlebot?
> > > what i mean is,

> > > Adam: Yes, although I've never seen it come from the same IP address.
> > > That's why the robots.txt file is important if you want to block it.

> > > Lawrence: i don't actually know whether they're indexing our web pages
> > > okay this week. maybe or maybe not, i don't know. But what i do know
> > > is that they are unable to access our sitemap.xml files
> > > we don't use robots.txt

> > > Adam: Are they in the same location you stated when you submitted the
> > > first time?

> > > Lawrence: yes
> > > in fact, 3 out of 4 of them haven't been changed at all since the last
> > > successful grabbing by Google
> > > so, i can only conclude that it's something changed at Google or
> > > something changed at Bluehost, (or some new and unfortunate
> > > interaction effect between the two)
> > > one day they're all fine, then suddenly all ERROR
> > > here's a blogger who encountered the same problemhttp://blog.mcnicholl.com/2007/02/15/google-sitemaps-url-timeout-dns-...

> > > Adam: Although I can't completely rule us out, I would suggest trying
> > > to get an answer out of google.

> > > Lawrence: hmmm... okay
> > > will i get a transcript of this chat after i close the window?

> > > Adam: I'll continue asking around, but I'm not aware of any change
> > > that we have made that would cause this.
> > > Yes, after you close the chat it will ask you if you want it to be
> > > emailed to you.

> > > Lawrence: Adam, thanks for your help so far... I'll see what response
> > > I get from Google
> > > bye for now

> > > Adam: Anytime, let us know how it goes.

> Our case is the same. We've made no changes to our DNS, no changes to
> our server configuration, nothing. We're hosted in a large, mainstream
> hosting facility, and our DNS is hosted by a very large, enterprise-
> grade DNS hosting service (that we've had zero problems with).

> So my original instinct was, "Oh, this is just a server networking
> problem on Google's end. They'll figure it out."

> But as it comes up on a week and I notice Googlebot is still staying
> away, that's becoming a concern.

> As I said, we've run the DNSreports tool at dnsstuff.com and it comes
> back perfectly clean and okay. Not a thing wrong or misconfigured.

> Hoping for a response from Google! :)

> On Sep 10, 4:08 am, armenb wrote:

> > This is pretty horrible.

> > My site is hosted with ThePlanet... so this is not confined to one
> > host. The issue seems to either be on Google's end, OR they started
> > somehow strictly enforcing some rules that in the past they were lax
> > with (E.g., making sure your DNS record is perfect).

> > Let's keep bumping this up... I spent about an hour trying to find the
> > right contact form and came up with *nothing*... I hope Google staff
> > monitors this forum-- this is important and we really need your help
> > to get this straightened out!

> > On Sep 9, 11:31 pm, LawrenceJT wrote:

> > > Naively, I tried sending my question to [email address]

> > > What I got back was:

> > > "Thank you for writing to Google. We'd like to assist you, but we only
> > > respond to messages submitted through our online contact form. Please
> > > visithttp://www.google.com/support/tosubmityour message, and we'll
> > > get
> > > back to you soon. We apologize for any inconvenience, and we look
> > > forward
> > > to hearing from you.

> > > Regards,
> > > The Google Team"

> > > So, when you go to google.com/support looking for the "online contact
> > > form" to "submit your message," guess what you find? A never-ending
> > > network of FAQs and no contact form anywhere where you can submit your
> > > message.

> > > Hmmmmmm.... We're doomed....

> > > On Sep 10, 3:12 pm, LawrenceJT wrote:

> > > > Same here!!

> > > > Suddenly Google is giving us an ERROR message when we try to update
> > > > our sitemap.xml files on Google Webmaster Tools.

> > > > Following is my online chat with Adam, our hosting company's support
> > > > rep.

> > > > Question: Google seems to be having a problem accessing our files.
> > > > Each of our sites is getting a "URL timeout." We know for certain that
> > > > all our files are right. Bluehost support rep the other day told me
> > > > that it was probably an issue that our non-Bluehost domain registrars
> > > > need to fix, but in fact that is very unlikely. Most likely is that we
> > > > need Bluehost to adjust folder permissions or firewall settings to
> > > > allow Google access.

> > > > Adam: Hello! Welcome to our live chat. Please allow me a moment to
> > > > read over your question. Keep in mind that we take multiple chats at
> > > > one time so my response to your questions may take a few moments.

> > > > Lawrence: Google seems to be having a problem accessing our files.
> > > > Each of our sites is getting a "URL timeout." We know for certain that
> > > > all our files are right. Bluehost support rep the other day told me
> > > > that it was probably an issue that our non-Bluehost domain registrars
> > > > need to fix, but in fact that is very unlikely. Most likely is that we
> > > > need Bluehost to adjust folder permissions or firewall settings to
> > > > allow Google access.

> > > > Adam: As long as you don't have a robots.txt file blocking google you
> > > > should be fine. We wouldn't block google's bot for any reason.

> > > > Lawrence: Hmmm... However, it's not fine. :-(
> > > > Here's the error message:
> > > > "URL timeout: DNS lookup timeout We encountered an error while trying
> > > > to access your Sitemap. Please ensure your Sitemap follows our
> > > > guidelines and can be accessed at the location you provided and then
> > > > resubmit"
> > > > Are you familiar with Google sitemaps.xml files?

> > > > Adam: Yes.

> > > > Lawrence: We have four different websites hosted on Bluehost.
> > > > I setup sitemap.xml files on each last year to help Google.
> > > > They were able to accept them without problem until recently.
> > > > Nothing changed on our side.
> > > > As you know, it's nearly impossible to open a support ticket with
> > > > Google. They refer users to Google Groups instead.
> > > > This issue was discussed by the Google Webmaster Help group

> > > > Adam: Yeah, I've noticed that.

> > > > Lawrence: each solution named something to do with hosting
> > > > Have no other Bluehost clients encountered the issue?

> > > > Adam: Not that I am aware of, no.

> > > > Lawrence: You know, it's always a mystery how Google works, and we're
> > > > all doing guesswork all the time.
> > > > Some have suggested that since DNS is mentioned it may have to do with
> > > > the domains' registrars
> > > > But we have multiple registrars for different sites and I find it
> > > > unlikely that all would become problematic at the same time. Know what
> > > > I mean?
> > > > It seems more likely that there would be a permissions or firewall
> > > > snafu on Bluehost (where all these sites are hosted)
> > > > Or, the problem is with Google, in which case I'll have to throw up my
> > > > hands and just give up having a business. (and die penniless in the
> > > > gutter somewhere)

> > > > Adam: I'm afraid we really wouldn't gain anything from blocking
> > > > googlebot :(

> > > > Lawrence: Now, googlebot indexes web pages, right?

> > > > Adam: Yes, that is the part of google that does that.

> > > > Lawrence: but when we request Google to come and download our latest
> > > > updated sitemap.xml do they index it the same way? same IP as their
> > > > normal googlebot?
> > > > what i mean is,

> > > > Adam: Yes, although I've never seen it come from the same IP address.
> > > > That's why the robots.txt file is important if you want to blo