Account Options

  1. Sign in
The old Google Groups will be going away soon.
Switch to the new Google Groups.
Google Groups Home
« Groups Home
Discussions > Crawling, indexing, and ranking > Google don't respect my robots.txt... WHY?
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  8 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Gio23  
View profile  
 More options Nov 28 2008, 2:01 pm
From: Gio23
Date: Fri, 28 Nov 2008 11:01:32 -0800 (PST)
Local: Fri, Nov 28 2008 2:01 pm
Subject: Google don't respect my robots.txt... WHY?
The Googlebot don't respect my robots.txt - i told him not to index my
videos, now already about 1,5 month since the 1st october and the
Googlebot do not deindex the videos...nothing is happening... WHY?

the video *.swf files are all in the folder /flv/ and are all
blocked...
http://www.italien-inseln.de/robots.txt
User-agent: *
Disallow: /flv/

the video *.fla files are also blocked
http://video.italien-inseln.de/robots.txt
User-agent: *
Disallow:     /

why does the Googlebot don't respect my robots.txt?

here you see still my videos...
http://video.google.de/videosearch?q=site%3Aitalien-inseln.de&hl=de&e...

here are my earlier postings, which does not effect anything... and
i'm asking me WHY?

where can i get help?

http://groups.google.com/group/google-video-issues/browse_thread/thre...

http://groups.google.de/group/google-video-issues-de/browse_thread/th...

http://groups.google.com/group/Google_Webmaster_Help-Indexing/browse_...

http://groups.google.com/group/google-video-general/browse_thread/thr...

http://groups.google.com/group/Google_Webmaster_Help-Indexing/browse_...

please, what can i do?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
BBdeath  
View profile  
 More options Nov 28 2008, 2:14 pm
From: BBdeath
Date: Fri, 28 Nov 2008 11:14:09 -0800 (PST)
Local: Fri, Nov 28 2008 2:14 pm
Subject: Re: Google don't respect my robots.txt... WHY?
With robots.txt you only disallowed crawling, not indexing.

On Nov 28, 7:01 pm, Gio23 wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
jevstar  
View profile  
 More options Nov 28 2008, 2:18 pm
From: jevstar
Date: Fri, 28 Nov 2008 11:18:17 -0800 (PST)
Local: Fri, Nov 28 2008 2:18 pm
Subject: Re: Google don't respect my robots.txt... WHY?
Something is happening because of you click on your videos on Google
they are not there. All that remains is the link and the thumbnails.

Are the thumbnails coming from you. If so I would block or move them
too.

Regards
JV

On Nov 28, 7:01 pm, Gio23 wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Gio23  
View profile  
 More options Nov 29 2008, 2:21 am
From: Gio23
Date: Fri, 28 Nov 2008 23:21:37 -0800 (PST)
Local: Sat, Nov 29 2008 2:21 am
Subject: Re: Google don't respect my robots.txt... WHY?
Thanks for your replies, i blocked everything... Thumbnails, Videos
and so on...

the Thumbnails are on a Google Server
like that

http://3.gvt0.com/ThumbnailServer2?app=vss&contentid=a21bd9b1c98c288f...

and so on... i removed all via GWT and i blocked all via robots.txt...
i renamed all files so Google gets an 404 Error BUT the videos are
still indexed ... i don't know why?

On 28 Nov., 20:18, jevstar wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Gio23  
View profile  
 More options Nov 29 2008, 2:26 am
From: Gio23
Date: Fri, 28 Nov 2008 23:26:17 -0800 (PST)
Local: Sat, Nov 29 2008 2:26 am
Subject: Re: Google don't respect my robots.txt... WHY?
ok, but if i block them via robots.txt they didn't have to show up on
search results right?
but they appear in Google Video Search and in the regular Search ...

somewhere in the support is written that i have to block them via
robots.txt and then they will
dissappear over time ... but i'm waiting now somewhat like 2 month ...
and that is crazy ...

i'm frustrated and i do not know why Google do not make them
dissappear ... the Googlebot got already the
robots.txt and he knows that i do not want them crawling or appearing
in Google Search or Video Search...

thanks in advance
Gio

On 28 Nov., 20:14, BBdeath wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
JohnMu Google employee  
View profile  
 More options Nov 29 2008, 8:31 am
From: JohnMu
Date: Sat, 29 Nov 2008 05:31:26 -0800 (PST)
Local: Sat, Nov 29 2008 8:31 am
Subject: Re: Google don't respect my robots.txt... WHY?
Hi Gio

I've passed this on to the video search team to look at (and fix :-)).
I don't know how soon you will see changes, but now that they're
informed things should generally change soon. If you don't see a
change in a few weeks, please post back here again.

Cheers
John


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Gio23  
View profile  
 More options Nov 29 2008, 9:20 am
From: Gio23
Date: Sat, 29 Nov 2008 06:20:05 -0800 (PST)
Local: Sat, Nov 29 2008 9:20 am
Subject: Re: Google don't respect my robots.txt... WHY?
Thanks so much JohnMu,

i hope it will be fixed all and i do not need to post again here...

thanks
Gio

On 29 Nov., 14:31, JohnMu wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
GaryTheScubaGuy  
View profile  
 More options Dec 1 2008, 10:38 am
From: GaryTheScubaGuy
Date: Mon, 1 Dec 2008 07:38:53 -0800 (PST)
Local: Mon, Dec 1 2008 10:38 am
Subject: Re: Google don't respect my robots.txt... WHY?
Wouldn't it be easier to use the noindex command?

Gary

On Nov 29, 2:20 pm, Gio23 wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »