I think you need to turn down the volume a little on the spyware/ automated requests challenges. I'm only performing normal keyword searches, no operators, 10 results per page, in my browser, and I'm increasingly getting challenged. Then on occasion the correct challenge is getting the 'sorry you look like spyware, please try again later' response....
Richard - I wrote a crawler for these forums a while back, so I'm intimately acquainted with the whole 'we think you are an automated robot' message. I came across two things that would piss their protection alg off -
1. Parallel get requests from the same IP. 2. Repeated get requests with < 20 secs between each get.
After alot of experimenting I got around it by running IPC across three different servers and running a round robin system.
But I've gotta say - it took a good half an hour of hard crawling without any limits before G got the poops and would throw that error - it was pretty difficult to attract the ire of their protection alg.
I'm wondering whether you might have some sort of download accelerator / caching software / firefox plugin that might be hammering their server with additional gets?
It could be worth using a tool like ethereal to be sure to be sure :)
> I think you need to turn down the volume a little on the spyware/ > automated requests challenges. I'm only performing normal keyword > searches, no operators, 10 results per page, in my browser, and I'm > increasingly getting challenged. Then on occasion the correct > challenge is getting the 'sorry you look like spyware, please try > again later' response....
Giving the puter a real thorough scan ATM - the warnings are now coming even when I haven't searched for substantial durations, and when using the toolbar search...
Will fire up ethereal and check some traffic patterns also.
> Richard - I wrote a crawler for these forums a while back, so I'm > intimately acquainted with the whole 'we think you are an automated > robot' message. I came across two things that would piss their > protection alg off -
> 1. Parallel get requests from the same IP. > 2. Repeated get requests with < 20 secs between each get.
> After alot of experimenting I got around it by running IPC across > three different servers and running a round robin system.
> But I've gotta say - it took a good half an hour of hard crawling > without any limits before G got the poops and would throw that error - > it was pretty difficult to attract the ire of their protection alg.
> I'm wondering whether you might have some sort of download > accelerator / caching software / firefox plugin that might be > hammering their server with additional gets?
> It could be worth using a tool like ethereal to be sure to be sure :)
> doc
> On Aug 26, 8:07 pm, Red Cardinal wrote:
> > Dear Google
> > I think you need to turn down the volume a little on the spyware/ > > automated requests challenges. I'm only performing normal keyword > > searches, no operators, 10 results per page, in my browser, and I'm > > increasingly getting challenged. Then on occasion the correct > > challenge is getting the 'sorry you look like spyware, please try > > again later' response....
I've seen these more and more as well... It's tough to analyze indexing issues where you do have to do a bunch of checks (it even triggers when you're doing things very slowly, as if certain combinations of keywords and parameters are signals, eg checking page 4 of 100 for a special keyword).
I think you should both lay off the double-double espresso in the mornings and have a calming chamomile tea instead - you're obviously so efficient that the alg thinks you couldn't possibly be human :P
> I've seen these more and more as well... It's tough to analyze > indexing issues where you do have to do a bunch of checks (it even > triggers when you're doing things very slowly, as if certain > combinations of keywords and parameters are signals, eg checking page > 4 of 100 for a special keyword).
Right, so I haven't used Google for about an hour, I type a site: search into Toolbar, I'm logged in under my own Google account, and I get told my query looks like spyware.
What type of user experience is this? Please Google, I've validated my identity by logging in to my Google account - why are you calling me spyware?
> I've seen these more and more as well... It's tough to analyze > indexing issues where you do have to do a bunch of checks (it even > triggers when you're doing things very slowly, as if certain > combinations of keywords and parameters are signals, eg checking page > 4 of 100 for a special keyword).
