[clean-mx] searching for black-hats is not possible

[Gerhard W. Recher (abuse)]

hi @all

we have a public project in complaining about malware sites see:
http://support.clean-mx.de/clean-mx/viruses.php

now we want to use google to dig out certain url's but results are not
really amazing.

take this php code:

<?php
$url =
"http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=allinurl%3Aalbums%2Fbsblog%2Fcategory";
$key="our api key";
$ref="http://our referer";
$url.="&rsz=large&num=100&key=$key";
// sendRequest
// note how referer is set manually
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $ref);
$body = curl_exec($ch);
curl_close($ch);

// now, process the JSON string
$json = json_decode($body);
print_r($json);


this will result in exactly TWO !! results ... but if you make this in
firefox:


stdClass Object
(
[responseData] => stdClass Object
(
[results] => Array
(
[0] => stdClass Object
(
[GsearchResultClass] => GwebSearch
[unescapedUrl] => http://www.xn--schang-ptz-heb.de/bilder/albums/bsblog/category/nice/
[url] => http://www.xn--schang-ptz-heb.de/bilder/albums/bsblog/category/nice/
[visibleUrl] => www.xn--schang-ptz-heb.de
[cacheUrl] => http://www.google.com/search?q=cache:cD1vpmD6uvIJ:www.xn--schang-ptz-heb.de
[title] => Pascual Yehieli
[titleNoFormatting] => Pascual Yehieli
[content] => nice,. Published on: 10.10.2009 07:17 &middot; home hud las vegas. home hud las vegas. nice,. Published on: 07.10.2009 03:03 &middot; pictures of the model jessica biel <b>...</b>
)

[1] => stdClass Object
(
[GsearchResultClass] => GwebSearch
[unescapedUrl] => http://www.xn--schang-ptz-heb.de/bilder/albums/bsblog/category/old/
[url] => http://www.xn--schang-ptz-heb.de/bilder/albums/bsblog/category/old/
[visibleUrl] => www.xn--schang-ptz-heb.de
[cacheUrl] => http://www.google.com/search?q=cache:a1KjLAhqMFMJ:www.xn--schang-ptz-heb.de
[title] => Pascual Yehieli
[titleNoFormatting] => Pascual Yehieli
[content] => coooooool (28); pictures (33); sex (28); old (3); lol (37); funny (40); wow (19) ; HAHAHAHAHAH (36); super (20); cool (26); free (25); naked (2); pics (27) <b>...</b>
)

)

[cursor] => stdClass Object
(
[pages] => Array
(
[0] => stdClass Object
(
[start] => 0
[label] => 1
)

)

[estimatedResultCount] => 2
[currentPageIndex] => 0
[moreResultsUrl] => http://www.google.com/search?oe=utf8&ie=utf8&source=uds&start=0&hl=en&q=allinurl%3Aalbums%2Fbsblog%2Fcategory
)

)

[responseDetails] =>
[responseStatus] => 200
)





http://www.google.de/search?hl=de&source=hp&q=allinurl:albums/bsblog/category&btnG=Google-Suche&meta=&aq=f&oq=

you will get est. 280 Thousand ...

but also only 5 pages and at least only 10 unique urls !

this is ridiculous !

any glue how to overcome this ?

-- gerhard

[Jeremy Geerdes]

Unfortunately, this is a fairly command - and known - bug. The AJAX
Search API does not always return the same results that a standard
Google search will, especially when using special operators (e.g.,
inurl: ). There is no known workaround.

Jeremy R. Geerdes
Effective website design & development
Des Moines, IA

For more information or a project quote:
http://jgeerdes.home.mchsi.com
http://jgeerdes.blogspot.com
http://jgeerdes.wordpress.com
jgee...@mchsi.com

Unless otherwise noted, any price quotes contained within this
communication are given in US dollars.

If you're in the Des Moines, IA, area, check out Debra Heights
Wesleyan Church!

And check out my blog, Adventures in Web Development, at http://jgeerdes.blogspot.com
!

> --
>
> You received this message because you are subscribed to the Google
> Groups "Google AJAX APIs" group.
> To post to this group, send email to google-ajax...@googlegroups.com
> .
> To unsubscribe from this group, send email to google-ajax-searc...@googlegroups.com
> .
> For more options, visit this group at http://groups.google.com/group/google-ajax-search-api?hl=
> .
>
>

[Gerhard W. Recher (abuse)]

Hi Jeremy,

thx for quick response !
due to the sensitive nature of my problem/request i suggest for further
discussion *not* to use this list....

would be nice to get a solution from google "off-list", because I think
this matter is urgent and vital for a better internet at all !

-- gerhard

[Jeremy Geerdes]

Actually, the problem has been known for some time. There is an bug
filed on the issue tracker which you can star to follow. Check it out
at the link below. I would not hold my breath to hear from Google
directly and personally regarding a solution.

http://code.google.com/p/google-ajax-apis/issues/detail?id=32

Jeremy R. Geerdes
Effective website design & development
Des Moines, IA

For more information or a project quote:
http://jgeerdes.home.mchsi.com
http://jgeerdes.blogspot.com
http://jgeerdes.wordpress.com
jgee...@mchsi.com

Unless otherwise noted, any price quotes contained within this
communication are given in US dollars.

If you're in the Des Moines, IA, area, check out Debra Heights
Wesleyan Church!

And check out my blog, Adventures in Web Development, at http://jgeerdes.blogspot.com
!

[Gerhard W. Recher (abuse)]

Hi Jeremy,

"hold my breath..." indeed this is not recommended if we deal with
google ;-)

I also have negative experience with google malware list.

it's just a shame google does not respond in a timely fashion to serious
requests !