Suggestion: Crawling API
1 view
Skip to first unread message
Sérgio Nunes
May 27, 2008, 9:13:43 PM5/27/08
to Google App Engine
Hi,
I would like to suggest the support of a Crawling API in Google App
Engine.
It would really bring Google's top expertise to developers.
Thanks for listening,
--
Sérgio Nunes
I would like to suggest the support of a Crawling API in Google App
Engine.
It would really bring Google's top expertise to developers.
Thanks for listening,
--
Sérgio Nunes
Filip
May 29, 2008, 1:04:46 AM5/29/08
to Google App Engine
Hi Sergio,
Could you expand on what you'd like to do with that? Even if there was
background processing, and you could buy more quota, crawling would
take enormous CPU time and hence be extremely expensive to do.
Filip
Could you expand on what you'd like to do with that? Even if there was
background processing, and you could buy more quota, crawling would
take enormous CPU time and hence be extremely expensive to do.
Filip
Patrick Boyd
May 29, 2008, 4:40:38 AM5/29/08
to Google App Engine
I think you'd be better off using wget or rolling something more
powerful in python (or nutch) on your home machine or EC2. What
problems are you running into with crawling that could be helped by an
API?
powerful in python (or nutch) on your home machine or EC2. What
problems are you running into with crawling that could be helped by an
API?
Reply all
Reply to author
Forward
0 new messages