Search all of Google with Google Python API
remarkov via StackOverflow
Recently I was also looking for Google Search API and was misguided by a lot of outdated information. Here is what I found on Google Developers website: https://developers.google.com/api-client-library/python/apis/customsearch/v1
According to the docs your function will be something like
from googleapiclient.discovery import build
def google_results_count(query):
service = build("customsearch", "v1",
developerKey="[put your API key here]")
result = service.cse().list(
q=query,
cx='[put your CSE key here']
).execute()
return result["searchInformation"]["totalResults"]
print google_results_count('Python is cool')
Unfortunately, using CSE API will give you different result count from the one you get using web search. In the example above I got 2 680 000 in Python and approx. 21 000 000 for the same query on Google.com Here is an explanation why: https://support.google.com/customsearch/answer/70392?hl=en
Getting the API and CSE keys and all the limitations of CSE is a whole different story, I highly recommend you looking at this answer: http://stackoverflow.com/a/11206266/1704272 and the next one below for the alternatives.
Another approach is to parse the HTML response from Google.com which will give you the most complete results but it is not very reliable because Google changes the HTML markup. And more important this is against their TOS, more to read here: Is it ok to scrape data from Google results?
My conclusion. You have three options:
- Use Google CSE API (free). Use this, if you need to stay legal and you are sure you won't exceed the limit. Can not be used in public application.
- Use paid API (Google or any other, less expensive). It is legal to use this for any public application but be ready to pay for that.
- Scrape Google web page. This will give you the best results but I would use this option only for private needs.
Please DO NOT REPLY directly to this email but go to StackOverflow:
http://stackoverflow.com/questions/35125696/search-all-of-google-with-google-python-api/35706692#35706692
remarkov via StackOverflow
Recently I was also looking for Google Search API and was misguided by a lot of outdated information. Here is what I found on Google Developers website: https://developers.google.com/api-client-library/python/apis/customsearch/v1
According to the docs your function will be something like
from googleapiclient.discovery import build
def google_results_count(query):
service = build("customsearch", "v1",
developerKey="[put your API key here]")
result = service.cse().list(
q=query,
cx='[put your CSE key here']
).execute()
return result["searchInformation"]["totalResults"]
print google_results_count('Python is awesome')