Click data retrieval taking too long

[Rami Hani]

I'm experiencing difficulties with retrieving click view data efficiently. I'm currently running a brand campaign, resulting in a large number of daily clicks (over 500,000). Previously, I was able to retrieve the data successfully, but now the process takes more than an hour and eventually fails.

I'm using the Google Ads API v13 in Python, and here is the relevant code snippet:

START_DATE = datetime.date(2023, 5, 24)
END_DATE = datetime.date(2023, 5, 24)
DAY_DELTA = datetime.timedelta(days=1)

def main(client, customer_id, start_date=datetime.date.today() - datetime.timedelta(days=1),
         end_date=datetime.date.today() - datetime.timedelta(days=1), day_delta=datetime.timedelta(days=1)):
    google_ads_click_view = pd.DataFrame()
    while start_date <= end_date:
        ga_service = client.get_service("GoogleAdsService")
        query = """
            SELECT click_view.ad_group_ad,
            click_view.gclid,
            segments.click_type,
            customer.descriptive_name,
            campaign.id,
            campaign.name,
            campaign.start_date,
            campaign.status,
            campaign.end_date,
            ad_group.id,
            ad_group.name,
            ad_group.tracking_url_template,
            ad_group.type,
            click_view.keyword,
            click_view.keyword_info.match_type,
            click_view.keyword_info.text,
            click_view.page_number,
            click_view.location_of_presence.city,
            click_view.location_of_presence.country,
            click_view.location_of_presence.region,
            click_view.location_of_presence.most_specific
            FROM click_view
            WHERE segments.date = '{date}'
    """.format(date=start_date.strftime('%Y-%m-%d'))
       
        # Issues a search request using streaming.
        search_request = client.get_type("SearchGoogleAdsStreamRequest")
        search_request.customer_id = customer_id
        search_request.query = query
        stream = ga_service.search_stream(search_request)

Is there a solution to speed up the data retrieval process?
Thank you.

[Rami Hani]

This is the function call

client = GoogleAdsClient.load_from_storage("../google-ads.yaml")

try:
    clicks = main(client, CLIENT_CUSTOMER_ID, START_DATE, END_DATE)
except GoogleAdsException as ex:
    print(
        f'Request with ID "{ex.request_id}" failed with status '
        f'"{ex.error.code().name}" and includes the following errors:'
    )
    for error in ex.failure.errors:
        print(f'\tError with message "{error.message}".')
        if error.location:
            for field_path_element in error.location.field_path_elements:
                print(f"\t\tOn field: {field_path_element.field_name}")
    sys.exit(1)

[Google Ads API Forum Advisor]

Hi Rami,

 

Thank you for reaching out to the Google Ads API Forum Channel.

 

Please do note that the performance of operation in your code depends on the environment in which you are running your python program (e.g. OS Platform and Distribution, Python version, etc.). I would suggest reaching out to the client library developers for more insight on possible code-related optimizations for retrieving click view data efficiently. You may do so by going to the `Issues` tab of your Python client library's GitHub page and clicking `New issue` to open up a new case. Kindly navigate here: https://github.com/googleads/google-ads-python/issues.

 

As an aside, you can also refer to this guide <https://developers.google.com/google-ads/api/docs/best-practices/overview#optimization> for the Best Practices related to Optimization.

 

Let us know if you have any clarifications or if we can assist you with anything else related to the Google Ads API <https://developers.google.com/google-ads/api/docs/start>.

 

This message is in relation to case "ref:_00D1U1174p._5004Q2m99LZ:ref"

Thanks,
 

Google Ads API Team