updating from Cross Client Report Download (ReportUtils::RunAsyncReport) to ReportUtils::DownloadReport

[iateadonut]

The following code generates a report for me:

$user = new AdWordsUser();
$user->LogDefaults();

$reportDefinitionId = (float) '131128463';

$result = ReportUtils::RunAsyncReport($reportDefinitionId,
$queryToken, $user, $options);

This is the old CrossClient type of download.

This code, however:
$user = new AdWordsUser();
$user->LogDefaults();

$reportDefinitionId = '131128463';

ReportUtils::DownloadReport($reportDefinitionId, $path, $user,
$options);

produces:
The client customer ID must be specified for report downloads.

Fair enough, change the above to:

$user = new AdWordsUser();
$user->LogDefaults();
$user->SetClientId('3924112376');

$reportDefinitionId = '131128463';

ReportUtils::DownloadReport($reportDefinitionId, $path, $user,
$options);

produces:
[ReportDefinitionError.INVALID_REPORT_DEFINITION_ID @
selector.selector; errorDetails:reportDefinitionId=ReportDefinitionId:
131128463, customerId=27968988, changeId=0]

which seems very strange to me - the customerId (27968988) not only
doesn't match $user->SetClientId, but it is also only 8 digits, when I
think clientId's are supposed to be at least 9 digits long.

Please let me know what I'm doing wrong here (and also, when you do
set the clientId, how to set it for all of your subaccounts).

[Eric Koleda]

Hi,

The customer ID shown is the error is a different internal ID, which is why it doesn't match up.  It is however referring to the same account.  The problem here is that the report definition you are referencing was created on the MCC account, and it's available on the client account you are running it against.  If you are switching to v201109 you should be using ad-hoc reports instead:

http://code.google.com/apis/adwords/docs/reportingtopics.html#adhoc

Best,

- Eric Koleda, AdWords API Team

[iateadonut]

Thanks.

How do I download a report that has information for all my clients
with adHoc reports?

Also, through adHoc reports, selector 'Cost' is a microAmount? How
can I change that back to a dollar amount?

Thank you.

[Eric Koleda]

Hi,

Cross-client reporting functionality is no longer available in v201109, although there are ways to achieve the same results using a series of single-client reports:

http://adwordsapi.blogspot.com/2011/10/downloading-reports-for-lots-of-client.html

The header "returnMoneyInMicros" header allows you to control the format of the returned currency amounts:

http://code.google.com/apis/adwords/docs/reportingtopics.html

Best,

- Eric

[iateadonut]

My client has more than 300 sub-accounts that his company manages.

Sometimes, the sub=account may have no data returned when generating a
report.

I should just fetch each report anyway? 10 at a time and fetch 300
reports?

(Just making sure.) Thanks.

On Dec 1, 2:03 pm, Eric Koleda <eric.kol...@google.com> wrote:
> Hi,
>
> Cross-client reporting functionality is no longer available in v201109,
> although there are ways to achieve the same results using a series of
> single-client reports:
>

> http://adwordsapi.blogspot.com/2011/10/downloading-reports-for-lots-o...

[Kevin Winter]

Hi,

  Given that AdHoc reports cost 0 units, the cost of requesting the report to you as a developer is the CPU cycles (and IO) required to request it.  The cost to us from a server perspective is a bit more.  However, if you as a developer don't know at report time whether or not there is data to request, it makes sense to request it anyway.  Yes, go ahead and request the report anyway.

- Kevin Winter

AdWords API Team

[iateadonut]

How do you suggest requesting 10 reports at a time (from an array of
300 customerId's) using php? or is this impossible?

[Kevin Winter]

Hi,

  I'm not very familiar with PHP, but I believe one suggested approach is to use pcntl_fork to spawn extra processes and handle multiple concurrent report downloads that way: http://php.net/manual/en/function.pcntl-fork.php

- Kevin Winter

AdWords API Team

On Wednesday, December 14, 2011 11:33:59 PM UTC-5, iateadonut wrote:

How do you suggest requesting 10 reports at a time (from an array of
300 customerId's) using php?  or is this impossible?

[iateadonut]

Thanks,

I did it with pcntl_fork. (had a lot of different advice on this -
yours turned out to be the best)

The only problem is that it is consistently triggering Captcha's
(except one doing them one at a time, which takes forever). How do I
deal with these?

Here is the code. I will post it in the other discussion groups where
I gave advice pertinent on CrossClient Reports after I complete this
taking care of Captcha's.

<?php

error_reporting(E_STRICT | E_ALL);

// You can set the include path to src directory or reference
// AdWordsUser.php directly via require_once.
// $path = '/path/to/aw_api_php_lib/src';
include('../definedb.php'); //
include('./accountsArray.php'); //sets $adwordsAPIpath, uses $selector
= new ServicedAccountSelector(); to make an array of all accounts
//$accountsArray = array('4503399530');
//print_r($accountsArray);exit;
$path = $adwordsAPIpath.'/../../live_api2/src/';
set_include_path(get_include_path() . PATH_SEPARATOR . $path);

require_once 'Google/Api/Ads/AdWords/Lib/AdWordsUser.php';
require_once 'Google/Api/Ads/AdWords/Util/ReportUtils.php';

// Get AdWordsUser from credentials in "../auth.ini"
// relative to the AdWordsUser.php file's directory.
$user = new AdWordsUser();
//$user->SetClientId($customerId);

// Log SOAP XML request and response.
$user->LogDefaults();

// Load ReportDefinitionService so that the required classes are
available.
$user->LoadService('ReportDefinitionService', 'v201109');

// Create selector.
$selector = new Selector();
$selector->fields = array('ExternalCustomerId',
'AccountDescriptiveName', 'PrimaryUserLogin', 'Date', 'Id', 'Name',
'Impressions', 'Clicks', 'Cost');
//$selector->predicates[] =
//new Predicate('Status', 'IN', array('ENABLED', 'PAUSED'));

// Create report definition.
$reportDefinition = new ReportDefinition();
$reportDefinition->selector = $selector;
$reportDefinition->reportName = 'Campaign performance report #' .
time();
$reportDefinition->dateRangeType = 'LAST_7_DAYS';
$reportDefinition->reportType = 'CAMPAIGN_PERFORMANCE_REPORT';
$reportDefinition->downloadFormat = 'CSV';
$reportDefinition->includeZeroImpressions = FALSE;

$options = array('version' => 'v201109', 'returnMoneyInMicros' =>
FALSE);

$pids = array();

$j = 0;

for ($i=0; $i<count($accountsArray); $i++) {

if($j==10) { echo $j." ---\n"; $j=0; }

$pids[$i] = pcntl_fork();

if(!$pids[$i]) {

// child process

try {
// Get AdWordsUser from credentials in "../auth.ini"
// relative to the AdWordsUser.php file's directory.

$customerId = $accountsArray[$i];

$user->SetClientId($customerId);

$fileName = "7days-".$customerId.".csv";
$path = dirname(__FILE__) . '/../reports/7days/' . $fileName;

ReportUtils::DownloadReport($reportDefinition, $path, $user,
$options);

printf("Report with name '%s' was downloaded to '%s'.\n",
$reportDefinition->reportName, $fileName);

} catch (Exception $e) {
print $e->getMessage();
}

exit();
}
$j++;

[Anash P. Oommen]

Hi,

You get Captcha errors because you are not reusing the AuthTokens. The detailed technical blog is given below for reference, but the summary is that if you hit the ClientLogin endpoint too frequently, it will challenge you with a captcha. However, AuthTokens are long-lived, so the parent script should generate one authToken, and ask all the child processes to reuse it. 

See http://adwordsapi.blogspot.com/2010/07/discover-v2009-working-with-authtokens.html for more details.

Thanks,
Anash

[iateadonut]

I'm misunderstanding:

I do this command only once:
$user = new AdWordsUser();

and then I loop through my customerId's like this:
$user->SetClientId($customerId);

I thought only 'new AdWordsUser()' would generate a new AuthToken?

On Dec 16, 3:11 am, "Anash P. Oommen" <anash.p.oommen

> Seehttp://adwordsapi.blogspot.com/2010/07/discover-v2009-working-with-au...for
> more details.
>
> Thanks,
> Anash

[Kevin Winter]

Hi,

  In the PHP library, the tokens are generated in a lazy fashion, i.e. right before they are needed.  If you don't cause the AuthToken to get generated BEFORE the fork, then each forked process will generate its own, leading to CAPTCHA challenged.  You can call GetAuthToken() (http://code.google.com/p/google-api-adwords-php/source/browse/trunk/src/Google/Api/Ads/AdWords/Lib/AdWordsUser.php#260) on the AdWordsUser which forces creation.  Do this prior to the fork and each process will have a complete copy of the AdWordsUser object, AuthToken and all, which can then be used to make requests without fear of CAPTCHA challenges.

- Kevin Winter

AdWords API Team

[iateadonut]

OK,

changed:
$user = new AdWordsUser();

to:
$user = new AdWordsUser();

$user->GetAuthToken();

and i was able to run it three times without a captcha, so that seemed
to do it... not to mention it runs faster now.

thank you all for your help, Kevin, Anash, Eric

on a side note, debugging was a lot easier with CrossClients. if you
have a single report to download, then you can just check the latest
date in the mysql table you import it into and know that the whole
process went fine.

now you have to check for all of your clients, but if one of the
reports is empty, then it's harder to know if you're missing just a
little bit of the data or there was no data in a report, etc.

On Dec 16, 2:18 pm, Kevin Winter <Kevin.Win...@google.com> wrote:
> Hi,
>   In the PHP library, the tokens are generated in a lazy fashion, i.e.
> right before they are needed.  If you don't cause the AuthToken to get
> generated BEFORE the fork, then each forked process will generate its own,

> leading to CAPTCHA challenged.  You can call GetAuthToken() (http://code.google.com/p/google-api-adwords-php/source/browse/trunk/s...)

[Alexander Nitschke]

Hi Kevin,

can you please tell me how to do this with the ASP.NET library? I
really looked for AuthToken related methods or attributes but didn't
find anything.

Or another way of doing it (which I might even prefer) - is it
possible to login with the MCC account and download the report once
for each Customer without changing the AdwordsUser? Like having a
selector.predicate.field = "CustomerId" or similar?

Best wishes and thanks for any answer
Alexander

On Dec 16 2011, 8:18 pm, Kevin Winter <Kevin.Win...@google.com> wrote:
> Hi,
>   In the PHP library, the tokens are generated in a lazy fashion, i.e.
> right before they are needed.  If you don't cause the AuthToken to get
> generated BEFORE the fork, then each forked process will generate its own,

> leading to CAPTCHA challenged.  You can call GetAuthToken() (http://code.google.com/p/google-api-adwords-php/source/browse/trunk/s...)

[Kevin Winter]

Hi Alexander,

  Unfortunately, while the DotNet library does expose the Captcha challenge, it does not provide any way to solve it and get an AuthToken: http://code.google.com/p/google-api-adwords-dotnet/source/browse/trunk/src/Common/Lib/AuthToken.cs#184

Could you please file an issue on the library's tracker? http://code.google.com/p/google-api-adwords-dotnet/issues/list

We do recommend setting up an MCC account that has all accounts linked to it.  Then you can create an AdWordsUser with the email and password of the MCC account and just change the clientCustomerId  each time you want to download the report for a different user.

- Kevin Winter

AdWords API Team

On Friday, January 20, 2012 12:21:26 PM UTC-5, Alexander Nitschke wrote:

Hi Kevin,

can you please tell me how to do this with the ASP.NET library? I
really looked for AuthToken related methods or attributes but didn't
find anything.

Or another way of doing it (which I might even prefer) - is it
possible to login with the MCC account and download the report once
for each Customer without changing the AdwordsUser? Like having a
selector.predicate.field = "CustomerId" or similar?

Best wishes and thanks for any answer
Alexander

[Alexander Nitschke]

Hi Kevin,

I tried that but where can I "just change the clientCustomerId"? This
sounds great but the .NET AdwordsUser does require the
clientCustomerId in the headers which seemingly can't be modified
after creation of the AdwordsUser object with New
AdwordsUser(headers). At least I didn't see how to that but I am very
eager to hear how to do just this.

Then I thought you meant setting the clientCustomerId as a predicate
to the ReportDefinition, but a try to do this of course only caused an
invalid authentication exception (clienCustomerId required) so it must
be the AdwordsUser headers.

Please help me here.

Best wishes
Alexander

[Peter S.]

Alexander,

You can set custom headers before you create the AdWordsUser in
the .NET API. Just use a Dictionary<string,string> where the first
string is the key and the second is the value and pass that to the
constructor of the AdWordsUser.

Like so:

Dictionary<string, string> customDictionary = new Dictionary<string,
string>();
customDictionary.Add("Email", UserName);

Then pass this to the AdwordsUser. You will need email, password,
developertoken, and authtoken passed in as well to get the AdWordsUser
to construct appropriately.


On Jan 23, 12:27 pm, Alexander Nitschke

[Anash P. Oommen]

Hi Alexander,

You could do this the following way:

1. Set your MCC email and password in your Web.config.

2. AdWordsUser user = new AdWordsUser(); // this picks up the credentials from the Web.config.

3. (user.Config as AdWordsAppConfig).ClientCustomerId = your_client_customerid; // now the object is customized to run for given customerId

4. Call whatever service or report download utilities here.

With this setup, AuthToken is generated only once, so you won't run into captcha errors.

Cheers,

Anash P. Oommen,

AdWords API Advisor.

[Alexander Nitschke]

Thanks Peter, but I actually already did it this way. I asked
specifically about changing headers *after* creation of the
AdWordsUser. Done your way this will only cause a Captcha exception to
come up.

Alexander

[Alexander Nitschke]

Thanks Anash, especially the step 3 was the missing piece. I even
studied the .NET library source code in depth and saw that object
structure and thought about recompiling with headers exposed in the
AdWordsUser object but it just didn't occur to me that they already
are accessible in this way. That's genius what you did there. :-) I
still think the AdWordsUser object would allow to directly set this
value.

Sadly it doesn't seem to also be possible to save the AuthTokenand
later set it again this way (I still would like it this way since your
method still can cause repeat logins albeit much reduced). It is
internally saved in an AuthTokenCache and it would be nice to have a
method for the AdWordsUser to manipulate this cache with preset
values. If they aren't valid anymore the Authorization detects that
anyway and acquires a new AuthToken. Would it be ok if I make such a
request on the library's issue list?

Best wishes
Alexander


On Jan 24, 3:22 am, "Anash P. Oommen" <anash.p.oommen

[iateadonut]

This no longer works and I'm getting a
QuotaCheckError.INVALID_TOKEN_HEADER error.

I understand this has something to do with the developerToken, but my
auth.ini file looks like this:

email = "**@gmail.com"
password = "**"
userAgent = "Test App"
developerToken = "*****"

So I thought that the developer token was assigned to $user in the
script at:
$user = new AdWordsUser(); //by default here
$user->GetAuthToken();

and then passed here:

ReportUtils::DownloadReport($reportDefinition, $path, $user,
$options);

I even tried things like $user->setDeveloperToken($token) in the
script, just to try. (This did not give me an error, incidentally,
but, then I'm not sure how to display errors in $user objects.)


Anyway, This doesn't work. Can you give me a clue how this is
supposed to work?

[Anash P. Oommen]

Hi Alexander,

The library also exposes some ways of manipulating the cache. Take a look at http://code.google.com/p/google-api-adwords-dotnet/source/browse/trunk/src/AdWords/Lib/AdWordsSoapClient.cs#144 to see how tokens are invalidated and pushed out of the cache. If you need something more detailed, please feel free to open an issue in the project's issue tracker.

[iateadonut]

was using too old of a version of 201109 - updated to the
2012.jan.something release and it works now.

On Feb 9, 10:49 pm, iateadonut <orienta....@gmail.com> wrote:
> This no longer works and I'm getting a
> QuotaCheckError.INVALID_TOKEN_HEADER error.
>
> I understand this has something to do with the developerToken, but my
> auth.ini file looks like this:
>

> email = "*...@gmail.com"

[iateadonut]

Great,  I'm glad it worked for you.  I've used pcntl for a few things since.

Be careful installing pcntl on a server you share with others, though.  A runaway fork can easily take down your whole server.

On Monday, June 3, 2013 9:44:19 PM UTC-4, Bill Zhang wrote:

Thank you iateadonut. Your solution help us reduce the synchronization time dramatically. From 7~8 hours to within 10 minutes.

Best regads,

Bill

[Paul Matthews (AdWords API Team)]

Anyone considering using pcntl, should also bear in mind that it's fine for a small script, but it's easy to end up exhausting resources. It's also very difficult to debug.

A much more scalable solution to the problem is implementing a Job Queue. This can allow you to synchronize tasks between servers as well as on a single machine. An example of a Job Queue compatible with PHP is Gearman. They've got a nice quick start guide that can help you get started with it.

Regards,

- Paul, AdWords API Team.

[iateadonut]

Very cool, Paul!  Thanks for letting us know!