FILE_FORMAT_ERROR using BatchJobService.

[lz...@marinsoftware.com]

I use BatchJobHelper to upload operations to AdWords, but there are a lot of FILE_FORMAT errors. Why does this happen even the request was handled by the client library? By the way, I upload the operations in an incremental way. like this: jobHelper.uploadIncrementalBatchJobOperations(currentOperations, lastUpload, lastUploadStatus).

<soap:Body>

        <getResponse xmlns="https://adwords.google.com/api/adwords/cm/v201509">

            <rval>

                <totalNumEntries>1</totalNumEntries>

                <Page.Type>BatchJobPage</Page.Type>

                <entries>

                    <id>1128059</id>

                    <status>CANCELED</status>

                    <progressStats>

                        <numOperationsExecuted>0</numOperationsExecuted>

                        <numOperationsSucceeded>0</numOperationsSucceeded>

                        <estimatedPercentExecuted>0</estimatedPercentExecuted>

                    </progressStats>

                    <downloadUrl>

                        <url>https://batch-job-download-prod-ebe9b43.storage.googleapis.com/79037285

/1128059.results.xml?GoogleAccessId=926043054183-rbu1gkdus...@developer.gservice

account.com&amp;Expires=1454745731&amp;Signature=J%2BHIGRT%2BCKhh5iUT40twZZwXexzeYJD%2FPwHXidZe170j7

BRROK75IJAW8p8%2FEQfSaSIRo7E4GOLvqthSqmuIw6aCL%2B3MZmGTu85WcOb6pG1qtJTO34maFf%2FVZ3ZEGjvqxhE0%2BrQ1%

2B0%2FkdWjA9K9EDi%2FT1Uf5827Q3I4SWYLs1zwRnRlgmaqc%2FoqXpA%2FAB6lFdfh9r7uCUw9fwIQDBuIeWmtnhG%2BxytQyZ

k%2FpMbcbkXdoFHQvJCAmqvWWxiT%2Fw%2BcSpd0BnWR5%2FKTTX755ozneOjKqN7xcS7REd4h7maefSRWLAOJIctEaENfN2bGwj

YyTidutingG5J6uV44xioJ4sw%3D%3D</url>

                        <expiration>20160206 000211 America/Los_Angeles</expiration>

                    </downloadUrl>

                    <processingErrors>

                        <fieldPath/>

                        <trigger/>

                        <errorString>BatchJobProcessingError.FILE_FORMAT_ERROR</errorString>

                        <ApiError.Type>BatchJobProcessingError</ApiError.Type>

                        <reason>FILE_FORMAT_ERROR</reason>

                    </processingErrors>

                </entries>

            </rval>

        </getResponse>

    </soap:Body>

</soap:Envelope>

[Yin Niu]

Hello, 

The FILE_FORMAT_ERROR indicates there's a format error in the input file. To upload file, the input xml file should have a format as specified in BatchJobOps.xsd. For incremental uploads, the first request will have the start of <mutate> element and the last request will have the end of </mutate> element. 

Thanks,

Yin, AdWords API Team. 

[lz...@marinsoftware.com]

Hi Yin,

I am using BatchJobHelper(a utility exists in Google ads Java library) to upload operations, so there will be no chance for me to encounter a error related to file, but unfortunately there was. 

The Java client I am using:

<dependency>

    <groupId>com.google.api-ads</groupId>

    <artifactId>adwords-axis</artifactId>

    <version>2.8.0</version>

</dependency>

Thanks a lot!

[Nadine Sundquist (AdWords API Team)]

Hi,

That is odd. In order to triage it, I would need to take a look at your code. Could you please click Reply privately to author in the forum, and send me what your code looks like? That way I can reproduce what you're trying to send.

Thanks,

Nadine, AdWords API Team

[lz...@marinsoftware.com]

The code involves a lot of context to make it understandable, so I am not going to send to you now. However, I could explain to you what the code really does:

1) There are some threads, let's say 10

2) For each threads, it will create one batch job using BatchJobService

3) For each threads, with the batch job upload url created above, we upload operations(1campaign which contains 10 groups, and each group contains 1000keywords, so that will be 10011 objects) incrementally, 1000 operations per incremental upload, and the parent operations goes before child operations.

4) All the thread will be run simultaneously.

5) When one thread finishes uploading its operations, it will send its batch job id to another dedicated thread(let's say BatchJobStatusQuery thread).

6) The BatchJobStatusQuery thread will periodically poll the status of the batch job.

If you are sill confusing what I am really doing, I will try to send a revised code to you.

Thanks

[lz...@marinsoftware.com]

Things are even getting worser when I have more than 10 such threads, most of batch jobs created by a thread will fail to upload operations due to the same error: FILE_FORMAT_ERROR.

[Nadine Sundquist (AdWords API Team)]

Greetings!

Thanks for the explanation and the update. It does sound complex enough that it wouldn't be easy to decouple that code from existing code. So, when you run just one thread everything works fine? Based on what you said, it sounds like it's sometimes happening, but not always. If so, it sounds like there may be a threading issue somewhere in the process. I'm asking questions to narrow down the scope of the issue. Are you using a different instance of BatchJobHelper for each thread?

Regards,

Nadine, AdWords API Team

[lz...@marinsoftware.com]

Here is testing result we have:

1) If the total number of operations is less than or equal to 100k(which means there will be less than or equal to10 threads, because one thread handle 10k operations), everything works fine, all objects are successfully be pushed to Adwords.

2) If the total number of operations is larger than 100k(for example, 150k or 1m), there will always be some errors from Adwords.

For your last question:

Yes, we do use different instance of BatchJobHelper for each thread, not just for each thread, we create a new BatchJobHelper even for each incremental upload inside a thread. Because we notice that it is a utility class & we have to make sure our session will not be expired.

Apart from FILE_FORMAT_ERROR, we also have another issue, but we are not sure what's going on. Sometimes, in our application, when we upload 10011 objects in one thread, only 9012 objects will be pushed to Adwords, the other 999 objects are missing from the mutate result. This issue only happens when we have operations more than 100k.

Thanks

[Nadine Sundquist (AdWords API Team)]

Hello,

Wow! You weren't kidding when you described this as complex. Let me run this by one of my teammates. There are a few possibilities, and I need to narrow it down. 

In order to dig into your secondary problem, I would need your customer ID. Could you please click Reply privately to author in the forum and send me the customer ID of one of the accounts where you noticed that you sent more operations than were executed? Is it that the items are just missing from the mutate result that comes back, or are they missing from the AdWords account as well?

Thanks,

Nadine, AdWords API Team