getAudienceSegmentsByStatement throws SERVER ERROR

[cvjetosi]

Hi,

We're experiencing issues with reading audience segment data from GAM.

At some point, the getAudienceSegmentsByStatement invocation throws a server error, without any specific details.

Please assist us in identifying the root cause for the issue.

Here are several request/response logs.

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Header>
<ns1:RequestHeader xmlns:ns1="https://www.google.com/apis/ads/publisher/v202208"
soapenv:mustUnderstand="0">
<ns1:networkCode>***</ns1:networkCode>
<ns1:applicationName>CloudSense Ltd Digital Fulfilment Integration (DfpApi-Java, Dfp-Axis/4.19.0, Common-Java/4.19.0, Axis/1.4, Java/1.8.0_202, maven)</ns1:applicationName>
</ns1:RequestHeader>
</soapenv:Header>
<soapenv:Body>
<getAudienceSegmentsByStatement xmlns="https://www.google.com/apis/ads/publisher/v202208">
<filterStatement>
<query>LIMIT 500 OFFSET 6500</query>
</filterStatement>
</getAudienceSegmentsByStatement>
</soapenv:Body>
</soapenv:Envelope>

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Header>
      <ResponseHeader xmlns="https://www.google.com/apis/ads/publisher/v202208">
         <requestId>5d5b89b0ac696de2f0bf165ae3db9aee</requestId>
         <responseTime>180061</responseTime>
      </ResponseHeader>
   </soap:Header>
   <soap:Body>
      <soap:Fault>
         <faultcode>soap:Server</faultcode>
         <faultstring>[ServerError.SERVER_ERROR @ ]</faultstring>
         <detail>
            <ApiExceptionFault xmlns="https://www.google.com/apis/ads/publisher/v202208">
               <message>[ServerError.SERVER_ERROR @ ]</message>
               <errors xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                       xsi:type="ServerError">
                  <fieldPath/>
                  <trigger/>
                  <errorString>ServerError.SERVER_ERROR</errorString>
                  <reason>SERVER_ERROR</reason>
               </errors>
            </ApiExceptionFault>
         </detail>
      </soap:Fault>
   </soap:Body>
</soap:Envelope>

-----

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                  xmlns:xsd="http://www.w3.org/2001/XMLSchema"
                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Header>
      <ns1:RequestHeader xmlns:ns1="https://www.google.com/apis/ads/publisher/v202208"
                         soapenv:mustUnderstand="0">
         <ns1:networkCode>***</ns1:networkCode>
         <ns1:applicationName>CloudSense Ltd Digital Fulfilment Integration (DfpApi-Java, Dfp-Axis/4.19.0, Common-Java/4.19.0, Axis/1.4, Java/1.8.0_202, maven)</ns1:applicationName>
      </ns1:RequestHeader>
   </soapenv:Header>
   <soapenv:Body>
      <getAudienceSegmentsByStatement xmlns="https://www.google.com/apis/ads/publisher/v202208">
         <filterStatement>
            <query>WHERE type = 'THIRD_PARTY' LIMIT 500 OFFSET 206500</query>
         </filterStatement>
      </getAudienceSegmentsByStatement>
   </soapenv:Body>
</soapenv:Envelope>

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Header>
      <ResponseHeader xmlns="https://www.google.com/apis/ads/publisher/v202208">
         <requestId>0a7a3adf5d18d953c960a7ca47f43aa5</requestId>
         <responseTime>180097</responseTime>
      </ResponseHeader>
   </soap:Header>
   <soap:Body>
      <soap:Fault>
         <faultcode>soap:Server</faultcode>
         <faultstring>[ServerError.SERVER_ERROR @ ]</faultstring>
         <detail>
            <ApiExceptionFault xmlns="https://www.google.com/apis/ads/publisher/v202208">
               <message>[ServerError.SERVER_ERROR @ ]</message>
               <errors xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                       xsi:type="ServerError">
                  <fieldPath/>
                  <trigger/>
                  <errorString>ServerError.SERVER_ERROR</errorString>
                  <reason>SERVER_ERROR</reason>
               </errors>
            </ApiExceptionFault>
         </detail>
      </soap:Fault>
   </soap:Body>
</soap:Envelope>

-----

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                  xmlns:xsd="http://www.w3.org/2001/XMLSchema"
                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Header>
      <ns1:RequestHeader xmlns:ns1="https://www.google.com/apis/ads/publisher/v202208"
                         soapenv:mustUnderstand="0">
         <ns1:networkCode>***</ns1:networkCode>
         <ns1:applicationName>CloudSense Ltd Digital Fulfilment Integration (DfpApi-Java, Dfp-Axis/4.19.0, Common-Java/4.19.0, Axis/1.4, Java/1.8.0_202, maven)</ns1:applicationName>
      </ns1:RequestHeader>
   </soapenv:Header>
   <soapenv:Body>
      <getAudienceSegmentsByStatement xmlns="https://www.google.com/apis/ads/publisher/v202208">
         <filterStatement>
            <query>WHERE type = 'FIRST_PARTY' LIMIT 500 OFFSET 25500</query>
         </filterStatement>
      </getAudienceSegmentsByStatement>
   </soapenv:Body>
</soapenv:Envelope>

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Header>
      <ResponseHeader xmlns="https://www.google.com/apis/ads/publisher/v202208">
         <requestId>e0cba20955737ee78709ba217441b0c4</requestId>
         <responseTime>10259</responseTime>
      </ResponseHeader>
   </soap:Header>
   <soap:Body>
      <soap:Fault>
         <faultcode>soap:Server</faultcode>
         <faultstring>[ServerError.SERVER_ERROR @ ]</faultstring>
         <detail>
            <ApiExceptionFault xmlns="https://www.google.com/apis/ads/publisher/v202208">
               <message>[ServerError.SERVER_ERROR @ ]</message>
               <errors xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                       xsi:type="ServerError">
                  <fieldPath/>
                  <trigger/>
                  <errorString>ServerError.SERVER_ERROR</errorString>
                  <reason>SERVER_ERROR</reason>
               </errors>
            </ApiExceptionFault>
         </detail>
      </soap:Fault>
   </soap:Body>
</soap:Envelope>

Thanks!

[Ad Manager API Forum Advisor]

Hi,

Thank you for reaching out to the Ad Manager API support team. I am Chekurthi and will be assisting you.

Kindly, allow me to raise this with the rest of our team in order to investigate further and one of our colleagues will reach out to you with an update.

We appreciate your patience and Thanks in advance.

Regards,

Chekurthi Ad Manager API Team

ref:_00D1U1174p._5004Q2hzFAm:ref

[cvjetosi]

Hi,

Is there any update on the issue?

Thanks!

[Ad Manager API Forum Advisor]

Hi everyone,

The AudienceSegment issue on our end has been resolved. If you are still getting errors let me know.

Thanks,

Chris Seeley Ad Manager API Team

ref:_00D1U1174p._5004Q2hzFAm:ref

[Ad Manager API Forum Advisor]

Correction: There are still some networks impacted by this. I'll post again when I have additional information.

[Rosana Elizabeta Sente]

Hi,

We kindly ask you to escalate this since it has passed 10 days without the fix and our clients are unable to sync the audiences.
Please keep us updated on the progress.
Thank you!

Kind regards,
Rosana

[Vedran Jazbec]

Hi GAM team,

following up on Rosana's mail.
After we got info from you yesterday at 8:58 PM CET saying that the issue was fixed, we initiated inventory sync for Audience segments.
While the inventory sync was ongoing, we got a new notification from you at 11:35 PM CET stating that the issue has not been resolved.

Inventory sync failed again, but this time we noticed a large number of records being created.
Before the sync, there were 26.5k audience segment records and after the sync, there were 219k audience segment records created. 

We  noticed the following:
the records which were created did not have the usual format of Targetable Name,
the description of each targetable was populated and none of the targetable could be found on the GAM side.

We can share the data we received if necessary, we already notified our client.
Please look into this on priority as this is causing a lot of manual work on our side for the last couple of weeks now.

Kind regards

Vedran

[Ad Manager API Forum Advisor]

Hi Vedran,

Thanks for the additional information. The audience engineering team is investigating why the fix was incomplete, I'll relay any updates as soon as I have them.

[Vedran Jazbec]

Hi team,

do you have any new information regarding the problem or an ETA for the issue?

Kind regards

Vedran

[DianaM]

Hi all,

We kindly ask you to escalate this since it has already passed 3 weeks without the fix and our clients are unable to sync the audiences.

Please keep us updated on the progress. Thank you!

With regards,
Diana

[Ad Manager API Forum Advisor]

Hi Diana,

Thank you for contacting Ad Manager API support team. I'm Ankam.

Based upon the investigation from our team, the Audience Segment issue on our end and it has been resolved, but there are still some networks impacted by this. And the audience engineering team is still investigating why the fix was incomplete. However, I have raised this issue with the rest of our team and one of our colleagues will reach out to you once we get an update on this issue.

Regards,

Ankam Ad Manager API Team

ref:_00D1U1174p._5004Q2iWPwG:ref

[ปรีชา บุญเกิน]

hảo hán

Vào Th 2, 6 thg 2, 2023 vào lúc 08:45 Vedran Jazbec <vedran...@cloudsense.com> đã viết:

--
You received this message because you are subscribed to the Google Groups "Google Ad Manager API Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-doubleclick-for-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-doubleclick-for-publishers-api/90b51aaf-2617-4543-90cc-69bdb03b65d2n%40googlegroups.com.

[Ad Manager API Forum Advisor]

Hi everyone,

Some networks had AudienceSegments with bad data. This has been corrected and I tested the example requests in this thread to verify the fix. Let me know if you're still seeing persistent SERVER_ERRORs and I can investigate further.

Do note that when reading large page sizes the AudienceSegmentService may time out. This also causes a SERVER_ERROR. These errors are intermittent and can be mitigated by retries with exponential backoff.

[Vedran Jazbec]

Hi team

we have run the Audience segment sync once again after you told us the issue was no longer present on your side.
The sync is still not working.
Again, as I posted here on January 29, the number of records created exceeds 200.000.

Furthermore, the records created don't have the usual format of Targetable Name, and while the description of each targetable was populated - none of the targetable could be found on the GAM side.

Please take into consideration that our client has in total (active and inactive) around 27.000 Audience segment records.
We fear that some of the data returned by your API do not belong to our client's org - which is potentially a big security issue and should be handled ASAP

This is the second time we noticed the same thing happening, and in the meantime, we are daily synchronizing our customer's Audience segment data manually, which is taking a lot of our time.

Please look into this on priority and if there is any other info we can provide, feel free to ask.

Kind regards
Vedran

[DianaM]

Hi team,

We have increased the timeout and we have retried another audience segment inventory sync,
but again the same thing happened, more than 200000 records were created
even though there are only around 27000 audience segments in Spotify's network.

Here are some audience segment examples that were synced in Salesforce org:

"Eyeota - UK/GB: Auto > Intent > Car Make > Saab", ad server id 828546784
"AdAdvisor Audiences powered by Neustar : AdAdvisor Consumer Audiences > Web Browsing > Services Site", ad server id 22727992
"Z_DO_NOT_USE_Eyeota - Aidata - RU: Hobbies > Handcraft > Modeling", ad server id 442461467
"LiveRamp Data Store > Cuebiq > Location Based > Dining and Restaurant Visitors > Marco's Pizza", ad server id 7305764113
"Twenty Ci > Tenure > Rental", ad server id 544592732
"Bluekai > Branded Data > Sirdata > Modeling > Habits & Lifestyle > Arts & Entertainment > Museum Lov", ad server id 849992610
"Retargetly > Branded Data > Equifax > Argentina > Rango de Edad > 65+ (TAXO-20116-444)", ad server id 559630582

We have checked if those records exist in Spotify's network by searching them in the list of audience segments,
and we are not getting any records with those ad server ids.

We have also tried editing the link as shown below and we couldn't find those records, and we are getting the error:
"An error has occurred while processing your request. Please try again."

An example link: https://admanager.google.com/6465052#inventory/audience_segment/detail/audience_segment_id=559630582

This is becoming a critical issue, so we would like to look into this as soon as possible as we don't know where those records are coming from.

Kindly let us know if you will need more information. Thank you.

With regards,

Diana

[Ad Manager API Forum Advisor]

Hi Diana,

That segment is a Third Party audience segment, and is available in that network under the third party view:
https://admanager.google.com/6465052#inventory/third_party_segment/list/view=all

Are you including an audience segment type in your filter?
If you want only first party segments you can filter on type FIRST_PARTY.

[DianaM]

Hi Chris,

no, we are not using any filters regarding audience segments, but we haven't in the past as well and we didn't have this kind of issue before.

We have checked the backup versions for inventory sync and I can confirm that in the last couple of years, no filtering was used to sync audience segments.

We just used the following:

    "audienceSegments": {
        "synchronize": true
    },

Could you please confirm if anything has changed recently regarding this? 

We could use the filtering as you suggested, but it is strange this started happening only recently and it was working fine before with the same configuration.

With regards,

Diana