Rewarded Video SSV callbacks randomly fail to include user_id and custom_data

[Juan Manuel Pereyra]

Hi,

Opening a new thread as I started experiencing a similar issue to other someone experienced last november. On that thread I was prompted to give:

• A brief description of what you are seeing. But from a user side as well as from your server.
  From a user perspective, what happens is that a player chooses to see a rewarded ad to be granted some bonus and when the video is completed he's not awarded that bonus.
  From a server side perspective, it receives the SSV callback but the user_id and custom_data info that we set for all ad requests is not set there so we don't have the proper data to reward the user.
  This doesn't happen always, is random and not specifically for a user, so for me it can work now but fail on the next video.
• A simple sample app that is able to display this issue.
  As it is a random issue with a specific application it's too complex to create a sample app and a server to receive the SSV and hope to reproduce the behavior, however I can provide the Ads library wrapper that we use to do every call if requested.
• Steps to reproduce the results that you mentioned.
  Just watch videos on the application if the amount of impressions and failures say something 1 of 6 calls should fail.
• Also have you contacted our Product Support Team , as this could also be a server side issue with data just not returning properly.
  I haven't but I will.
• Any Charles logs that display this.
  We don't have Charles logs for it but I can provide an example server side log of the received request:
  • valid callback:
    {transaction_id=[XXXXXXf597d4bafe04261c25b36b993e], ad_network=[XXXXXX3213286189855], ad_unit=[XXXXXX4726], reward_amount=[1], user_id=[XXXXXX-3a01c68e-3e16-45ed-8f02-e678b4243d94], signature=[XXXXXX...], key_id=[XXXXXXXXXXXX], reward_item=[Reward], custom_data=[{"custom_operation":"c9386f08-d303-4746-84f7-d0a9606fbc76","custom_card_id":"01002023","placement":"KeepAnotherCard"}], timestamp=[1620746623838]}
    
  • invalid callback:
    {transaction_id=[XXXXXXeaeef0aa229547d45ae877447], ad_network=[XXXXXX3213286189855], ad_unit=[XXXXXX4726], reward_amount=[1], signature=[XXXXXX...], key_id=[XXXXXXXXXXXX], reward_item=[Reward], timestamp=[1620722581965]}

Some extra information:

Platform experiencing problems: Android.

Unity Version: 2019.4.23f (now 2019.4.26f)

AdMob SDK: GoogleMobileAds-v5.4.0 (for unity)

Also used: GoogleMobileAdsFacebookMediation-v3.2.0 (for unity)

Please request for any other information that I might provide.

Regards,

Juan Manuel

[Mobile Ads SDK Forum Advisor Prod]

Hi Juan,

Thank you for the response back and answering the questions that I ask. Allow me to try and add more context to what you have said and what could be done to try to help you. Recently Rewarded ads have gone through large changes, have you adapted to the newest API's? Also with v20/v8 we have had to make even larger changes that caused known issues. The other issue with that, was that we do not yet have the newest plugin for the newest v20/v8.

Lack of a sample app also makes it hard for us to look more into this. We suggest using one of our samples if you can replicate it there. Also without steps, then it is harder to find the issue. We would definitely also suggest to contact the Product Support Team, as this looks like it could be a server side issue also. Although I believe it has more to do with the recent changes made to rewarded ads and the newer v20/unity plugin.

Regards,

William Pescherine Mobile Ads SDK Team

ref:_00D1U1174p._5004Q2HA4D3:ref

[Juan Manuel Pereyra]

Hi William,

Ok got it, I don't know which changes you mean but we're using the "official" AdMob plugin for unity (exactly this one), so what I understand from your message is that it doesn't include the most recent changes?

I also get you about the lack of a sample app, but you got to understand that this being a problem with the Server Side Verification callback, setting up an app that you can use to test, plus a server to receive the callback and setting it up on the AdMob console to just see if we can reproduce an error that is also kind of randomic would be a long path to thread. What I can definitely do if you want is to provide the wrapper class we use to load and show all of our RewardedAds, which will give you an idea of how we load the SSV parameters every time into a rewarded ad request.

If I have to guess, the problem resides either:

• on the SDK failing to set those parameters when launching the add and requesting the ad server
• on the server failing to relay those parameters for some reason to our server at SSV callback time
  

I've also opened a thread on the support group (this one for reference) in case you want to contact with them about the problem.

Thanks for looking into it!

Cheers,

Juan Manuel

[Mobile Ads SDK Forum Advisor Prod]

Hi Juan,

Thank you for the response back. I completely understand the complexity of this for you to build out some of the systems for something that is hard/random to reproduce. So, I believe we can look at several other ways for us to get the desired information that we need as you have mentioned. First, I would point to this article about how the AdMob SDK just recently went through significant changes. While I know these do not show direct effect with Unity, they can be causing issues as we are needing to push out a new plugin to adjust for some of these changes.

As for gaining more data? Would the wrappers would be helpful to see what you are doing and how it is being passed into the system. Also as you have shown before you can send more of the responses and any logcats that display the issue. For that you can send it to mobileads...@gmail.com.

[Juan Manuel Pereyra]

Hi William,

Thanks for the info! I'm sending the required data to that email with a link to this thread for context.

Cheers,

Juan Manuel

[Mobile Ads SDK Forum Advisor Prod]

Hi Juan,

Thank you for sending over the email, we did receive it and I was able to look over it. When looking at the SSV documents that we have here . I see that when looking at this section with the parameters, that the SetUserId is no longer linked to GitHub as well as the SetCustomData does not appear to be available. I do believe this all has to do with the changes that were made as I previously mentioned.

Allow me to bring this to my team and see what feedback they give regarding this. Once we hear back, we will be certain to pass this along to you.

[Mobile Ads SDK Forum Advisor Prod]

Hi Juan,

I agree with you that the problem is one of:

• on the SDK failing to set those parameters when launching the add and requesting the ad server
• on the server failing to relay those parameters for some reason to our server at SSV callback time

but only once we can rule out app implementation of simply not setting the parameters.

Usually in situations like this, it's because the parameters don't make it to the server.

Could you share the full transaction IDs for both a call that had the custom data, and one that didn't? With that, I can try
to ask around internally if those transactions can be looked up to see if the server got the params and at least rule out the
server component. More recent IDs (within the last few days) are preferable if available.

Thanks,
Eric, Mobile Ads SDK Team
 

ref:_00D1U1174p._5004Q2HA4D3:ref

[Juan Manuel Pereyra]

Hi William, Eric,

Here are the required transaction IDs:

• valid: 0c37e64622feb06a1356313c321c44d3
• invalid: 21522d38aee5a14e270ea483486b4239

Both of them are no more than a few hours old.

Also Eric, I've shared how we include the parameters on the calls on an internal mail if you want to check.

Regarding what William was saying, is there any possibility also that you have 2 parallel backends, one to cater to the legacy ads and other to the new, and for some reason some calls are "bleeding" to the new backend when they should be attended by the old one? That might explain the randomness of the situation. Maybe a load balancing issuing requests to a node of the new backend or something like that (sorry for the wld guessing here).

Thanks to you both.

[Mobile Ads SDK Forum Advisor Prod]

Hi Juan,

Thanks for sharing the transaction IDs. I filed a request to have those investigated.

Looking at your code, from what I can tell, you are always setting the SSV data prior to showing the ad. However, there's not enough information to prove that the custom data passed in is not null, or whether your GlobalStateControl.PlayerInfo.GetIdentifier() could potentially still return null.

[Juan Manuel Pereyra]

Hi Eric,

Actually there is for the custom data, please see:

if (serverParams == null) {
    serverParams = new Dictionary<string, string>();
}
serverParams["placement"] = placement;
_.rewardedAd.SetServerSideVerificationOptions(new ServerSideVerificationOptions.Builder()
    .SetUserId(GlobalStateControl.PlayerInfo.GetIdentifier())
    .SetCustomData(JsonConvert.SerializeObject(serverParams))
    .Build()
);

if the server params are null they are initialized there with at least a placement, and believe me, if your GlobalStateControl.PlayerInfo.GetIdentifier() is null all the app would be blowing up to bits before even trying to load an ad.

Thanks,

Juan Manuel

--

Juan Manuel Pereyra Co-founder Skype: juan.m.pereyra Mobile: +59899945623 web: www.pincergames.com

[Mobile Ads SDK Forum Advisor Prod]

Hi Juan,

Thanks for the correction on customData.

I'm still waiting to hear back from engineering to debug those 2 transaction IDs.

[Juan Manuel Pereyra]

Cool Eric, thanks for the update!

[Mobile Ads SDK Forum Advisor Prod]

Hi Juan,

I'm still waiting to get some traction on the internal ticket to diagnose further (ticket is getting bounced around), but wanted to let you know this is still being tracked.

[Juan Manuel Pereyra]

Hi Eric,

Thanks, I really appreciate it.

[Mobile Ads SDK Forum Advisor]

Hi Juan,

We're a bit stuck on being able to make further progress. Our internal logs strip out the user_id and custom_data from the pings, so from that we can't verify either way from our end whether the pings coming in actually have the fields set or not.

Are you still experiencing this issue? If so, what is the frequency that you're seeing failure? And any chance you're able to reproduce this with a Charles log capture?

Thanks,
Eric

ref:_00D1U1174p._5004Q2HA4D3:ref

[Juan Manuel Pereyra]

Hey, we still have this issue from time to time, is there any update on it?

[Mobile Ads SDK Forum Advisor]

Hi Juan,

Did my update from July 27th make it to you? Here it is again:

"We're a bit stuck on being able to make further progress. Our internal logs strip out the user_id and custom_data from the pings, so from that we can't verify either way from our end whether the pings coming in actually have the fields set or not.

Are you still experiencing this issue? If so, what is the frequency that you're seeing failure? And any chance you're able to reproduce this with a Charles log capture?"

Have you ever been able to reproduce this in your own environment?

Thanks,
Eric

ref:_00D1U1174p._5004Q2HA4D3:ref

[Juan Manuel Pereyra]

Hi Eric,

For some reason it went to spam.

We are and the symptoms are the exact same (random behavior for same users, etc etc). 

We have updated every SDK to the latest and still have the same problem.

The only other thing I can say is that as you saw in the sample we always call the SDK in the same way, whatever is lost, is lost after that call.

I can provide newer occurrences of it (logs of the problem) if you need newer report ids.

Best

Juan Manuel

--

---
You received this message because you are subscribed to a topic in the Google Groups "Google Mobile Ads SDK Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-admob-ads-sdk/Zdao6M_jjBo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-admob-ads...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-admob-ads-sdk/bWsCs000000000000000000000000000000000000000000000QXE1T700Rb2tNd0-Soql5kzoyTIITQ%40sfdc.net.

[Mobile Ads SDK Forum Advisor]

Hi Juan,

Thanks for confirming. Unfortunately, new transaction IDs won't help because when tracing that back to the SDK ping that triggers the SSV callback, the logs will look the same regardless of whether the user_id/custom_data is passed.

Do you have any estimates on how often this happens?

[Juan Manuel Pereyra]

Hey!

Yes, it happens in bursts, I'm attaching a picture of the occurrences in the last 7 days.

As you can see there are several occurences at a time, then stops, then again and so on several times a day.

I can't stress how much this is a problem on our side as we rely on server side verification to award some of the best "goodies" we give on rewarded ads and a user that doesn't receive them because of this will definitely be impacted.

Thanks!

--

---
You received this message because you are subscribed to a topic in the Google Groups "Google Mobile Ads SDK Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-admob-ads-sdk/Zdao6M_jjBo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-admob-ads...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-admob-ads-sdk/tqk0H000000000000000000000000000000000000000000000QXS7JT00UW8uTwN0Q4SRhOjPiRdEcg%40sfdc.net.

[Mobile Ads SDK Forum Advisor]

Thanks for sharing more info Juan. I am pushing monitoring here to better detect client vs. server-side issue, but to set expectations, that monitoring is being treated more as a feature request than a bug.

Interesting that it happens in bursts. One radical theory is that malicious users are trying to manually modify the URL and tries removing fields in the process, but that looks distributed enough to be unlikely.

When the callback fails, do you have a way to log app version or GMA SDK version on your side to see if there is any correlation there? Unfortunately, the params in the SSV call itself aren't useful to correlate to the app software running.

[Juan Manuel Pereyra]

Hey, the issue here is that the callback fails server side... the client just waits then for something that doesn't happen and assumes that the server side verification was not triggered. 

I can build something that would log everytime the client expects a reward and then the server never sends it but it'll take time until we can push that update out and get data.

On the other hand what I can tell you is that our user base is fairly low yet in terms of numbers as we're on open beta (which also adds to the severity of the issue because if this affects this much with not many users, then imagine what it can be with a high number) and we have 100% adoption of versions using the latest GMA Unity SDK, so I don't really think spending time building that log will benefit a lot.

[Mobile Ads SDK Forum Advisor]

Hi Juan,

Understood and fair enough. I'll continue pushing on my side for better logging so we can diagnose on our end. Just as you have explained that every code path is setting the fields, I'm getting similar confidence on the backend side that if the fields are in the request back up to the server, the S2S call will have the fields.

If logging is implemented, we can detect client vs server issue and narrow down the problem. And if client-side, it could still be a combination of your code and any SDKs involved (Unity plugin, GMA SDK). But hopefully at the same time we'll be able to see trends by tying it back to ad requests to see any trends (e.g. is it 100% unity apps, 100% a particular SDK version).

[Juan Manuel Pereyra]

Hi Eric,

That would be great, and of course that's what I'm hoping for! Finding the issue, if we can do that, whether it's on the API side or the implementation doesn't matter, what matters is that we can fix it =)

I'm also open if anyone on your side needs me to trigger some ads at a specific time from our application to try and reproduce it. What I'm thinking also is that maybe if I have some time I can try to implement that log you mentioned, but because we can try and find out other data from it, like whether the requests are coming from a region in particular, or they're also random, etc etc.

Let me see what I can do.

Cheers

--

---
You received this message because you are subscribed to a topic in the Google Groups "Google Mobile Ads SDK Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-admob-ads-sdk/Zdao6M_jjBo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-admob-ads...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-admob-ads-sdk/g6Ftm000000000000000000000000000000000000000000000QY1LSD004Pz2giKiQdu3LQqa2h5HMQ%40sfdc.net.

[Mobile Ads SDK Forum Advisor]

Hi all,

I'm syncing with the engineering team this week to see how we can improve logging to detect if the problem is client vs. server-side.

If there's any way for you to consistently reproduce, that would help speed up the investigation to identify the root cause. Regardless, I'll still continue to escalate this on my end.

[Juan Manuel Pereyra]

Hi,

I'm bumping this up again because we're still seeng this issue almost 8 months after reporting it. 
Do you have a bug reference or something that we can use to track its state? 

Regards,

Juan Manuel

[Mobile Ads SDK Forum Advisor]

Hello Juan,

Thanks for reaching out.

I will sync with the engineering team and get back to you once there is an update.


Regards,
Joshua
 

ref:_00D1U1174p._5004Q2HA4D3:ref

[Juan Manuel Pereyra]

Hi Joshua,

Thanks for that, but it would be great if there is a way for us to submit and track this kind of bugs, we're left with radio silence most of the times otherwise.

Regards,

Juan Manuel

[Mobile Ads SDK Forum Advisor]

Hello Juan,

We do have an internal bug ticket for tracking this.

I will check in with the engineering team periodically and will let you know once there is an update.

Thanks!