Debugging FCM Sends/Deliveries
Mike DiGiovanni
I'm an Android developer trying to track down some options for debugging an FCM delivery issue that we've been experiencing for about a month now.
Mid-January we began experience a drop in how many push notifications are delivered to our users, this drop continued and got significantly worse over time. This appears to have been both iOS and Android across multiple app versions, although iOS started about a week later. We aren't ruling out a client side issue despite the breadth of the problem, because we have many feature toggles and theoretically something could have been switched on that affects many versions and both platforms. Initial research didn't reveal anything that would have triggered this on the app side though.
While we haven't been able to reproduce this exact issue through developer and QA testing, we've seen instances where pushes were delivered over a day late. Since we haven't seen the delivery loss ourselves, it's just an educated guess that push notifications are not being delivered, based on our own metrics for when our backend tells FCM to send a push, and our own metrics from our client apps saying that a push was delivered. But we have seen a financial impact and impact to visits commensurate with the decrease in delivers.
On
the client side, bad payloads and exceptions are handled and tracked without
displaying notifications, but logs did not indicate an increase in those
problems. Nor have we seen an increase in unhandled exceptions.
On
the server side, our team confirmed that they are not receiving an increase in errors
sending and has systems in place for errors like DeviceMessageRateExceeded or
TopicsMessageRateExceeded. Those don't appear to be happening during our delays
either.
We've enabled delivery data export in our next app release for Android, hoping that this would help us confidently say that it's not a client side issue, since this would negate the impact of any client side delivery tracking logic or quirks. https://firebase.google.com/docs/cloud-messaging/understand-delivery#enable_delivery_data_export_for_app_instances
This is in a set of systems that have been in production for years at this point. The systems we've previously had in place just aren't giving us enough information to get to the bottom of the problem this time.
Are there any other options other suggestions for debugging this on either the app or backend side? Are there more FCM stats or tracking we can use to compare to our proprietary tracking?
Kato Richardson
--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/861b97bf-824c-4d0f-b51e-ace1f8350e7dn%40googlegroups.com.