webhook event called multiple times for single event

[Nathan Stowell]

Hello,  I am working through building out a subscription system for connected accounts.  I have successfully built the required backend to manage creating a subscription for a connected account.  I setup a specialized endpoint on my api to catch the "invoice.payment_succeeded" event to know when that client's card had been charged for the subscription.  This endpoint captures the event and records it to a generic "event" table and then processes the client's new credit that is owed to them.  Having just set all of this up (and firing many test web hook events), I pushed this out to a server and opened the endpoint up.  I then pointed Stripe to this endpoint but only for the event of "invoice.payment_succeeded".  I then logged into my app created a user and set them up on $1/day subscription with no trial so that I could ensure that the endpoint was being hit properly and that my system would process the data correctly.

I just checked my "event" table in my production db and this is what I found:

If that image isn't showing, basically I have 28 rows of "invoice.payment_succeeded" all for that one client.  

I looked in my app and sure enough that client now has a credit of "28" in his account.  I quickly logged into the strip dashboard and navigated out to that connected user's account and noticed that there where actually 2 $1 charges in the logs. One was for today and the other is for tomorrow.  I went back to my database and sure enough there are six records with tomorrow's date and 22 records with today's date.

Ok so now here goes my questions:

Why did this event get hit 22 times for one single event?

Are webhooks not  designed to work this way?  

Should I not rely on them to process when a client has been charged for their subscription?  

Should I be looking for other ways to know that a user has been charged for their subscription?

Are there any other "gotchas" when dealing with web hooks?

to be clear going forward I will be validating against the event's "id" property, as this is was just a raw test to make sure I was heading in the right direction.

Thanks so much in advance for any help or advice anyone can offer!

[Remi J.]

Hey Nathan,

When we send an event to your webhook endpoint, we expect a 2xx response code indicating that you were able to process the event properly. If you return any other response code (or simply timeout) we'll assume that you didn't get the event. When that happens, we will re-send the event every hour for up to 3 days before giving up on that specific event.

I'm assuming that your code is not responding properly here which led us to send it 22 times. If you look at that event in the dashboard, at the bottom, you'll see the state of that event and what error code your endpoint kept giving us. That should help you diagnose what the issue is. You should also look into your error logs to see if anything specific pops up there that could indicate why you're not responding properly.

I hope this helps but please do not hesitate to get back to me if you need more details.

All the best,

Remi

--
You received this message because you are subscribed to the Google Groups "Stripe API Discussion" group.
To post to this group, send email to api-d...@lists.stripe.com.
Visit this group at http://groups.google.com/a/lists.stripe.com/group/api-discuss/.

To unsubscribe from this group and stop receiving emails from it, send an email to api-discuss...@lists.stripe.com.

[Nathan Stowell]

Ah that makes a lot of sense.  I will test that and make sure that solves the problem.  Thank you for your help!

[Ali Ahmed]

Hey Remi,

Your answer solved many confusions and Thanks a lot for that. But I've another question, which doesn't seem to be a matter, but why you've made it so like it always take 1 hour to re-send the request even if we only want to test the webhook with our backend code that either our backend code correctly working (The DB updates etc when the the webhook comes with a response). With this happening, when webhook fails due to our backend code we have to wait for this long time to test it again otherwise we see that webhook keep gives the Timeout error.

Is there any other way that we can get rid of this long waiting only in Stripe Test mode?

My another question is that, that what is is it possible that a same webhook can be out sent twice even the first webhook was successful?

[Remi J.]

Hello Ali!

We don't retry events constantly because it tends to make things worse on both sides. If a webhook handler is not responding fast enough, it often means it can't keep up with the load or is experiencing issues in general. The faster we retry, the worse it would get for that server as they would get new events and numerous retries of past events over and over, increasing the load and increasing the chance that the server goes down. We retry events slower and slower over time to give time to the server to catch up.

The reason we do this in Test mode is similar. The majority of endpoints that fail the first delivery in Test mode will fail all future delivery. The reason is that it often just means that the endpoint was turned off by the developers who was originally testing. It would not be productive to retry the events constantly in Test mode when we know the likelihood of a successful delivery is low.

It should not be an issue when testing though. If your endpoint is having an issue with an event, the easiest way to move forward is to simply run your test again to generate a different event that would be delivered immediately instead of having to wait for the one that failed delivery and will retry later.

As for the duplicate delivery, it should be rare but it is possible that it happens. The idea is that you might think you responded with a 2xx in time but we did not receive it and assumed it failed, in which case we would deliver the event again even though you thought it was handled on your end. Similarly, bugs can happen and even though we have a lot of failsafes to prevent this you can always imagine that something could fail on our end and we would redeliver an event. For those reasons, I always recommend tracking the events you handled in your database to ensure that if you receive the same event again you can simply skip it.

Hope this helps!

Remi

Visit this group at https://groups.google.com/a/lists.stripe.com/group/api-discuss/.

[ali.ahme...@gmail.com]

Thank you so much Remi. What ever you described was very significant and cleared many thoughts. However, with that being said that I can keep track of events to be free from troublesome duplicate events in my database, I also read in one of your docs that you also provide idempotent key approach, generated from client, which can aid to solve this issue.

Just for clarification, can you also tell me that if we use this approach in Subscription charges and receive invoice.payment_succeeded webhook notification, then would it work with it as well? So that even if it would send out the selfsame event it would include the same response as of the last successful event. I was thinking and read somewhere that we can only use idempotent key approach in One Time Charge but not in the subscription.

My case is actually that I need to make Instalment Plans which can be smoothly achieved through Subscriptions + Webhook. So I was just worrying about this duplicate events otherwise I have made everything so far.

Sent from

[Remi J.]

Hello,

The `request[idempotency_key]` would only be set if you passed a key when you made the API request that caused the event to be created. In the case of Subscriptions though that would not happen since you are not making any API request and the billing cycle renews automatically on our end.

I would recommend just tracking event ids (evt_XXXX) in your database instead to detect a duplicate and just skip it if it ever happens.

Best,

Remi