Web hook not invoked everytime

[mingssn]

We have a problem with web hook.

Our lists has web hook set up. This web hook function suppose to be
invoked while our subscribers who click unsubscribe link in the email,
it will update our database to keep our mail list keep synchronize
with MailChimp. However, we noticed this web hook feature seems not
working every time. It is working some time, or some day but not
always working. We ruled out the problem from our side because we use
the same code to update our records, seems the web hook NOT invoked
every time when our subscriber click the unsubscriber link.

Anybody had the same experience?

[DavidM]

Yes, I've had the exact same experience too. I contacted MC support
and they said to ask here. It feels like it works around 70% of the
time for us, but there'll also be long spells where any activity done
within a few hours won't cause them to fire. It's quite annoying, and
means our frontend to manage MC subscribers is all out of sync and
everything.

About 15 minutes ago, I deleted/edited about 20 addresses in one of
our lists. The Webhooks weren't pinged at all, despite settings saying
that they should be getting it.

[Wei Ming]

David

Looks like those user who report email as spam will not have the web
hook triggered. You can find those user rating = 1 if you download
unsubscriber user list from mailchimp.

That's so far we have found but not co firmed by mailchimp.

Good luck.

Sincerely

Support Team
MT Solution Ltd.

http://www.PowerDataRecovery.com

> --
> You received this message because you are subscribed to the Google
> Groups "MailChimp API Discuss" group.
> To post to this group, send email to mailchimp-...@googlegroups.com
> .
> To unsubscribe from this group, send email to mailchimp-api-di...@googlegroups.com
> .
> For more options, visit this group at http://groups.google.com/group/mailchimp-api-discuss?hl=en
> .
>
>

[jesse]

Someone who reports abuse will be included in a "cleaned" webhook, not
an unsubscribe. You may want to make sure you are logging them all
just to make sure you don't miss something somewhere you didn't expect
it.

Looking in our logs, we've not been experiencing anything out of the
ordinary. I also have not seen our job servers get so backed up
recently that it would take over 30+ minutes to get a webhook
dispatched, though I'm continuing to keep an eye on that queue.

If you post the url for the list's subscribe form that we host, I can
take a look and see what, if any, errors we've encountered while
trying to post to your receiver url.

jesse

[hoks]

I tested with PostBin by editing a single member's profile.
Found that the webhooks only fired several hours later, and did so
multiple times per change i made.

[jesse]

Looking through our logs, it appears PostBin started returning a 201
instead of a 200 upon receiving a successful POST request at some
point. Since we were expecting a 200, that made us retry the request,
which would have caused duplicates. Our side has been adjusted treat
any 2xx response as a success so that that will no longer happen.

I'm still looking for an instance of us being backed up enough to
cause that sort of lengthy delay. Any occurrence of that is most
definitely the exception, and not the rule. If you are seeing that
often, I'd imagine your side is not responding in a timely manner
which is causing us to retry. As I'd mentioned before, if you'd like
to post your list's subscribe form URL, I can look through our logs
and see what we're seeing.

jesse

[DavidM]

Going through logs, it looks like changes I made came through about a
day later. Next time I do a batch-update of some things via the API, I
will see if there's any difference (we don't use the sub form, just
API for everything). I don't see why there'd be problems connecting to
the hook - the URL for it is on the same server as the one I'm sending
API calls from.

[jesse]

That use-case is actually exactly why we've considered disabling
Webhooks for API calls. That is because:
1) it can cause jobs to back-up when large batches are run
2) it makes so little sense to us - why would you need a webhook to
tell you what you just did? It seems redundant and unnecessary.

Honestly the only reason we enabled them was under the (incorrect)
assumption that people would just use them in conjunction with single
calls.

We are looking into ways to increase the number of simultaneous
Webhooks that can be fired without impacting other systems. However,
even the point of that is not to support this use-case, but rather to
further decrease the chance of any bottle necks occurring.

I would be interested to know the reasoning you all (or others) have
to use Webhooks with batch updates via the API.

jesse

[DavidM]

We send out daily newsletters to companies, depending on what they
have subscriptions to. Each company has a login, and under that login
they can add/remove email addresses of employees that want to receive
our stuff. They put in the address of the person, their name/position,
and tick the boxes of what they should receive (depending on what
subscriptions the company pays for).

When they hit "submit", the info gets bundled up, some other data
added, and it gets sent off to MC via the API. Then when MC responds
by hitting the WebHook, the database on our side is then updated with
the new details. I used to just update the database before sending to
MC, but there have been a few times when MC timed out etc, leading our
stuff to get out of sync with what is sitting on your servers. I
figured the WebHook would be the best way to keep the database
synchronised, as emails only get added/removed/cleaned when MC itself
reports that something has changed.

Having written this, I suppose a less stupid solution would just to be
make absolutely sure that MC sent back an OK response when batch
subscribing, then add everything to the database then. However I've
also had times when MC doesn't respond at all even though it turns out
that whatever operation performed completed successfully.

Only around 40 addresses a day from our side get updated, so surely
sending out 40 pings isn't going to completely kill the servers?

[hoks]

I've switched from PostBin to using my own server yesterday, the delay
between an update and a event fired was about 3 minutes, so it's
acceptable.
But the long delay problem seems to be back today.

My server has no problems responding, because when I update the
WebHook URL, my server received a probe from mailchimp almost
immediately.

I've not implemented a form yet. Was just manually editing member info
as admin.

[jesse]

First off, the thing you all will be most happy about is that after
managing to catch Webhook jobs backed up today, I found something I
could actually change. Our Webhook jobs were getting placed in a
priority level that included several job types which also
occassionally would spawn off a large number of jobs. They have been
adjusted to have their own priority level which should greatly
increase performance and turn around times.

There is still the chance a user could cause a large number of Webhook
jobs to be created and cause some delays, though those should also
clear out much more quickly now. To be clear, when I say a large
number I mean thousands or tens of thousands, not hundreds.

DavidM - if you are having instances of not getting a reply from us,
you should increase the timeout settings in your client as it is
likely dropping the connection before our reply. However, as you say,
40 (even 400) a day is not going to cause any issues, especially since
it sounds like they are spaced out.

jesse