API Send Message is very slow

[John Jarrard]

Hi,

I've been sending message with the mailchimp API for over a year now,
and just recently - in the last 10 days or so - it has becoming
unbearably slow... causing all my apps to timeout. Is there something
wrong with the system? What details would you need from me to help
troubleshoot?

Thanks
John Jarrard
Sterling Payment Technologies
john.j...@sterlingpayment.com

[jesse]

The first thing to talk about would be exactly how your process works
and specifically what types of "messages" (campaigns, right?) you are
sending and how. Have you also verified the call times in your API
dashboard?

http://admin.mailchimp.com/account/api


jesse

> john.jarr...@sterlingpayment.com

[John Jarrard]

Hi Jesse,

The issue that is with the lists() method. It takes over 60 seconds
to run... everything else is like 2 to 5 seconds (normal).... Can
you look into it?

JJ

> --
> You received this message because you are subscribed to the Google Groups "MailChimp API Discuss" group.
> To post to this group, send email to mailchimp-...@googlegroups.com.
> To unsubscribe from this group, send email to mailchimp-api-di...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mailchimp-api-discuss?hl=en.
>
>

[jesse]

I'm pretty sure I see your calls - have you logged into the account
and looked at your lists recently? I'd actually be surprised if that
page loads. You have 7076 of them, which would appear to be 7070 more
than you need. That's what's caused lists() to slow down - there's not
an automated way you can clean those up.

Since you are looking at this code, you absolutely need to make
several changes:

* stop using login all the time - cache your api key
* stop running lists all the time - cache the data, including list
ids, that you need
* don't make API calls when you don't need to - I see lots of login(),
then lists() calls which makes me guess you are using that to build a
form on every page load somewhere. don't do that.
* looks like campaignFolders is being run way too much as well - cache
that data, too
* you are using psuedo-transactional campaigns completely incorrectly,
so it's lucky you ran into this now. They are intended to be created
*once* and then sent repeatedly. It makes no sense to create one per
email - how would they be any different from regular campaigns in that
case? You should also go ahead and clean up old ones (there are 2662
trans campaigns created right now)

If you fix those things and provide the list_ids you want to keep
around, I'll be willing to look at automating cleaning up the extra
lists that got in there somehow. Once you've cached everything, you
also should make it clear to anyone manually accessing the account
that they should let you know before data related changes (new lists,
merge vars, interest groups, etc) are wanted to make sure it doesn't
break the API.


jesse

[John Jarrard]

I found out why - something happened on 3-22-2011 that caused the
system to add 7000+ lists all with the same name (resourceCenter) -
Is there any way to easily delete all those empty lists???

[John Jarrard]

Ok, I will go cache everything... and look at how to fix the psuedo-
transactional emails. In the meantime, will you please delete the
empty lists?

JJ

[John Jarrard]

These are the lists I want to keep:

CREATEDON ID MEMBERCOUNT NAME
1 {ts '2010-03-23 20:12:33'} 10fcb8d710 7147 Sterling Merchants
2 {ts '2010-03-24 18:31:44'} 70cf78bd75 3204 Sterling Agents
3 {ts '2010-03-24 18:32:46'} a38bd217fc 241 Rewards
4 {ts '2010-04-05 20:59:25'} b16db35a93 62 Sterling Employees
5 {ts '2010-06-14 17:03:38'} e2d02344ae 11 Sterling Emps
(Personal)
6 {ts '2011-01-05 18:59:06'} 76d29b07d1 409 forgotPasswords
7 {ts '2011-03-22 16:15:53'} 22665ddfe6 1 resourceCenter

On Mar 25, 3:57 pm, jesse <je...@mailchimp.com> wrote:

[John Jarrard]

Jesse, How did those erroneous 7070 lists get created? I thought I
had to create Lists in your web app (not through the API) so how did
7070 get created? Was that a bug on your site or as result of
something in my code?

JJ

On Mar 25, 3:57 pm, jesse <je...@mailchimp.com> wrote:

[jesse]

Thanks, the batch delete has been started - I'll post back when it's
finished. I have no idea how all of the lists were created, but since
the create times were around the same time, it looks like something
tried to automate creating a bunch of them.


jesse

[jesse]

The extra lists have been deleted. It looks like you've cleaned up
everything but pseudo-transactional campaigns creation and the old
ones. You absolutely need to stop creating campaigns them every time
you send them as you *will* run into problems. I'll let that go until
at least tomorrow, but we'll suspend sending permissions if that
doesn't get fixed.



jesse

[John Jarrard]

I'll get it fixed tomorrow afternoon... I've got so much stuff going
on right now.