Mailing alerts with appengine

[Michal Makarewicz]

A have a scenario where I have to send customized alerts to large
amount of users (~30k) in certain, specified beforehand, time. My main
idea was to make a cron job, which would check every minute if there
is an alert to send and if so mark this alert as queued and add task
to task queue (this would be done in transactional manner). Now I have
two available approaches:
Approach I:
Have two different queue commands:
1. Split email list into smaller chunks (I'm not sure how big, I was
thinking about ~500) and for each of them put second command on queue.
(this one will be queued in cron job)
2. Send mail for each email address in the list.
My main concern with this approach is lack of control on how many
emails will be send simultaneously and possibility that my email will
be classified as spam.

Approach II:
Just one task:
1. Send emails for predetermined number of people (again ~500)
starting from start_index add to queue the same task with start_index
+ ~500.
This approach will be slower, so is less preferable.

I have pretty much unlimited quota so that wouldn't be a problem.
Which approach is better in this situation?

[MiuMeet Support]

One thing you should consider is if you want to use appengine as an email infrastructure.

We run a social network on appengine with 1M+ users and we love it to run everything on it,

except email. EMail deliverability on appengine is not really great. You can't do DKIM signing,

you can't send from a dedicated IP address and so on.

We send about 250k emails out per day, but for this we host our own mailserver with an

http interface that we call via urlfetch. Deliverability is much higher... but of course it's

an additional point of failure. This morning greengeeks (our hosting provider) turned off the

server without notice "due to extensive use". So make sure you choose a dedicated server

provider that doesnt just turn off your machines when they send a lot of emails. We are now

switching to hetzner.de. A friend of mine runs mailservers there with millions of emails per day

and claims he never had problems, as long as the emails are legitimate.

Just my 20 cents.

-Andrin

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Ernesto Oltra]

Other option: Amazon SES:

  http://aws.amazon.com/ses/

Just out of curiosity, been from Google you can't use GMail infrastructure?

[Robert Kluin]

Hey Michal,
   You can get good performance while controling the rate by using a combination of the two methods.  Split the list into several "large" chunks, inserting a processor for each chunk. Then each of those processors can run over their respective sets of addresses either mailing or inserting mailer tasks.   Just  be sure to persist your current position in the lists and use small batch sizes in your mailer tasks. If a task reruns for some reason it will reduce the number of people who get duplicate mails. 

Robert