Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
managing bulk emails from app engine app
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  17 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Vivek Kumar  
View profile  
 More options Jul 7 2012, 1:19 pm
From: Vivek Kumar <vik....@gmail.com>
Date: Sat, 7 Jul 2012 10:19:12 -0700 (PDT)
Local: Sat, Jul 7 2012 1:19 pm
Subject: managing bulk emails from app engine app

Hello

We have around 10000 registered users with us who are blood donors. We want
to send them periodic emails like every month to remind them or their
account status etc.
Obviously, a single request cannot process so many emails.

So, what is the suggested way to handle this use case without hitting
request time limits etc on app engine?

our app is in GAE for java

Vik


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Joakim  
View profile  
 More options Jul 8 2012, 3:54 am
From: Joakim <joakim.edenh...@gmail.com>
Date: Sun, 8 Jul 2012 00:54:32 -0700 (PDT)
Local: Sun, Jul 8 2012 3:54 am
Subject: Re: managing bulk emails from app engine app

Split the work into manageable chunks and create one TaskQueue task per
chunk. TaskQueue jobs have a request time limit of ten minutes [0], which
may well be enough to send all your emails.

[0]: https://developers.google.com/appengine/docs/java/backends/overview

Joakim


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Jeff Schnitzer  
View profile   Translate to Translated (View Original)
 More options Jul 8 2012, 4:31 am
From: Jeff Schnitzer <j...@infohazard.org>
Date: Sun, 8 Jul 2012 01:31:31 -0700
Local: Sun, Jul 8 2012 4:31 am
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app
I find it most convenient to make the 'manageable chunk' one single email.

It's a map/reduce-type problem. You want to map the "enqueue email
task" across every user.  The simplest solution is to iterate across
the users and enqueue a task for each.  You can do it in batches of
100 (actually QueueConstants.maxTasksPerAdd()); I find the Guava
Lists.partition() and Lists.transform() methods really handy here.

With a keys-only query you can probably iterate across 100k records in
the ~60s before a query times out.  If not, you can enqueue the
mapping process.  You could even use the google-provided MapReduce
toolkit.

Ultimately you end up with a zillion "email this user" tasks which you
can run as parallel as you wish.

Jeff


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Per  
View profile  
 More options Jul 8 2012, 11:27 am
From: Per <per.fragem...@gmail.com>
Date: Sun, 8 Jul 2012 08:27:04 -0700 (PDT)
Local: Sun, Jul 8 2012 11:27 am
Subject: Re: managing bulk emails from app engine app

Hi Vik,

technical details aside, keep in mind that the spam filter settings on App
Engine can be pretty strict. We have been sending mails for 18 months now,
increasing the number gradually, and we also used DKIM signing. Still,
suddenly some time last week our mails stopped getting sent. From all I can
tell some Google mechanism decided that we're a spammer. We have inquired
with Google instantly, but of course the GAE customer service is poor as
always and never got back to us. So we switched to an external email
provider now (mailgun, but there are dozens out there), and I'm quite happy
with that, since tracking of emails is a lot more convenient too. It's not
free though.

Long story short, if you all of a sudden send 10k mails, be prepared to get
filtered by Google because they think you're a spammer. Make sure to spread
the load, and track what you have sent already, so you know what you'd need
to resend if push comes to shove.

Cheers,
Per


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Richard Watson  
View profile   Translate to Translated (View Original)
 More options Jul 8 2012, 11:57 am
From: Richard Watson <richard.wat...@gmail.com>
Date: Sun, 8 Jul 2012 08:57:05 -0700 (PDT)
Local: Sun, Jul 8 2012 11:57 am
Subject: Re: managing bulk emails from app engine app

Offload both transactional and bulk email to a 3rd-party email specialist.
 It's their core competence, and you're likely to be able to send bulk
email with one api call once your list is set up.  If you have issues
you'll have better insight into what those are.  You can get delivery and
read reports, and automatic remove-me-from-your-list handling.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Jeff Schnitzer  
View profile  
 More options Jul 8 2012, 12:13 pm
From: Jeff Schnitzer <j...@infohazard.org>
Date: Sun, 8 Jul 2012 09:13:15 -0700
Local: Sun, Jul 8 2012 12:13 pm
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app
I should add that we also use a third-party email system.  Ultimately
it's no harder to make a REST call to an external service than to make
a call to MailService, and many services offer a much more robust
solution.  GAE's mail system has silly limitations like the inability
to send emails with inline (Content-ID) image attachments.

Just make sure that your 'unit of work' is the same as a single
urlfetch.  You wouldn't want to put 100 REST calls in a single task -
imagine it failed at #50, then the whole task will retry and re-spam
#1-#49.  If your mail service does "send this message to these 100
people", great; otherwise stick to one-email-per-task.  We do
one-email-per-task.

Jeff


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Vivek Kumar  
View profile  
 More options Jul 8 2012, 1:26 pm
From: Vivek Kumar <vik....@gmail.com>
Date: Sun, 8 Jul 2012 10:26:19 -0700 (PDT)
Local: Sun, Jul 8 2012 1:26 pm
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app

Hie

Actually that was my worry as well like sending through app engine may mark
emails spam and thus wasting our efforts.

We ocassionaly send newsletter to all our subscribers and for that we are
already using VerticalResponse.com which gives us 10000 emails credit every
month for free being a non profit.

Right now, we want to handle use cases like:
" Sending email once a month to all  the blood donors to verify/update
their profile"

So, if we use any 3rd party system then any recommendations? If non -profit
friendly then that would be awesome!

Vik


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
pdknsk  
View profile   Translate to Translated (View Original)
 More options Jul 8 2012, 2:45 pm
From: pdknsk <pdk...@googlemail.com>
Date: Sun, 8 Jul 2012 11:45:58 -0700 (PDT)
Local: Sun, Jul 8 2012 2:45 pm
Subject: Re: managing bulk emails from app engine app

> suddenly some time last week our mails stopped getting sent. From all I can
> tell some Google mechanism decided that we're a spammer. We have inquired

http://code.google.com/p/googleappengine/issues/detail?id=6181

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Vivek Kumar  
View profile  
 More options Jul 13 2012, 1:51 am
From: Vivek Kumar <vik....@gmail.com>
Date: Thu, 12 Jul 2012 22:51:24 -0700 (PDT)
Local: Fri, Jul 13 2012 1:51 am
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app

Hie Jeff

Which thrid party system do you guys use? please share the details like how
u r using their service to send emails from your gae app

Vik


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Vivek Kumar  
View profile  
 More options Jul 19 2012, 12:32 am
From: Vivek Kumar <vik....@gmail.com>
Date: Wed, 18 Jul 2012 21:32:11 -0700 (PDT)
Local: Thurs, Jul 19 2012 12:32 am
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app

hie can some one help us on suggesting third party emails systems which are
good to go to integrate with app engine?

Vik


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Richard Watson  
View profile  
 More options Jul 19 2012, 5:16 am
From: Richard Watson <richard.wat...@gmail.com>
Date: Thu, 19 Jul 2012 02:16:30 -0700 (PDT)
Local: Thurs, Jul 19 2012 5:16 am
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app

Sendgrid, or Google "sendgrid vs" and you'll see comparisons.  Note that
some specialise in bulk email, some transactional, some do both.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Stephen Lewis  
View profile  
 More options Jul 19 2012, 5:51 am
From: Stephen Lewis <step...@sock.org.uk>
Date: Thu, 19 Jul 2012 02:51:27 -0700 (PDT)
Local: Thurs, Jul 19 2012 5:51 am
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app

I'm not the best person to provide a comparison between services because
we've only tried Sendgrid. That said, they've been excellent for our needs
- we've sent just shy of 2 million mails via them so far.

Stephen


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Vik  
View profile  
 More options Jul 19 2012, 12:56 pm
From: Vik <vik....@gmail.com>
Date: Thu, 19 Jul 2012 09:56:45 -0700
Local: Thurs, Jul 19 2012 12:56 pm
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app

Hey Guys

Thanks for the reply...

Any experience with Amazon SES ? Pricing looks good to me however, I am not
sure it works smooth with google app engine for java? Please advise

Thankx and Regards

Vik
Founder
http://www.sakshum.org
http://blog.sakshum.org


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Jeff Schnitzer  
View profile  
 More options Jul 19 2012, 2:18 pm
From: Jeff Schnitzer <j...@infohazard.org>
Date: Thu, 19 Jul 2012 11:18:04 -0700
Local: Thurs, Jul 19 2012 2:18 pm
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app

On Thu, Jul 12, 2012 at 10:51 PM, Vivek Kumar <vik....@gmail.com> wrote:
> Hie Jeff

> Which thrid party system do you guys use? please share the details like how
> u r using their service to send emails from your gae app

Sorry I missed this earlier.  We use messagebus.com.  We're very happy
with them, although I don't think their service is generally available
to the public yet.

Jeff


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Vik  
View profile  
 More options Jul 19 2012, 2:31 pm
From: Vik <vik....@gmail.com>
Date: Thu, 19 Jul 2012 11:31:44 -0700
Local: Thurs, Jul 19 2012 2:31 pm
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app

Jeff

Does it play nice with google app engine for java?  Also how the pricing
looks like? As you said not yet public so emailed them.

Thankx and Regards

Vik
Founder
http://www.sakshum.org
http://blog.sakshum.org

On Thu, Jul 19, 2012 at 11:18 AM, Jeff Schnitzer <j...@infohazard.org>wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Jeff Schnitzer  
View profile  
 More options Jul 19 2012, 3:03 pm
From: Jeff Schnitzer <j...@infohazard.org>
Date: Thu, 19 Jul 2012 12:03:25 -0700
Local: Thurs, Jul 19 2012 3:03 pm
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app
It's a simple REST API.  Works fine with GAE.  I'm sure they all do.

Jeff


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
SGBoulder  
View profile  
 More options Jul 24 2012, 4:35 pm
From: SGBoulder <dave.schwa...@sendgrid.com>
Date: Tue, 24 Jul 2012 13:35:24 -0700 (PDT)
Local: Tues, Jul 24 2012 4:35 pm
Subject: Re: [google-appengine] Re: managing bulk emails from app engine app

Would highly recommend SendGrid over Message Bus...

3 years mature and proven. It's my understanding that Message Bus is not
yet scalable and drops often.

2 cents!


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »