What happened to the promised reliability improvement for email delivery?

[pdknsk]

Google announced this in May.

"We have added a few restrictions to the Mail API to improve the
reliability and reputation of the service for all applications. First,
emails must be sent from email accounts managed by Google (either
Gmail, or a domain signed up for Google Apps). Second, we’ve reduced
the number of free recipients per day from 2000 to 100 for newly
created applications. Both of these will help ensure mail from your
application arrives at the destination reliably."

http://googleappengine.blogspot.com/2011/05/app-engine-150-release.html

Well, it doesn't seem to have made a difference because many mails to
specific providers are still not delivered reliably, as has been
documented multiple times in this group and elsewhere. Another
complaint about it was posted today. I had really hoped Google would
have DKIM in place for new billing to improve reliability.

http://code.google.com/p/googleappengine/issues/detail?id=3161

[Daniel]

+1

[Mos]

+1;  would be great to see something in the upcoming 1.6.1

 +1

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Daniel Florey]

Yes, please!

http://code.google.com/p/googleappengine/issues/detail?id=3161

and

http://code.google.com/p/googleappengine/issues/detail?id=1800

[pdknsk]

Google should have a two tiered approach. If you're on a free app,
this is as good as it gets. You won't get far with 100 mails per day
anyway. For paying customers though (especially with domains
registered through Google Apps), Google should really improve the
service. I'm willing to pay extra too. Google already increased the
price by $0.19 per day, but alright.

I really want to avoid to use Amazon SES, Postmarkapp, Sendgrid, ...,
because I prefer to have billing and service in one place.

[pdknsk]

I'd really appreciate a response from a Googler, as to what the plan
is. I know Google doesn't give ETAs for new features, but just a
general response will do, like so.

We will improve email delivery reliability soon.
We will improve email delivery reliability at some point, but not
soon.
We have no plans to improve email delivery reliability.

[Jeff Schnitzer]

Assume #3. Use a different email provider. There are countless to choose from.

Jeff

[voscausa]

Can you recommend one? What about :  http://mailchimp.com/

[JH]

I've had a lot of success with Amazon SES. I switched when I noticed
a lot of GAE email being flagged as spam and google employees on this
very group said they really do intend to become a mass email platform.

I have an extremely high deliverability rate with SES.

[James Broberg]

SES, Dynect email, many others to choose from.

[pdknsk]

I'm disappointed no Googler responded. I'll have to interpret this as
suggested by Jeff then.

[jon]

> Assume #3.  Use a different email provider.  There are countless to choose from.

Jeff can you recommend one?

[Jeff Schnitzer]

On Fri, Nov 11, 2011 at 7:07 PM, jon <jonni...@gmail.com> wrote:
>> Assume #3.  Use a different email provider.  There are countless to choose from.
>
> Jeff can you recommend one?

We're about to integrate with messagebus.com but I don't know enough
about their service yet to give the big thumbs up. They're run by
very cool people, so all lights are green so far.

I'm pretty close to adopting a "startup-only" policy for service
providers. The support we've been getting from MessageBus and WePay
(especially WePay!) has been nothing short of astounding. You'll
never get the time of day out of Amazon or PayPal. Despite Google's
silence on this particular issue (which I suspect is due to a
questionable commitment), the support here in this forum has been
really good - I'm not sure if you realize it, but the people answering
questions in this forum are often the actual engineers developing
features.

As far as email goes - personally, I'd rather Appengine shut down the
email API and put those developers to work on something else. Email
is something that many other providers do well, and there's no
particular benefit to having it built in (getting two bills is truly a
"first world problem"). I'd rather Google focus on the hard problems
which can't easily be implemented by third parties - spatial indexes,
pull queue grouping, transactional task deleting, etc. These enable
whole new solution domains. Think about the major new features that
the GAE team has rolled out in the last year... the HRD, XG
transactions, Backends, Exploding Index Repellant, Pull Queues,
Memcache CAS, Deferred Tasks, Cursors, Async Datastore Operations...
every time one of these features comes out I think "How did I ever
write code without this?" That's what I want more of.

Jeff

[Adrian Scott]

As far as email goes - personally, I'd rather Appengine shut down the
email API and put those developers to work on something else.  Email

I agree, completely...

well almost completely... Keep the current basic email to help newbies play and test, but minimize devoting any further significant resources and possibly relabel that api as beta/experimental or at least as not destined to receive lots of attention right away, to be up front about it. It's a non-trivial, high-nuisance service to provide/maintain.

-A

--
Adrian Scott, Ph.D.
CEO, Founder
CoderBuddy
http://www.coderbuddy.com/ <-- Create a Facebook or Google App Engine app in a minute without installing anything


 

is something that many other providers do well, and there's no
particular benefit to having it built in (getting two bills is truly a
"first world problem").  I'd rather Google focus on the hard problems
which can't easily be implemented by third parties - spatial indexes,
pull queue grouping, transactional task deleting, etc.  These enable
whole new solution domains.  Think about the major new features that
the GAE team has rolled out in the last year... the HRD, XG
transactions, Backends, Exploding Index Repellant, Pull Queues,
Memcache CAS, Deferred Tasks, Cursors, Async Datastore Operations...
every time one of these features comes out I think "How did I ever
write code without this?"  That's what I want more of.

Jeff

[pdknsk]

If Google announced to drop further support for the mail API, that
would be fine, because developers can make an informed decision then.

I'm mostly interested in DKIM, which has the potential to really
improve reliability and seems (from my basic understanding) almost
trivial to implement. Why? It's already set up when you register a
domain through Google Apps. Google adds DKIM headers automatically to
all emails send from the Google Mail interface. So all that's missing
is an additional dkim parameter which automatically adds those headers
when sending through App Engine.

It might even be possible for developers to calculate and add those
headers themselves, if Google adds DKIM-Signature to allowed headers.

[Martin Waller]

The email needs to be sorted. No email sent to a hotmail account from the app engine seem to arrive!

Sent from my iPhone

[Thomas Wiradikusuma]

+1 with me.
This sort of thing gives GAE a bad name. Better implement it good or
not at all.

But of course people would wonder, "aint email one of Google's
strength? (Gmail)"

[jon]

messagebus.com is in private beta now. Does anyone have any other
suggestions?

I hear you Jeff about GAE ditching the email service but startups have
enough 3rd party services to deal with as it is (analytics, mobile
push, web channel). Having fewer providers to work with is always
better.

On Nov 13, 1:09 pm, Thomas Wiradikusuma <wiradikus...@gmail.com>
wrote:

[Ernesto Oltra]

Sendgrid.com ; built in top of Amazon SES, but it gives you a nice API through simple web requests, and it helps with statistics, bounces, etc.

[pdknsk]

As we've discussed on the bug report, DKIM could literally be a one
line bug fix.

[voscausa]

I have SES up and running making use of this python 2.6 SES API.  https://github.com/pankratiev/python-amazon-ses-api

As you said, this is working fine. And I'am now able to handle bounced e-mail messages with Return-Path.

To use this API for GAE Python 2.5 I had to make some changes by overriding two methods. Here is the code for sending e-mails.

To make the code even more reliable, reviews are welcome. This is my first experience with Amazon services.

#!/usr/bin/python

# -*- coding: utf-8 -*-

from my_settings import AMAZON_ACCESS_KEY_ID, AMAZON_SECRET_ACCESS_KEY

from amazon_ses import AmazonSES, EmailMessage

from google.appengine.api import urlfetch

from google.appengine.runtime import DeadlineExceededError

import urllib, logging

class gaeAmazonSES(AmazonSES):                                               # make the API work for GAE python 2.5                                                     

        

    def _performAction(self, actionName, params=None):

        if not params:

            params = {}

        params['Action'] = actionName        

        #https://email.us-east-1.amazonaws.com/

        reason = ''                                                                     # exception

        retry = 0                                                                       # download error retry

        while retry <= 1 :                                                              # a one time retry        

            try :  

                url = 'https://email.us-east-1.amazonaws.com'

                response = urlfetch.fetch(url=url, payload=urllib.urlencode(params), method=urlfetch.POST, headers=self._getHeaders())

                break

            except (urlfetch.DownloadError, DeadlineExceededError),e :

                logging.debug('Amazon SES download or deadline error : %d' %(retry + 1)) 

                if retry == 0 :

                    retry = retry + 1

                    continue                                                            # retry

                else :

                    reason = e                                                         

                    break

        

        if reason == '' :                                                               # check for fetch errors          

            responseResult = response.content

            status_code =  response.status_code

            logging.debug(response.headers)

        else :

            responseResult = None

            status_code = -1

        result = self._responseParser.parse(actionName, status_code, reason, responseResult)

        logging.debug(result)

        return result

    def sendEmail(self, source, toAddresses, message, replyToAddresses=None, returnPath=None, ccAddresses=None, bccAddresses=None):

        params = { 'Source': source }

        if not replyToAddresses : replyToAddresses = [source,]                          # always reply_to

        for objName, addresses in zip(["ToAddresses", "CcAddresses", "BccAddresses", "ReplyToAddresses" ], 

                                      [toAddresses, ccAddresses, bccAddresses, replyToAddresses]):            

            if addresses:

                if not isinstance(addresses, basestring) and getattr(addresses, '__iter__', False):

                    for i, address in enumerate(addresses):

                        if objName == 'ReplyToAddresses' :                              # reply_to                                          

                            params['ReplyToAddresses.member.%d' %(i + 1)] = address

                        else :                                                          # destination

                            params['Destination.%s.member.%d' % (objName, i + 1)] = address

                else:

                    if objName == 'ReplyToAddresses' :

                        params['ReplyToAddresses.member.1'] = addresses

                    else :

                        params['Destination.%s.member.1' % objName] = addresses

                  

        if not returnPath:

            returnPath = source

        params['ReturnPath'] = returnPath

        params['Message.Subject.Charset'] = message.charset

        params['Message.Subject.Data'] = message.subject.encode('utf-8')

        if message.bodyText:

            params['Message.Body.Text.Charset'] = message.charset

            params['Message.Body.Text.Data'] = message.bodyText.encode('utf-8')

        if message.bodyHtml:

            params['Message.Body.Html.Charset'] = message.charset

            params['Message.Body.Html.Data'] = message.bodyHtml.encode('utf-8')

        logging.debug(params)

        return self._performAction('SendEmail', params)  

    

def sesMail(sender, mail_to, subject, body_text, reply_to=None, bcc=None):

    amazonSes = gaeAmazonSES(AMAZON_ACCESS_KEY_ID, AMAZON_SECRET_ACCESS_KEY)

    message = EmailMessage()

    

    message.subject = subject

    message.bodyText = body_text  

    result = amazonSes.sendEmail(sender, mail_to, message, reply_to, 'return path@ mydomain.com', None, bcc)

    logging.debug(result)

[Vivek Puri]

FWIW, we are moving over all the critical emails to sendgrid.com.
Current blackbox nature of email delivery(no reporting) plus the
missed emails to users is impacting our end users. Will switch back
once things change for better.

[voscausa]

Why Sendgrid and not AWS SES. I moved to AWS SES and it is looking veyy good.

[Vivek Puri]

'Cause we like startups!

[Andrin von Rechenberg]

We are sending out about 1.8 million emails per month.

SES or sendgrid would be way to expensive. (~$1000).

We do it from a single server (2ghz, 4gb) hosted somewhere

outside of google. It costs us about $60 per month and we

have a great delivery rate (spif, dkim, dedicated ip and

also very important: we handle all email bounces in

appengine and dont send any further email to an address

if it bounces)

Our delivery rate is quite high, way beyond GAE, except

for hotmail.

I can highly recommend building setting up your own email

server and just add a simple php script to expose a http

api. Took me about a day to setup and its 20x cheaper

then sendgrid. It's been running for almost a year and

i never even had to login into the mailserver. it just

runs super smooth. I'm monitoring it with prodeagle.com

Cheers,

-Andrin

'Cause we like startups!

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

[Brandon Wirtz]

Server Beach. Pick a Canada CoLo

Pricing about what Andrin said.  Being in Canada you screw up and violate some Cann Spam law… Or if you do so intentionally… Being in Canada will prevent your system from just disappearing, and having your domain stolen… At least for a little while.

 

They have a 100% SLA  (.5% better than Google’s) Because they are hard core like that.

 

I have coupons if you need them.   

 

I run quite a few servers there.

[pdknsk]

I had still hoped Google is quiet on this because they wanted to
surprise us and sneak this into 1.6.1 but no such luck.

[pdknsk]

I've noticed this in the 1.6.2 release notes. While not DKIM, it might
help.

- Mail Quota for App Engine apps that have signed up for billing will
only be
increased after the first payment for the app is processed.