email sent by GAE would be regarded as spam for some email services

[Eric Ka Ka Ng]

we use mail.send_mail() to send some important messages to our users through an app hosted on GAE, in which the 'from' has been set to one of a registered admin for the app. 

it works quite well for most users, except for some email service provider (e.g. '@yahoo.com.hk'), they would always automatically treat these emails as spam and put them into spam box of the user's email account (if the user has enabled the spam filtering feature, which is by default "ON") . in this case, many of our users do not aware for these important messages (they would seldom look into mails in their spambox) 

we have tried using different "from", "subject", "body" and the results are the same. we suspect that it is the email server does matter, and maybe before there were some other apps hosted on GAE sending spams to like '@yahoo.com.hk', so '@yahoo.com.hk' would regard all emails sent from this email server in GAE as spam.

do anyone share similar experiences? or there should be other causes? any ideas we can solve this problem? (successfully deliver the email to our users using those email service without being regarded as spam) thx in advanced!

- eric

[Ernesto Karim Oltra]

Ask the user to add the e-mail from address of your e-mails to their
contacts list, so your e-mails would never been sent to spam again
(for that users, at least).

It's a bit hacky, but can do a great work meanwhile you find another
solution.

On 18 mar, 07:52, Eric Ka Ka Ng <ngk...@gmail.com> wrote:
> we use mail.send_mail() to send some important messages to our users through
> an app hosted on GAE, in which the 'from' has been set to one of a
> registered admin for the app.
>
> it works quite well for most users, except for some email service provider

> (e.g. '...@yahoo.com.hk'), they would always automatically treat these emails

> as spam and put them into spam box of the user's email account (if the user
> has enabled the spam filtering feature, which is by default "ON") . in this
> case, many of our users do not aware for these important messages (they
> would seldom look into mails in their spambox)
>
> we have tried using different "from", "subject", "body" and the results are
> the same. we suspect that it is the email server does matter, and maybe
> before there were some other apps hosted on GAE sending spams to like '@

> yahoo.com.hk', so '...@yahoo.com.hk' would regard all emails sent from this

[Eric Ka Ka Ng]

Hi Ernesto,

thx for your suggestion! yes, i think it shall work. 

but in general if we can't ask all our users to add our email to their contact list (or they are not willing to do so), how can emails sent by us prevent to be regarded as spam? all our subject, body etc. are nothing spam-liked, and seems it's the problem of the email server that actually sends the email. anyone share similar case, or has other solution? 

regards,

eric

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Nickolas Daskalou]

Hi Eric,

Have you looked at Amazon's Simple Email Service (SES)?

http://aws.amazon.com/ses/

Pretty much the same cost as sending email using GAE, except with SES you only get 2,000 free emails if you're sending email from an EC2 instance.

I haven't used it myself but apparently they scan outgoing mail to make sure it meets "ISP standards". This leads me to believe emails sent via SES would be less likely to be automatically marked as spam than on GAE, since SES's spam-sending-reputation would be lower than GAE's, due to their pre-filtering and rate-limiting.

Nick

[Ernesto Karim Oltra]

I use too random directions for each user.This means, I take its
user.user_id(), build sha1, and send e-mail to each user from
directions like that:

alert-[user]@[app].appspotmail.com

I think you have to enable inbound email for this to work, but i'm not
sure. This is the method I have seen in Twitter, they send e-mails
from, as an example:
follow-rearfgbxnevz=tznvy.p...@postmaster.twitter.com

clearly automatic generated emails.

On 21 mar, 04:31, Nickolas Daskalou <n...@daskalou.com> wrote:
> Hi Eric,
>
> Have you looked at Amazon's Simple Email Service (SES)?
>
> http://aws.amazon.com/ses/
>
> Pretty much the same cost as sending email using GAE, except with SES you
> only get 2,000 free emails if you're sending email from an EC2 instance.
>
> I haven't used it myself but apparently they scan outgoing mail to make sure
> it meets "ISP standards". This leads me to believe emails sent via SES would
> be less likely to be automatically marked as spam than on GAE, since SES's
> spam-sending-reputation would be lower than GAE's, due to their
> pre-filtering and rate-limiting.
>
> Nick
>
> On 21 March 2011 13:51, Eric Ka Ka Ng <ngk...@gmail.com> wrote:
>
>
>
>
>
>
>
> > Hi Ernesto,
>
> > thx for your suggestion! yes, i think it shall work.
>
> > but in general if we can't ask all our users to add our email to their
> > contact list (or they are not willing to do so), how can emails sent by us
> > prevent to be regarded as spam? all our subject, body etc. are nothing
> > spam-liked, and seems it's the problem of the email server that actually
> > sends the email. anyone share similar case, or has other solution?
>
> > regards,
> > eric
>

[Jamie H]

Wow, great idea! I have been having some delivery issues lately with
GAE and looking for an alternative.

I just downloaded a pythong Amazon SES library, signed up for SES, and
put it all together and it works like a charm with GAE!

On Mar 20, 10:31 pm, Nickolas Daskalou <n...@daskalou.com> wrote:
> Hi Eric,
>
> Have you looked at Amazon's Simple Email Service (SES)?
>
> http://aws.amazon.com/ses/
>
> Pretty much the same cost as sending email using GAE, except with SES you
> only get 2,000 free emails if you're sending email from an EC2 instance.
>
> I haven't used it myself but apparently they scan outgoing mail to make sure
> it meets "ISP standards". This leads me to believe emails sent via SES would
> be less likely to be automatically marked as spam than on GAE, since SES's
> spam-sending-reputation would be lower than GAE's, due to their
> pre-filtering and rate-limiting.
>
> Nick
>
> On 21 March 2011 13:51, Eric Ka Ka Ng <ngk...@gmail.com> wrote:
>
> > Hi Ernesto,
>
> > thx for your suggestion! yes, i think it shall work.
>
> > but in general if we can't ask all our users to add our email to their
> > contact list (or they are not willing to do so), how can emails sent by us
> > prevent to be regarded as spam? all our subject, body etc. are nothing
> > spam-liked, and seems it's the problem of the email server that actually
> > sends the email. anyone share similar case, or has other solution?
>
> > regards,
> > eric
>

[Eric Ka Ka Ng]

me 2. Have integrated the SES service within our GAE apps.  work good and solved our problems 

sure it would be more nice if GAE can provide email service at the quality level similar to the Amazon SES one (at least would not be classified as SPAM), so we dont need to manage both . could we make this suggestion to GAE team?

- eric

[Ikai Lan (Google)]

I actually think we are moving away from this direction. For customers sending large volumes of email or who require delivery guarantees, we'll be encouraging the use of services that are specifically designed for this functionality. 

Ikai Lan 
Developer Programs Engineer, Google App Engine

Blog: http://googleappengine.blogspot.com 

Twitter: http://twitter.com/app_engine

Reddit: http://www.reddit.com/r/appengine

[Jamie H]

Ikai,

On that note, do you have any services besides SES that you
recommend? SES works well with GAE however you must slowly up your
quotas, forcing you to send some mail via SES and some through GAE
until your quotas are adjusted high enough...

[James Broberg]

SendLabs is another option:

http://dyn.com/enterprise-email/sendlabs-solutions

[Eric Ka Ka Ng]

Hi Ikai,

thx for your note. with this, we could better decide on which cases we would like to send emails through GAE and which we would like to use other external services. 

and would it be more formally stated in any documents, or any guideline for these considerations be documented? if i'm a new GAE user, i would just use the email service straight forwardly without these considerations and background information.

- eric

[Ikai Lan (Google)]

That sounds like a good idea. There's a good chance we'll change the quotas for email sometime in the near future. When we do that it would make sense for us to document that we do not suggest App Engine as a service for sending large amounts of email, and for reliable email delivery, to look at third party services.

Ikai Lan 
Developer Programs Engineer, Google App Engine

Blog: http://googleappengine.blogspot.com 

Twitter: http://twitter.com/app_engine

Reddit: http://www.reddit.com/r/appengine

[nickmilon]

IMHO it is not a good policy for GAE to abandon(?) services middle way
instead of improving - enhancing those. ;-(
Regards
Nick

[Ikai Lan (Google)]

It's not an ideal situation, I agree. However - we're just not at capacity to deal with negotiating with email providers, policing policy violations, and so forth. Providing an email service isn't anywhere near as trivial as people think. We don't want App Engine to become a mass emailing service and we'd rather focus on things like removing limits, hardening production, figuring out how to provide SLAs, and so forth.

In general, we launch products optimistically, but we're much more in favor of providing innovative services rather than pursue improvements in areas where we can't make an impact.

Ikai Lan 
Developer Programs Engineer, Google App Engine

Blog: http://googleappengine.blogspot.com 

Twitter: http://twitter.com/app_engine

Reddit: http://www.reddit.com/r/appengine

[nickmilon]

@Icai
I agree on most of what you write above, and I understand that you
prefer to focus on more important things, also having run Email
services for enterprises in the past I do know it is not trivial.
But ....
still I believe Email service is a major asset for GAE and dropping
it (or anything to that effect) will constitute a major blow to App
Engine.
Gae offers a limited subset of services compared to what a LAMP box
or a IaaS box can offer but been a PaaS provides trouble free
operation and automatic scalability.
Email service usually is part of any web operation so by dropping it
out you farther limit the number of potential applications that fit
well into what GAE offers.
Of course, developers can look into alternative options but this makes
our life difficult since we have to integrate several other third
party services to make a working web solution i.e. setting up multiple
accounts, feed traffic back and forth to other services, having to
monitor and deal with one more possible point of failure. All this
defies to some extend the benefits of GAE as a PaaS.
Also, dropping a service in a time when competition is adding
services, will sent the wrong signal to App Engine's developers/users
echo system and having in mind that G is associated with the best
email service can possibly turn into a PR disaster.

Further more, been a regular reader of the groups and following App
Engine since the very early days I do not see that Email service has
raised a lot of issues. I believe for most people who know what they
are doing and do not abuse the service it works quite smoothly. Some
of the issues raised (mainly spam flagging)
a) happen to the best of Email services b) are addressed by well known
techniques and practices described by others here and elsewhere.

In conclusion:
I would welcome any measure taken to fight service abuse like using
GAE primarily as a mail server - we all understand that this is not
what GAE is all about.
Instead of dropping the service I would prefer to consider:
a) put false positive spam flagging issues under the responsibility of
developers.
b) exclude the service or part of it (like delivery assurances) from
the future SLA offer.
c) think about the technical possibility to integrate it to gmail
which is the *most* reliable email service in town.

Regards
Nick

[n8gray]

Ikai,

I totally understand that you guys want to focus on the interesting parts of the problem, but you could make one really simple change that would probably *solve* the problem for most of your users: allow us to set the DKIM header on outgoing mail.  All it would require is allowing write access to one more header.

See this bug for details:

http://code.google.com/p/googleappengine/issues/detail?id=3161

Cheers,

-n8

[Adam Sah]

Ikai-- totally agree re:bulk email, but please please please add DKIM, which (I'm told) is 90% of the solution for

enterprise/extranet apps that need to use email for workflow.  The Big Reason I chose AppEngine was to get

an integrated solution from one vendor: forcing me to use multiple vendors makes GAE far less compelling.

http://code.google.com/p/googleappengine/issues/detail?id=3161

thanks again!

adam

[nickmilon]

+1

[nickmilon]

+1

[Ikai Lan (Google)]

Points taken. I'll bring up the issue next chance I get.

I can't make any promises though, because some things that should be easy are actually really difficult.

Ikai Lan 
Developer Programs Engineer, Google App Engine

Blog: http://googleappengine.blogspot.com 

Twitter: http://twitter.com/app_engine

Reddit: http://www.reddit.com/r/appengine

+1

[Nathan Gray]

Thanks Ikai, it's good to know that the issue will be considered.

--
HexaLex: A New Angle on Crossword Games for iPhone and iPod Touch
http://hexalex.com
On The App Store: http://bit.ly/8Mj1CU
On Facebook: http://bit.ly/9MIJiV
On Twitter: http://twitter.com/hexalexgame
http://n8gray.org

[Greg]

AWESOME!

This is my biggest issue at the moment - something like 25% of my
customers use Yahoo Mail, and don't receive emails they have
requested. DKIM will solve the problem, because I've got it working on
Google Apps for the same email addresses, and those messages get
through. I know this is caused by Yahoo's broken filtering system, but
not being able to rely on email delivery is a pain-point for apps that
uses email, and I'm guessing that would be the majority of apps.

I'd appreciate it if you could also let us know if the decision goes
against a speedy implementation of DKIM headers, because I'll need to
figure out another solution in short order.

Cheers!
Greg.

On Apr 19, 6:47 am, "Ikai Lan (Google)" <ika...@google.com> wrote:
> Points taken. I'll bring up the issue next chance I get.
>
> I can't make any promises though, because some things that should be easy
> are actually really difficult.
>
> Ikai Lan
> Developer Programs Engineer, Google App Engine
> Blog:http://googleappengine.blogspot.com
> Twitter:http://twitter.com/app_engine
> Reddit:http://www.reddit.com/r/appengine
>