Tickets on send_mass_mail functionality (multipart/encoding/TLS/BCC)
Simon G.
I've spent some time looking over the email related tickets in Trac,
and think I've got them sorted out a bit. I'm hoping to spur some
discussion on some of these, so here's a brief recap.
We currently have requests to:
#1541 - Add Multipart Message capability to mail
This ticket's first birthday is creeping up. There's a patch here
that adds a safe method for sending multipart messages. Jacob has said
this is great, but it needs docs. HOWEVER, Gary Wilson's pointed out
that we're getting a rather ugly data tuple going on, and created 3366
->
#3366 - objectify send_mass_mail parameters
This aims to clean up the send_mass_mail arguments by plugging them
into an object. This looks fairly sensible, and has a patch, but has
some backwards incompat. issues. This needs a design decision - and if
it's "yes", then the other tickets will probably need to be rewritten
to take this into account.
#2007 - Enable email sending in a non-default encoding.
A pure feature request to add a new setting that could override the
charset that emails are sent in. No patch, and I'm not sure how useful
this would really be. Let me know if I'm wrong here.
#2897 - Support TLS
This has a simple patch, and could add some nice functionality.
Adrian wontfixed it 6 months ago, but it's been reopened and a couple
of people are arguing that it's a good feature. [yes/no]?
#3307 - Add BCC to mail framework.
I've renamed this ticket from what it used to be. This originally had
two issues - one which overlapped with #1541, and the other this BCC
request. There's a patch here with adds a parameter to send_mass_mail
to suppress the recipient list (i.e. by BCC'ing them), but Gary
Wilson's argued that it's more logical to have two parameters:
recipient_list & bcc_list. So - this just needs a design decision on
what path to take (if any).
#3605 - Simple patch to give content-type definition for sending html
mails.
This one looks like a quick design decision.
Tickets:
http://code.djangoproject.com/ticket/1541
http://code.djangoproject.com/ticket/2007
http://code.djangoproject.com/ticket/2897
http://code.djangoproject.com/ticket/3307
http://code.djangoproject.com/ticket/3366
http://code.djangoproject.com/ticket/3605
Comments?
--Simon
Malcolm Tredinnick
> Good Evening Djuggernauts,
>
> I've spent some time looking over the email related tickets in Trac,
> and think I've got them sorted out a bit. I'm hoping to spur some
> discussion on some of these, so here's a brief recap.
Nice work, again, Simon.
>
> We currently have requests to:
>
> #1541 - Add Multipart Message capability to mail
> This ticket's first birthday is creeping up. There's a patch here
> that adds a safe method for sending multipart messages. Jacob has said
> this is great, but it needs docs. HOWEVER, Gary Wilson's pointed out
> that we're getting a rather ugly data tuple going on, and created 3366
The main reason I suspect it hasn't been resolved one way or the other
is in Russell's comment #23 in early March: it's become a nest of
competing implementations, which is never an easy thing to untangle.
Having discussion in bugs is good, but it sometimes ends up like that.
I haven't read the patches closely, but my immediate concern is whether
we are duplicating too much of Python's existing mail framework and just
wrapping it. That would be bad design. I have no opinion on this yet.
> #3366 - objectify send_mass_mail parameters
> This aims to clean up the send_mass_mail arguments by plugging them
> into an object. This looks fairly sensible, and has a patch, but has
> some backwards incompat. issues. This needs a design decision - and if
> it's "yes", then the other tickets will probably need to be rewritten
> to take this into account.
If things like #1541 are deemed useful, this is a probably a good idea.
Particularly because so many of the arguments are optional.
I'm not sure why Gary wants to use an object, rather than a dictionary,
since dictionaries are often easier to construct in client code if
you're doing it all in once place. There's also nothing really object-y
about the data: it's just a bag of name/value pairs. Using an object
makes using the values easier (foo.bar versus foo["bar"]), but creating
them is more verbose.
The principle seems sound; we should decide about this pre-1.0.
>
> #2007 - Enable email sending in a non-default encoding.
> A pure feature request to add a new setting that could override the
> charset that emails are sent in. No patch, and I'm not sure how useful
> this would really be. Let me know if I'm wrong here.
Not really sure of the benefits of this. The main motivation given in
the comments is to avoid the base-64 encoding, which is #3472 (you
missed that one). I think we should fix #3472 in some way (I'm with
Jacob on not quite understanding the patch fully, yet, but it looks like
the right idea) and then if people want ultimate customisation they have
the Python email support anyway. Django's email stuff is just for
convenience in common cases.
>
> #2897 - Support TLS
> This has a simple patch, and could add some nice functionality.
> Adrian wontfixed it 6 months ago, but it's been reopened and a couple
> of people are arguing that it's a good feature. [yes/no]?
Mostly because of Google (as mentioned in one of the comments), I would
vote to fix this.
> #3307 - Add BCC to mail framework.
> I've renamed this ticket from what it used to be. This originally had
> two issues - one which overlapped with #1541, and the other this BCC
> request. There's a patch here with adds a parameter to send_mass_mail
> to suppress the recipient list (i.e. by BCC'ing them), but Gary
> Wilson's argued that it's more logical to have two parameters:
> recipient_list & bcc_list. So - this just needs a design decision on
> what path to take (if any).
I'd argue again that complex stuff can already be done with the Python
framework. Why are we wrapping it? So I guess I'm -1 on this at the
moment (I'm willing to wrong about why we are wrapping certain things
and not others).
> #3605 - Simple patch to give content-type definition for sending html
> mails.
> This one looks like a quick design decision.
Anything that encourages people to send pure HTML email without
including a text copy deserves to be wontfix-ed. :-)
It has the same problem as you pointed out earlier: yet another
parameter. I'm probably +0 on this one (not enthused, but it wouldn't
hurt too much).
We need an easy way to send 404 and 500 emails and some integration with
the site's settings (the latter being the motivation for wrapping so
much stuff, I guess). Most other stuff seems like gravy.
For my money, #3472 and #2897 look like important ones that need fixing
soon. #3366 looks like a good idea for period between 0.96 and 1.0
because of the backwards-compat changes. For the rest, I'm willing to
defer to others, since I don't use them and so am probably being too
negative.
Regards,
Malcolm
James Bennett
> The main reason I suspect it hasn't been resolved one way or the other
> is in Russell's comment #23 in early March: it's become a nest of
> competing implementations, which is never an easy thing to untangle.
> Having discussion in bugs is good, but it sometimes ends up like that.
I'd like to see this happen just because sending multipart email --
mostly with an HTML version and a plain-text version -- is a fairly
common use case, and I feel like we ought to make the common case as
easy as possible. Not sure which implementation I like best, though;
I'll have to poke through and have a look at them all.
> If things like #1541 are deemed useful, this is a probably a good idea.
> Particularly because so many of the arguments are optional.
Agreed on this, and with doing it as a dictionary instead of an
object. Jacob's concerns about backwards-compatibility are worth
noting, though, so this one may have to wait.
> Not really sure of the benefits of this. The main motivation given in
> the comments is to avoid the base-64 encoding, which is #3472 (you
> missed that one). I think we should fix #3472 in some way (I'm with
> Jacob on not quite understanding the patch fully, yet, but it looks like
> the right idea) and then if people want ultimate customisation they have
> the Python email support anyway. Django's email stuff is just for
> convenience in common cases.
Fixing the encoding is the right thing to do here, since that seems to
be the root of the spam-filtering problem.
> Mostly because of Google (as mentioned in one of the comments), I would
> vote to fix this.
+1
> I'd argue again that complex stuff can already be done with the Python
> framework. Why are we wrapping it? So I guess I'm -1 on this at the
> moment (I'm willing to wrong about why we are wrapping certain things
> and not others).
I won't argue for or against this one since I submitted it. It's just
a modified version of something we've been using because we had to
manage situations where we'd send mail to lots of people and didn't
want them all seeing each others' addresses (not sure if mailing lists
like that are a common enough use case, and I'd almost be willing to
argue that it's better to just set up an actual mailing-list manager
anyway).
> Anything that encourages people to send pure HTML email without
> including a text copy deserves to be wontfix-ed. :-)
:)
> We need an easy way to send 404 and 500 emails and some integration with
> the site's settings (the latter being the motivation for wrapping so
> much stuff, I guess). Most other stuff seems like gravy.
Yeah, though being able to send HTML email from
send_mail/send_mass_mail does have some big benefits (think dumping
that pretty traceback screen into the email when DEBUG=False). I'd
like to see that come in in some form.
--
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."
Russell Keith-Magee
>
> On Thu, 2007-03-08 at 04:09 -0800, Simon G. wrote:
> > Good Evening Djuggernauts,
> >
> > I've spent some time looking over the email related tickets in Trac,
> > and think I've got them sorted out a bit. I'm hoping to spur some
> > discussion on some of these, so here's a brief recap.
>
> Nice work, again, Simon.
Ditto. This sort of filtering isn't easy - good stuff.
> > We currently have requests to:
> >
> > #1541 - Add Multipart Message capability to mail
> > This ticket's first birthday is creeping up. There's a patch here
> > that adds a safe method for sending multipart messages. Jacob has said
> > this is great, but it needs docs. HOWEVER, Gary Wilson's pointed out
> > that we're getting a rather ugly data tuple going on, and created 3366
>
> The main reason I suspect it hasn't been resolved one way or the other
> is in Russell's comment #23 in early March: it's become a nest of
> competing implementations, which is never an easy thing to untangle.
> Having discussion in bugs is good, but it sometimes ends up like that.
There have been a few attempts to get a mailing list discussion
started in the past few months, but nobody has taken a serious swing
at the problem.
> I haven't read the patches closely, but my immediate concern is whether
> we are duplicating too much of Python's existing mail framework and just
> wrapping it. That would be bad design. I have no opinion on this yet.
Looking over the patches again, I share this concern. The original
intent of send_mail was to provide a simple "just send a message" API,
mostly for the purposes of emailing the site admin error messages.
James' comment about mailing perty HTML stack traces is a good one,
but otherwise the python email libs provide a much richer API, with
exposure of the full set of possible email options.
I'm +1 on exposing some extra options that fit with existing options
(like SSL sending and BCC's - 2897,3366), and the problems with
charset encoding should probably be fixed, too (2007,3472).
Personally, I'm not a big fan of HTML email, but I seem to be in the
minority on this one; as long as the interface (1) remains simple, and
(2) enforces best practice (such as requiring text as well has HTML),
I could live with the change. Some cleanup of option passing might be
called for as well (3366, but I prefer Malcolms dict-based approach).
However, over all these changes, we need to be careful that we don't
just replicate the python mail APIs.
Yours
Russ Magee %-)
Gary Wilson
wrote:
> On Thu, 2007-03-08 at 04:09 -0800, Simon G. wrote:
> > Good Evening Djuggernauts,
>
> > I've spent some time looking over the email related tickets in Trac,
> > and think I've got them sorted out a bit. I'm hoping to spur some
> > discussion on some of these, so here's a brief recap.
>
> Nice work, again, Simon.
Yes, good work!
> > #3366 - objectify send_mass_mail parameters
> > This aims to clean up the send_mass_mail arguments by plugging them
> > into an object. This looks fairly sensible, and has a patch, but has
> > some backwards incompat. issues. This needs a design decision - and if
> > it's "yes", then the other tickets will probably need to be rewritten
> > to take this into account.
The send_mail() and send_mass_mail() could be re-implemented to
maintain backwards compatibility.
> If things like #1541 are deemed useful, this is a probably a good idea.
> Particularly because so many of the arguments are optional.
>
> I'm not sure why Gary wants to use an object, rather than a dictionary,
> since dictionaries are often easier to construct in client code if
> you're doing it all in once place. There's also nothing really object-y
> about the data: it's just a bag of name/value pairs. Using an object
> makes using the values easier (foo.bar versus foo["bar"]), but creating
> them is more verbose.
I figured an object would be nice since we have a clump of related
data and could have some nice methods for working with that data, like
send(), add_attachment(), __str__(), etc. I had just written a lot of
code that sent different emails and was getting fed up with all the
tuples.
A dictionary would be a little awkward since you don't exactly have
required keys. Verboseness might be slightly increased, but so is
readability and introspection.
message = {'to': email_list, 'from': m...@email.com, 'subject': "Test",
'body': "hello world"}
or
message = EmailMessage(to=email_list, from='m...@email.com',
subject="Test", body="hello world)
Criticism welcome.
> The principle seems sound; we should decide about this pre-1.0.
I asked Jacob to take a quick look at this ticket during the PyCon
sprint. His comments were (1) the send method on EmailMessage is
kinda neat and (2) we would need to keep send_mail() and
send_mass_mail() functions around for backwards compatibility, but
could deprecate them for 1.0. From my standpoint, however, he made no
definite decision.
> For my money, #3472 and #2897 look like important ones that need fixing
> soon. #3366 looks like a good idea for period between 0.96 and 1.0
> because of the backwards-compat changes. For the rest, I'm willing to
> defer to others, since I don't use them and so am probably being too
> negative.
I agree that #3472 and #2897 are probably the two most important.
Other than that, I would like multipart and BCC capabilities because I
think these have good use cases, like HTML+plaintext newsletters for
example.
Gary
Gary Wilson
> we are duplicating too much of Python's existing mail framework and just
> wrapping it. That would be bad design. I have no opinion on this yet.
Speaking of duplicating Python's mail framework, it appears that the
Message-ID header stuff that was added recently could instead be done
with Python's make_msgid() function.
Jacob Kaplan-Moss
> I asked Jacob to take a quick look at this ticket during the PyCon
> sprint. His comments were (1) the send method on EmailMessage is
> kinda neat and (2) we would need to keep send_mail() and
> send_mass_mail() functions around for backwards compatibility, but
> could deprecate them for 1.0. From my standpoint, however, he made no
> definite decision.
Sorry for my radio silence on this; I've been busy elsewhere...
This is a good summary of how I feel. I do agree with Malcolm that
dicts are slightly more transparent than objects, but I think I like
Gary's idea of letting message objects grow add_attachment(), etc. As
an added bonus, letting the message object send() itself makes it easy
to mock that function during testing (currently, testing email sending
kinda sucks).
The other nice thing about this is that the change can be made in a
backwards-compatible way: we can keep send_mail() and friends around,
but change the docs to reflect the new way of doing things (construct
an EmailMessage and then send() it).
Jacob
Lachlan Cannon
> #3307 - Add BCC to mail framework.
> I've renamed this ticket from what it used to be. This originally had
> two issues - one which overlapped with #1541, and the other this BCC
> request. There's a patch here with adds a parameter to send_mass_mail
> to suppress the recipient list (i.e. by BCC'ing them), but Gary
> Wilson's argued that it's more logical to have two parameters:
> recipient_list & bcc_list. So - this just needs a design decision on
> what path to take (if any).
I'd argue that both BCC and Reply-to are common enough that their inclusion in
the e-mail framework would be extremely useful.
Reply-to in particular, since some hosts won't let you send e-mails from
addresses you haven't registered on their servers, but if you're building a
quick and dirty contact form you don't want to have to copy and paste an address
to your to field to make a reply.
I think bcc is pretty useful too for limited newsletter use. Obviously for big
stuff you'll need to look at your own implementation, but for the little stuff I
think we should support it.
Sorry if this e-mail is a little brief, I'm trying to catch up on the last few
weeks of e-mails here with not much time. Let me know if I need to expand on
anything.
--
Lach
Personal: http://illuminosity.net/