[Django] #26802: Sending mails with attachment results in 'bytes' object has no attribute 'encode'
Django
'encode'
-----------------------------+--------------------
Reporter: Brandl | Owner: nobody
Type: Bug | Status: new
Component: Core (Mail) | Version: 1.9
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------+--------------------
When trying to send a EMail with attachment, it always failed with the
following Exception:
{{{
'bytes' object has no attribute 'encode'
}}}
At first I thought this is a bug in django-post-office or a duplicate of
this bug:
https://code.djangoproject.com/ticket/24623
But I'm running the newest Version of Django(1.9.7) and django-
post_office(2.0.7) on Python Python 3.5.1
Here is the trace:
{{{
File "/home/me/Projects/MyProject/.env/lib/python3.5/site-
packages/post_office/models.py", line 119, in dispatch
self.email_message(connection=connection).send()
File "/home/me/Projects/MyProject/.env/lib/python3.5/site-
packages/django/core/mail/message.py", line 292, in send
return self.get_connection(fail_silently).send_messages([self])
File "/home/me/Projects/MyProject/.env/lib/python3.5/site-
packages/django/core/mail/backends/smtp.py", line 107, in send_messages
sent = self._send(message)
File "/home/me/Projects/MyProject/.env/lib/python3.5/site-
packages/django/core/mail/backends/smtp.py", line 121, in _send
message = email_message.message()
File "/home/me/Projects/MyProject/.env/lib/python3.5/site-
packages/django/core/mail/message.py", line 256, in message
msg = self._create_message(msg)
File "/home/me/Projects/MyProject/.env/lib/python3.5/site-
packages/django/core/mail/message.py", line 344, in _create_message
return self._create_attachments(msg)
File "/home/me/Projects/MyProject/.env/lib/python3.5/site-
packages/django/core/mail/message.py", line 357, in _create_attachments
msg.attach(self._create_attachment(*attachment))
File "/home/me/Projects/MyProject/.env/lib/python3.5/site-
packages/django/core/mail/message.py", line 399, in _create_attachment
attachment = self._create_mime_attachment(content, mimetype)
File "/home/me/Projects/MyProject/.env/lib/python3.5/site-
packages/django/core/mail/message.py", line 370, in
_create_mime_attachment
attachment = SafeMIMEText(content, subtype, encoding)
File "/home/me/Projects/MyProject/.env/lib/python3.5/site-
packages/django/core/mail/message.py", line 171, in __init__
MIMEText.__init__(self, _text, _subtype, None)
File "/usr/lib64/python3.5/email/mime/text.py", line 34, in __init__
_text.encode('us-ascii')
AttributeError: 'bytes' object has no attribute 'encode'
}}}
As it turns out:
{{{
if _charset == 'utf-8':
# Unfortunately, Python < 3.5 doesn't support setting a
Charset instance
# as MIMEText init parameter
(http://bugs.python.org/issue16324).
# We do it manually and trigger re-encoding of the payload.
MIMEText.__init__(self, _text, _subtype, 'utf-8')
}}}
instead of
{{{
MIMEText.__init__(self, _text, _subtype, None)
}}}
fixes the bug, but I'm not sure if that's a clean solution.
--
Ticket URL: <https://code.djangoproject.com/ticket/26802>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: new
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
Can you please give steps to reproduce? Ideally, a test case for
`tests/mail/tests.py`.
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:1>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: closed
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* status: new => closed
* resolution: => needsinfo
Comment:
I'm surprised that bytes are given as text input.
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:2>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: closed
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Brandl):
{{{
from post_office import mail
mail.send(
'so...@email.com',
'so...@email.com',
subject='My email',
message='Hi there!',
attachments={
'manage.py': 'manage.py',
},
priority='now',
)
}}}
As I said, the exception happens while using a third-party library, but
it's basically a wrapper around the Django core mail functionality:
https://github.com/ui/django-
post_office/blob/master/post_office/models.py#L95
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:3>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: closed
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by timgraham):
Could you please give steps to reproduce without a third-party library to
confirm that it's not a bug there?
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:4>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: new
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* status: closed => new
* resolution: needsinfo =>
Comment:
I have been able to reproduce. This happens because post-office attach
files with binary content (using `FileField.read()`) whatever the mime
type (see for example how Django distinguish file read mode for text-based
attachments in `EmailMessage.attach_file`).
I have not made my mind yet if and how Django should safeguard against
such issues...
Test to reproduce:
{{{
def test_attach_text_as_bytes(self):
msg = EmailMessage('subject', 'body', 'fr...@example.com',
['t...@example.com'])
file_path = os.path.join(os.path.dirname(upath(__file__)),
'attachments', 'file.txt')
with open(file_path, mode='rb') as fh:
msg.attach('file.txt', fh.read())
msg.send()
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:5>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: new
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* stage: Unreviewed => Accepted
Comment:
Accepting for further investigation/thought.
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:6>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: new
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Brandl):
That may be naive, but am I really the first one, who encounters a problem
with saving something in a FileField and then sending it per Mail?
So this is where the file gets stored:
{{{
if isinstance(content, string_types):
# `content` is a filename - try to open the file
opened_file = open(content, 'rb')
content = File(opened_file)
attachment = Attachment()
attachment.file.save(filename, content=content, save=True)
}}}
This how it gets attached to the mail:
{{{
msg.attach(attachment.name, attachment.file.read())
}}}
At the moment the Django offers two methods of attaching files:
{{{
def attach_file(self, path, mimetype=None):
}}}
and
{{{
def attach(self, filename=None, content=None, mimetype=None):
}}}
The first one has the benefit of some sophisticated Mimetype guessing,
when reading the file, but does not allow to supply a filename, so when
the attachment names get hashed, this would destroy the name information.
The other one does allow this, but seemingly throws an exception, when
provided with a binary file and no Mimetype.
So what are my/our options here? Of course me and others, who encounter
the same problem, would need to replicate the functionality of
attach_file, but since the implementation is by no means straight forward,
I would prefer Django would provide a convenience method for this, maybe
even in the Django FileField.
Since the Django FileField also has a path attribute, I could also utilize
the attach_file(), method, but then I would love to have a way of
overriding the file name.
Or maybe I forgot something more simple than that? Also I wonder, why my
quick fix:
{{{
MIMEText.__init__(self, _text, _subtype, 'utf-8')
}}}
is solving this more complicated problem and in which cases that fix would
still cause an exception?
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:7>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: new
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by claudep):
The reason we are using `None` instead of `utf-8` as the charset is that
with `utf-8`, Python will take its default 'utf-8' Charset instance which
does BASE64 body encoding. And we don't want to use that body encoding, we
use either Quoted-printable or None at all (that was
[ececbe77ff573707d8f25084018e66ee07f820fd] and recently
[836d475afefecd643d5e7f44027d7209df3ac690]).
We could easily fix this in Python 3.5 by using a real Charset instance
instead of `'utf-8'`). Python 2.7 is not affected because there is no
charset sniffing with `encode()`. We are left with Python 3.4, which we
could special-case and decode the text before passing it to
`MIMEText.__init__`. I'll suggest a patch.
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:8>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: new
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* has_patch: 0 => 1
Comment:
Unfortunately, I'm just realizing that the fix proposed to Python in
http://bugs.python.org/issue16324 only partially fixes the issue, as the
Charset instance isn't pass to the `set_payload()` as is. We'll have to
keep the workaround for some more years :-(.
[https://github.com/django/django/pull/6860 Patch] updated.
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:9>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: new
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Ready for checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:10>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: closed
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Ready for checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Claude Paroz <claude@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"04b7b28812fa6b2721096da6e00b929784c00da6" 04b7b28]:
{{{
#!CommitTicketReference repository=""
revision="04b7b28812fa6b2721096da6e00b929784c00da6"
Fixed #26802 -- Prevented crash when attaching bytes as text message
Thanks Tim Graham for the review.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:11>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: closed
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Ready for checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Harris Lapiroff):
What version of Django is this set to be merged into? I ran into this bug
on 1.10.3 and 1.10.4.
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:12>
Django
'encode'
Reporter: Brandl | Owner: nobody
Type: Bug | Status: closed
Component: Core (Mail) | Version: 1.9
Keywords: | Triage Stage: Ready for checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Claude Paroz):
This was not backported to 1.10, so you'll have to wait for Django 1.11.
--
Ticket URL: <https://code.djangoproject.com/ticket/26802#comment:13>