[Bug 2920] Discussion about deleting attachments
Charles Cooley
> (note: this is a reply to comment 71 on Bugzilla for bug 2920
> <http://bugzilla.mozilla.org/show_bug.cgi?id=2920#c71>)
>
>> Is deleting the attachment data an mbox violation?
Only if you think of the mbox format as a "standard" that can be violated.
Given mbox's history, pretty much anything goes. In fact there isn't just
one mbox format, but a collection of, sometimes incompatible, formats.
> > [...] mbox requires that messages be stored in rfc822 format. The
> issue isn't really mbox, it's rfc822. By removing the attachment data
> you are making a new message which is missing data from the original
> message, and by retaining the message id you have caused your message to
> be different from the message that all other recipients and the sender.
Technically, by rfc822 standards it may be a new message, but in the
eyes of those of us who want this feature, it's the same message with
an attachment removed. Many of us make a distinction between the simple
body of an message and the MIME (or even UUEncoded) attachments to the
message. Even Mozilla's message viewer and composer make this distinction
since the "text" of the message shows in the message pane and the
attachments are treated as separate but related items.
The mbox format uses rfc822 format for messages only because that's the
format coming in from the network and is the easiest to store on disk.
There is nothing magical or sacred about it. I will agree that deleting
an attachment or making other changes and not leaving some trace of the
modification is wrong, but only because it would mislead someone into
believing that the current version is the original version of the message.
> > [...] if i'm the sender and i mean to send you a picture and you
> decide to remove my picture from my message, you have changed what I
> intended to convey, and therefor by the rfc you are obligated to use a
> new unique message id.
As the reciever, I'm not obligated to anything other than not resending
an altered version of the message as if it were the original.
> > [...] If I send you an image/gif and you store an rfc822 image/png
> then i expect that it will have a different Message-ID.
Why? It's the same content. After all is it the image you sent or it's
encoding? From a user's perspective it's the same. If I send a plain
text 7-bit ASCII message to someone using a UNICODE based system, then
I fully expect my message to be encoded and stored in UNICODE using the
original Message-ID even though it's not in ASCII anymore.
> > [...] Imagine rfc822 as an envelope (see rfc822 1.1), mbox stores it
> and magically allows you to examine its contents without violating the
> seal (Message-ID), what you are doing is taking an envelope and breaking
> its seal, removing pieces from it and then sealing it up again and
> claiming you haven't violated the sanctity of the seal -- which clearly
> you have.
> >
> > [...] once you've broken the seal, it should be trivial for you to
> add extra contents, the issue is with the seal. Change the Message-ID
> and do whatever damage you need to do.
I don't agree with the Message-ID as Seal analogy. It's an identifier not
a security and integrity safeguard. Yes, technically the rfc822 message is
being altered, but if we remove an attachment and clearly note our change,
then I consider this to be fundamentally the same message. In the real
world there are cases where a multi-page memo is sent to a group of people
and then copies of only certain pages are forwarded to others (for security
or just paper-saving reasons). I've never heard anyone claim that the
partial memo was a NEW memo, just an incomplete one. Now if we rewrite
sections or even add new material (other than "look at what John sent me"),
that would constitute a new memo, but just omitting a few pages doesn't.
> Is it possible to change the message ID without breaking threading?
No, not if you are talking about real threading using the References
header. If the message ID changes then you have a new message that is
not part of the thread.
> If not, then between breaking threading (changing the message ID) and
> violating the mbox seal (not changing the message ID), is one of the two
> actions acceptable, at least as an option that can be disabled?
Dropping an attachment and keeping the Message-ID is the behavior that
many (most?) of us expect. We aren't creating a new message, we are
just removing part of an existing one.
> If neither of the two actions is acceptable, then there's no reasonable
> way to delete attachments; however, consider that attachment deletion
> and/or externally autosaving attachments is a feature in some
> widely-used e-mail clients (mutt, Eudora, and Outlook; and Pegasus
> offers some form of this). So what do these e-mail clients do with
> their handling of removing attachments that apparently (?) is accepted
> as an adequate solution?
>
> Eudora does not change the message id when they externally autosave the
> attachments. Eudora replaces the attachment section with text like the
> following:
> Attachment Converted: "c:\program files\eudora\attach\filename.ext"
Yes, this is exactly the behavior that I would expect.
> Is this a case of an mbox violation? If so, is there adequate
> documentation indicating that the violation occurred, such that the
> violation itself wouldn't pose any legal risk to individuals if they had
> to provide such an e-mail in court? (Although the seal has been broken,
> the documentation (the "Attachment Converted..." text) openly declares
> that the seal was broken, and it openly states what was removed from the
> e-mail.)
> Whether or not this, technically speaking, is a violation of the rfc822
> format, would it pose any legal risk to individuals who are in
> possession of such altered e-mails?
Legal risk? You'd have to ask a lawyer. But the Message-ID is NOT a
seal or any other type of security or integrity check. The only legal
issue I can see would be if you stripped an attachment or modified the
contents of the message WITHOUT documenting the change and then forwarded
the changed message to others or submitted it as evidence in court while
claiming that it was the original, unaltered message. If you state that
it was changed, then I don't see how there can be a problem (unless of
course you intentionally deleted an attachment that contained critical
evidence of some sort, but in that case you probably wouldn't want to
tell anyone that you did it and it would be much easier to just delete
the entire message). Again, if someone sends me a four page memo and I
decide to throw away page 3, where does "legal risk" come into play?
> > [...] I suppose a corporation could set a policy prohibitting people
> from deleting attachments :-). ok ... so maybe you do need to allow it
> to be disabled. w/o that argument I would have said you shouldn't
> provide the option to disable it.
Why? Do they also prohibit people from deleting messages? While I can't
say there isn't some Dilbert-style boss that has made that policy, I
don't see a reason for Mozilla to provide technological enforcement. If
a company really wants to keep information, it should be trapping data
at the servers during delivery not relying on the recipients.
Charles Cooley
Matt Coughlin
>> threading?
>
> No, not if you are talking about real threading using the
> References header. If the message ID changes then you have
> a new message that is not part of the thread.
Well, that pretty much rules out changing the message ID.
As for the option of leaving the message ID as is, there's bound to be a
feasible way to approach this, a way that would allow the real-world
need for this to be met, while also alleviating (to a reasonable extent)
concerns about altering a received e-mail - some combination of adding
additional info to the e-mail, along with (possibly) allowing people to
confirm, undo, or disable the deletion of attachments.
Personally, I don't care if the rfc822 format is broken, technically
speaking, as long as any changes made don't hinder the user from later
migrating away from Mozilla if they choose to (mbox portability). I'd
be resistant to making any change to received e-mails that would cause
the mailbox to be "best viewed in Mozilla".
> As the reciever, I'm not obligated to anything other than
> not resending an altered version of the message as if it
> were the original.
Whatever changes that are made to the e-mail need to address this
concern in a generic-enough way so that if the user later migrates to a
different e-mail client and resends the e-mail from that other e-mail
client, it will still be obvious (easily or upon close internal
inspection) that the e-mail was an altered version of the original message.
I'll include this in the general concerns about ensuring mbox portability.
> [...] I will agree that deleting an attachment or making other
> changes and not leaving some trace of the modification is wrong,
> but only because it would mislead someone into believing that
> the current version is the original version of the message.
What would be a minimal amount of info that would adequately indicate
(visibly or non-visibly) that the e-mail was altered by the e-mail client?
One suggestion was to add a new header type, along the line of
"X-Mozilla-Altered: <whatever info here>".
Some other suggestions involve adding visible info, such as "Attachment
deleted: <name> [<type>] [<size>] [<date & time>]".
Does there have to be some visible info added (like the above, which
might be just as useful in any other e-mail client)?
Does there have to be some non-visible internal info added (like
X-Mozilla-Altered)? If Mozilla makes use of this extra internal info
though (as a way to keep track of which e-mails have been altered, and
in what way), that could be perceived as interfering with mbox
portability, since no other e-mail client could be reasonably expected
to support this Mozilla-specific tracking of alterations.
If on the other hand this info is only added for posterity and not made
use of in any way, then there wouldn't be any functionality related to
tracking alterations that the user would lose if they migrated to a
different e-mail client, and hence no loss of mbox portability.
Internal info of this sort could at least indicate (on close inspection)
that Mozilla was the e-mail client that altered the e-mail (no idea if
there'd ever be a need to know that though).
If visible info needs to be added, does it have to be appended to the
viewable text of the e-mail (including the end of every alternate
version of the viewable text. i.e. html vs plain text)?
Or would the attachment have to be replaced with a different attachment
that contains text info about the deletion? (I suspect this approach
would be very impractical and would interfere too much with mbox
portability, but I thought I'd toss the idea out there for good measure.)
Any viewable info along the lines of a hyper-link to an externally-saved
location of the file (not presently considered for this current bug, but
mentioned since it's still under much discussion) could be perceived as
interfering with mbox portability, unless it made use of a link
mechanism that just about any reasonable e-mail client (that supports
mbox) would support (i.e. unless the user could still easily access
their attachments if they migrate to a different e-mail client).
As food for thought, Eudora saves attachments externally, but Mozilla is
able to put the attachments back inside the e-mails when it imports the
mailboxes. I don't know if it'd be reasonable though to depend on other
e-mail clients to undo any loss of mbox portability caused by Mozilla.
Perhaps it wouldn't be as much of an issue if Mozilla itself allowed the
user to restore Mozilla's own externally-stored attachments back inside
the e-mail (most likely no one would seriously consider adding
functionality like that though, including myself).
(Realistically, some loss of mbox portability would probably be
acceptable. But for now I'm taking the viewpoint of resisting any loss
of mbox portability, in the interest of minimizing such loss, and to
ensure that there's at least one such view in the discussion. A good
mixture of views will still be very much needed in the discussion.)
> [...] The only legal issue I can see would be if you stripped
> an attachment or modified the contents of the message WITHOUT
> documenting the change and then forwarded the changed message
> to others or submitted it as evidence in court while claiming
> that it was the original, unaltered message.
Some viewable info, as described above, most likely in the body of the
e-mail itself, would probably be needed so that the user could easily
remember that the e-mail has been altered. This would be a reasonable,
but not excessive, effort to help prevent the user from presenting the
e-mail to others and, unintentionally, mistakenly claiming that it was
unaltered. The rest would be up to the user to not be overly forgetful
or dishonest.
Someone else mentioned about being able to optionally not store
attachments in sent messages in the "sent" folder.
Do you see any concerns specific to this type of an action? (in the
event that this feature would one day be implemented)
>>> [...] I suppose a corporation could set a policy prohibitting
>>> people from deleting attachments :-). ok ... so maybe you do
>>> need to allow it to be disabled. w/o that argument I would have
>>> said you shouldn't provide the option to disable it.
>
> Why? Do they also prohibit people from deleting messages? While
> I can't say there isn't some Dilbert-style boss that has made that
> policy, I don't see a reason for Mozilla to provide technological
> enforcement. If a company really wants to keep information, it
> should be trapping data at the servers during delivery not relying
> on the recipients.
I'd only consider adding this option if there was a strong demand for it
or if there was a very compelling reason to add it.
I expect there might be a reasonable need to allow the users to confirm
and/or undo the deletion of attachments. Part of the decision on the
undo option may depend on how difficult or awkward it turns out to be to
add that functionality. It's also possible that an undo operation would
absolutely have to be provided.
If there's an undo, then maybe it wouldn't be necessary to have a
confirmation window (one less annoying thing to deal with). Certainly
at the bare minimum, either a confirmation window or an undo operation
would be needed.
thanks for all the feedback :-)
--
Matt Coughlin
d...@sp4m-less.forestpath.org
<remove "sp4mless_" from the e-mail address to reply>
Charles Cooley
Replacing the actual attachment with a simple text file attachment
describing the deleted contents is a very good idea. Those who view
attachments in-line will automatically see the notification and
those who don't will see it when they try to open the attachment.
An extra header would also be appropriate as an additional way for
people and software to know the message was altered.
> Any viewable info along the lines of a hyper-link to an externally-saved
> location of the file (not presently considered for this current bug, but
> mentioned since it's still under much discussion) could be perceived as
> interfering with mbox portability, unless it made use of a link
> mechanism that just about any reasonable e-mail client (that supports
> mbox) would support (i.e. unless the user could still easily access
> their attachments if they migrate to a different e-mail client).
>
> As food for thought, Eudora saves attachments externally, but Mozilla is
> able to put the attachments back inside the e-mails when it imports the
> mailboxes. I don't know if it'd be reasonable though to depend on other
> e-mail clients to undo any loss of mbox portability caused by Mozilla.
> Perhaps it wouldn't be as much of an issue if Mozilla itself allowed the
> user to restore Mozilla's own externally-stored attachments back inside
> the e-mail (most likely no one would seriously consider adding
> functionality like that though, including myself).
>
>
> (Realistically, some loss of mbox portability would probably be
> acceptable. But for now I'm taking the viewpoint of resisting any loss
> of mbox portability, in the interest of minimizing such loss, and to
> ensure that there's at least one such view in the discussion. A good
> mixture of views will still be very much needed in the discussion.)
Actually, this isn't at all similar to what Eudora does. Eudora
physically separates the attachments from the messages for storage
but still considers them part of the original message and maintains
enough information to be able to put things back together when
presented to the user. We are discussing removing the attachment
entirely. Since the user intends to DELETE the attachment, I don't
see the need to be able to reconstruct the original. If you implement
the "save and delete" option then it would make sense to record the
save location in the delete notification message. That could in
turn be used to reconstruct the original (but only if the saved file
isn't later moved or altered).
>> [...] The only legal issue I can see would be if you stripped
> > an attachment or modified the contents of the message WITHOUT
>> documenting the change and then forwarded the changed message
> > to others or submitted it as evidence in court while claiming
> > that it was the original, unaltered message.
>
> Some viewable info, as described above, most likely in the body of the
> e-mail itself, would probably be needed so that the user could easily
> remember that the e-mail has been altered. This would be a reasonable,
> but not excessive, effort to help prevent the user from presenting the
> e-mail to others and, unintentionally, mistakenly claiming that it was
> unaltered. The rest would be up to the user to not be overly forgetful
> or dishonest.
>
>
> Someone else mentioned about being able to optionally not store
> attachments in sent messages in the "sent" folder.
>
> Do you see any concerns specific to this type of an action? (in the
> event that this feature would one day be implemented)
>
I would use the same strategy. Save a simple text message attachement
that gives the details about the file included in the outbound message.
>
>>>> [...] I suppose a corporation could set a policy prohibitting
>>>> people from deleting attachments :-). ok ... so maybe you do
> >>> need to allow it to be disabled. w/o that argument I would have
> >>> said you shouldn't provide the option to disable it.
>>
>> Why? Do they also prohibit people from deleting messages? While
>> I can't say there isn't some Dilbert-style boss that has made that
> > policy, I don't see a reason for Mozilla to provide technological
> > enforcement. If a company really wants to keep information, it
> > should be trapping data at the servers during delivery not relying
> > on the recipients.
>
> I'd only consider adding this option if there was a strong demand for it
> or if there was a very compelling reason to add it.
>
> I expect there might be a reasonable need to allow the users to confirm
> and/or undo the deletion of attachments. Part of the decision on the
> undo option may depend on how difficult or awkward it turns out to be to
> add that functionality. It's also possible that an undo operation would
> absolutely have to be provided.
>
> If there's an undo, then maybe it wouldn't be necessary to have a
> confirmation window (one less annoying thing to deal with). Certainly
> at the bare minimum, either a confirmation window or an undo operation
> would be needed.
A confirmation warning than can be disabled would be better. Once the
attachment has been separated from the message there is no way to
guarantee that you can undo the change. Don't give people the false
hope of an undo. If the option is called "Delete Attachment" and
there is a confirmation warning by default then people will know what
to expect. If you let them disable the confirmation dialog they should
also know what to do.
Given the raw text format of a MIME mail message this feature is fairly
easy in a storage sense. I don't know what impediments the internal
Netscape data structures will present or where to add the UI hooks but
I envision a change something like:
--------------050606040201070208030509
Content-Type: image/png;
name="names.png"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
filename="names.png"
iVBORw0KGgoAAAANSUhEUgAAANAAAACTCAIAAABwCgDhAAAABGdBTUEAALGOfPtRkwAAACBj
SFJNAAB6JQAAgIMAAPn/AACA6AAAdTAAAOpgAAA6lwAAF2+XqZnUAAANYElEQVR4nGJgGAWj
NJrgRgFdAUCAAQAzf2wO1s6SyQAAAABJRU5ErkJggg==
--------------050606040201070208030509--
becoming:
--------------050606040201070208030509
Content-Type: text/plain;
name="deleted.txt"
Content-Transfer-Encoding: 7bit
The original attachment was removed from this message to conserve space
on Fri, 16 Aug 2002 21:08:45 -0400 by Mozilla 1.1b ....buildID.
The original content MIME type info was:
Content-Type: image/png;
name="names.png"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
filename="names.png"
--------------050606040201070208030509--
Obviously, the actual message needs a little more work and it would
be nice if it could be edited by the user who could then put in more
specific or relevent information than we can get from the MIME headers.
And my example certainly didn't save space. :-)
Popping up a dialog with the text (editable) that's about to be
inserted and a warning message would make the confirmation useful.
Matt Coughlin
> easy in a storage sense. I don't know what impediments the internal
I've already identified how and where to add the source code to get the
additional UI for the attachment list context menu and the main menu.
That turned out to be pretty easy since it was just a matter of modeling
it after the existing code for saving attachments.
And yeah, on paper when you look at how easy it is to understand the
internal layout of an existing e-mail, and when you see that it's just a
plain text file, you'd think it'd be pretty simple to make the program
add in another header, and delete or replace an attachment. At least,
it seems like a small program or script could do it pretty easily.
But so far, with no familiarity with the code base, it's been difficult
sifting through the code looking for any sign of where the real
functionality is for reading and writing the headers, the body, and the
attachments, and understanding how the new code would have to be
consistent with the existing layers of abstraction and all the
exceptions to the usual rules that have been added in over the years for
things like the somewhat similar actions of moving and copying e-mails.
The knowledge barrier for making a change like this appears to be quite
high. That's why I've stopped trying to make a quick change, and why
I'm looking at a time frame of at least half a year of gradually
learning a little bit at a time. It'll also be essential to proactively
get help from people familiar with the relevant portions of the code base.
Of course, if someone really experienced jumps in and makes a patch for
this in just a few days, that would be a much better scenario (though
after 3 1/2 years of this bug sitting around relatively untouched, it'd
be unlikely. Ah, but one can dream :-)
> I envision a change something like:
> --------------050606040201070208030509
> Content-Type: image/png;
> name="names.png"
> [...]
I think there'd be a lot of end-user considerations with this approach
that would need to be looked at pretty thoroughly, if it's to be pursued
seriously. In my first attempts of manually editing an e-mail to make a
change like this, I got the feeling that the UI would wind up being too
confusing. I'd recommend trying it out yourself manually and seeing
what you think.
The most I can remember right now is that it seemed a little weird that
after deleting the attachment there was still an attachment there that
you could possibly delete or save. It might significantly complicate
the UI, and perhaps harm the intuitiveness of it, to find a way to
handle this. Maybe a clean solution is possible, but I'm having trouble
picturing it right now.
Are you aware of any [somewhat] widely-used e-mail clients that use an
approach like this for deleting or externally storing attachments? I'd
also be interested in hearing other people's thoughts about this.
Karthik Sheka
Quick question - Is it possible to send a message that appears to have a
message deleted on the *receiving* person's hard drive? (Basically, is
it possible to fake a deletion?) Could this be concidered a security
violation?
Matt Coughlin
> message deleted on the *receiving* person's hard drive? (Basically, is
> it possible to fake a deletion?) Could this be concidered a security
> violation?
Need a little more info on this...
Which (if any) of the following cases do you mean?
1. Resending an existing message that's had a legitimate attachment deleted.
2. Resending an existing message that's been doctored to look like an
attachment was deleted (an attachment that was never present in the
message).
3. Sending a new message that's been doctored to look like an attachment
was deleted (an attachment that was never present in the message)?
Someone could doctor an existing message by hand, by manually editting
the mailbox file. They could doctor it to look like anything they
wanted it to look like. It wouldn't be any fault on Mozilla's part
though if they do so.
Someone could also doctor the body and attachments of a new message by
hand, before sending it, although AFAICT they wouldn't be able to doctor
the headers to be consistent with this (if something like an
X-Mozilla-Altered header winds up being used to indicate internally that
an attachment has been deleted); so in this case the user would not be
able to completely doctor the message, only partly. I tried it myself,
and I wasn't able to force the extra header info into the sent message.
Generally speaking, I'm not sure what does and does not constitute a
security violation when sending and storing e-mail. I've been relying
on the knowledge of others to help avoid situations that would cause
security violations.
I might not be able to answer that part of your question myself. But
AFAIK if someone doctors a message by hand, it's not Mozilla's fault if
that causes a security violation. It would only be Mozilla's fault, and
hence something that may need to be avoided in the code base, if a
security violation could occur without the message having been doctored
by hand.
Karthik Sheka
I guess I was implying number 3, sending a new message that's been
doctored to look like an attachment was deleted (an attachment that was
never present in the message). If I had a message in my mailbox that
looked like an attachment was deleted from my local system, I may be
afraid that my local system was compromised (depending on how paranoid I
was that day :-) I guess I'll take your word for it that the message ID
and the rest of the header could not be doctored properly. However, if
a non-mozilla mail client sent the message, it's possible they could
send out the message with the proper information in the
X-Mozilla-Altered field.
Matt Coughlin
> I guess I'll take your word for it that the message ID
> and the rest of the header could not be doctored properly. However, if
> a non-mozilla mail client sent the message, it's possible they could
> send out the message with the proper information in the
> X-Mozilla-Altered field.
The test I did wasn't totally thorough. What I did was basically this:
Create a new message. Save the message. Close Mozilla. Edit the
message in the Drafts mailbox by hand. Delete the Drafts.msf file. Run
Mozilla. Reopen the message. Send the message to yourself. Receive
the message.
When I edited the message by hand, I saw that there wasn't already a
"Message-ID" header. Apparently this doesn't get created until the
e-mail is actually sent; so I don't see an easy way to doctor this.
There was a "References" header, for establishing threading; I think
it'd be possible to send an altered "References" header.
I manually added an "X-Mozilla-Altered" header (note: this is a header
that does not currently exist in Mozilla. It's being considered as a
possible header to add when an attachment is deleted). When I later
sent the message and received it, I noticed that the "X-Mozilla-Altered"
header was no longer in the message, which wasn't much of a surprise.
Mozilla did not include it in the sent message, and rightly so.
I'm not sure if there's an option to add custom headers to a new
message, in Mozilla or in other e-mail clients. If there is, then
someone might be able to convincingly doctor a new message to make it
look like an attachment was deleted.
Karthik Sheka
[Stuff deleted]
> I'm not sure if there's an option to add custom headers to a new
> message, in Mozilla or in other e-mail clients. If there is, then
> someone might be able to convincingly doctor a new message to make it
> look like an attachment was deleted.
>
Well, I guess the question is if there's any sort of security violation
when a message is sent that appears to have had an attachment that was
deleted at the destination computer. (And, I guess, if anyone cares. :-)
Anyone that this can be kicked to? Should I refer this to the
netscape.public.mozilla.security newsgroup?
Bigfeot
[Stuff deleted]
> message, in Mozilla or in other e-mail clients. If there is, then
> someone might be able to convincingly doctor a new message to make it
> look like an attachment was deleted.
>
Well, I guess the question is if there's any sort of security violation
Matt Coughlin
> Well, I guess the question is if there's any sort of security violation
> when a message is sent that appears to have had an attachment that was
> deleted at the destination computer. (And, I guess, if anyone cares. :-)
> Anyone that this can be kicked to? Should I refer this to the
> netscape.public.mozilla.security newsgroup?
>
where to go next. I'm newer to Mozilla than just about anyone else here
:-) All I can do (as I've done so far) is describe how Mozilla would
behave if the considered patch for bug 2920 is applied.
Offhand it sounds like a good idea to post the question to
n.p.m.security; maybe that's the best next option. I'm not sure how
specific their focus is there, or if they handle pretty much any issue
related to security. If you happen to know who some of the more
security-knowledgable people are who deal with mailnews or who deal with
security issues in general, you could also try e-mailing them to find
out how and where to pursue this further.
If you find out that the currently-considered patch would pose an
unacceptable security risk along these lines, please let me know (or let
someone else related to mailnews know) along with whatever details you
can provide (i.e. which person said what about the issue).
If you'd like to suggest any ideas for how the deletion of attachments
should be handled, which is still very much up in the air at this point,
please feel free to post further messages to this thread as needed.
Karthik Sheka
netscape.public.mozilla.security (I hope :-)
There's a discussion going on in the mail-news group about the recipient
of a message deleting attachments in the message and storing the message
(without attachments) in the mbox.
The question that it brings up is:
Is there any security issue if a malicious person were to send a
malformed email that appears to have a deleted attachment (an attachment
that was actually never present) that appears to have been deleted on
the target computer.
Curtis Jewell
> approach like this for deleting or externally storing attachments? I'd
> also be interested in hearing other people's thoughts about this.
Yes. Pine. When it saves mail that has been sent, it can strip attachments
in the saved copy of the e-mail.
Here's an example...
Date: Mon, 5 Aug 2002 11:08:35 -0500 (Central Daylight Time)
From: Curtis Jewell <********@******.***>
Reply-To: ********@******.***
To: ********@******.***
Subject: Here's the playlists I have.
Message-ID: <Pine.WNT.4.43.0207221814460.208-500000@curtisjewell>
X-Warning: UNAuthenticated Sender
MIME-Version: 1.0
Content-Type: MULTIPART/MIXED; BOUNDARY="116267-5710-1027379797=:208"
Content-ID: <Pine.WNT.4.43.0207222152000.208@curtisjewell>
--116267-5710-1027379797=:208
Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
Content-ID: <Pine.WNT.4.43.0207222152001.208@curtisjewell>
These playlists have a lot of the music I like, and most of it isn't
played yet. (they come from my CD's of MP3's - David can borrow them
anytime he wants in order to grab them, or you can in order to check
them out.)
music.txt is a quickie list of all my music.
I just kept forgetting to get this to you...
--Curtis
--
Curtis Jewell
http://www.curtisjewell.com/ ********@******.***
http://curtis.livejournal.com/ ********@******.***
http://new-york.utica.mission.net/ ********@******.***
--116267-5710-1027379797=:208
X-Content-Type: TEXT/HTML; NAME="Playlist 2.htm"
X-Content-Transfer-Encoding: BASE64
Content-ID: <Pine.WNT.4.43.0207221816370.208@curtisjewell>
Content-Description:
X-Content-Disposition: ATTACHMENT; FILENAME="Playlist 2.htm"
Content-Type: TEXT/PLAIN
The following attachment was sent,
but NOT saved in the Fcc copy:
A Text/HTML segment of about 3,718 bytes.
--116267-5710-1027379797=:208
X-Content-Type: TEXT/HTML; NAME="Playlist 1.htm"
X-Content-Transfer-Encoding: BASE64
Content-ID: <Pine.WNT.4.43.0207221816371.208@curtisjewell>
Content-Description:
X-Content-Disposition: ATTACHMENT; FILENAME="Playlist 1.htm"
Content-Type: TEXT/PLAIN
The following attachment was sent,
but NOT saved in the Fcc copy:
A Text/HTML segment of about 4,264 bytes.
--116267-5710-1027379797=:208
X-Content-Type: TEXT/HTML; NAME="Oldies Playlist.htm"
X-Content-Transfer-Encoding: BASE64
Content-ID: <Pine.WNT.4.43.0207221816372.208@curtisjewell>
Content-Description:
X-Content-Disposition: ATTACHMENT; FILENAME="Oldies Playlist.htm"
Content-Type: TEXT/PLAIN
The following attachment was sent,
but NOT saved in the Fcc copy:
A Text/HTML segment of about 7,414 bytes.
--116267-5710-1027379797=:208
X-Content-Type: TEXT/plain; name="music.txt"
X-Content-Transfer-Encoding: BASE64
Content-ID: <Pine.WNT.4.43.0208051107500.3176@curtisjewell>
Content-Description:
X-Content-Disposition: attachment; filename="music.txt"
Content-Type: TEXT/PLAIN
The following attachment was sent,
but NOT saved in the Fcc copy:
A Text/plain segment of about 96,666 bytes.
--116267-5710-1027379797=:208--
Matt Coughlin
The text below contains portions of an e-mail from timeless, which was a
response to the previous post in this thread, as well as a few other
posts made later in the thread. I'm posting this with the permission of
timeless. This is an awkward way to continue the discussion, but for
the moment it can't be helped. There are additional e-mails from
timeless that I'll try to post to the thread when I get the chance.
It would be helpful if any responses made are posted to the newsgroup
thread, preferably without including the text I added at the beginning
and end of the message. I added some hyperlinks (?) for the thread at
the end of the message.
The indented "> " portions of the e-mail are my own comments from the
previous post. The non-indented portions are by timeless, except for
the occasional [some text here -Matt] that I add to clarify some of
timeless's comments.
===============================================================
> AFAICT, the immediate need for deleting attachments is much greater
> than the need for consistency between the two context menus (even
> if it means a little more work later on for when this inconsistency
> is resolved) (anyways, I might consider doing the work myself, since
> I've already been dealing with some of the relevant areas of code,
> and since the new "delete" code might be mine).
not being consistent for even a day is a disaster. someone will decide
that your 'delete all' is an essential feature and when you fix the
inconsistency you'll get five bugs complaining about it being missing.
> "Delete", and using the letter "D", matches the attachments-list
> context menu for when you're creating an e-mail. Either both of
> them should be "Delete" or both of them should be "Remove".
sure if we make this change it would be for all places where one can
remove an attachment from a message. on this point we should probably
talk to jgl...@netscape.com and rob...@netscape.com directly.
> Since "Delete" has already been established in the other
> attachments-list context menu, and since that generally seems
> to be the standard term used by applications,
you haven't provided a survey of applications.
when making sweeping generalizations, a survey is appreciated.
(be sure to include the applications that don't provide the feature, try
to get the top 3-5 apps for most OS's, if you can't get any for a given
OS please indicate that you felt the OS was worth getting info from but
did not have access to it.)
for this specific feature, I'd like to know about OS/2, QNX, BeOS,
Solaris (openwin?), Mac OS 9, Mac OS X, windows, x11, gnome, KDE, and
console.
-- yes x11, gnome and KDE aren't OS's, nor is console, whereas Solaris
is. Wow that's evil of me, well why did i split it like that.
Solaris has a mailer (dtmail?) which people from sun working on mozilla
actively care about (i think it's from openwin, but i can't remember)
most x11 apps are really defined by their toolkit. Each toolkit has a
bias for feature-set and behaviors. Also if i only asked for the top
three mailers for Unix I'd surely miss a few of those categories.
why OS/2, QNX, BeOS? they all have mailer apps and they have more object
oriented views than most of the other OS's.
what's console? well pine, mutt, mail are the ones that come to mind.
what about gnus or however the emacs one is spelled? yeah you might as
well mention that, but don't feel obligated to categorize it (esp since
it can probably be a console or GUI app)
> I'd lean heavily towards using that (unless there was an overwhelming
> majority of people that thought it should be different).
let's get info from the survey (note that if you can't actually survey
enough yourself you're free to ask people on IRC, by email or in the
newsgroup to complete the survey, it is after all a survey)
[Note: Any help from others in identifying the relevant UI details for
other e-mail clients, ones that allow some form of deletion or external
storage of attachments, would be much appreciated. Personally, I've
only used Eudora and Mozilla. -Matt]
> Is it possible to change the message ID without breaking threading?
it should be. but that depends on what you mean.
if there's an original message A (which had no attachments) and replies
B1..B3 they'll all have a field referencing A's message-id. if you
munge B1 to b1 and make it include a reference to A and B1 then a
followup (C1) to B1 with a valid date should appear after b1 (by
reconstructed thread) although it might appear as a sibling instead of
as a child. This is the first approach that comes to mind, however
given that you'll be munging the envelope you could use some other
approach or even create a mozilla field (there are actually some extra
fields that act as message ids, you should research them and see if any
satisfy your requirements)
remember, you're changing Mozilla's code when you do this, you're also
asserting that the message is not supposed to be resent, so there's very
little harm in having another message-id field (perhaps
original-message-ID:) which has the original id, and teaching mailnews
threading to use it when it calculates threads.
[...]
for reference,
GNKSA is another kind of law which mail is expected to obey, and mozilla
isn't really supposed to enable users to break GNKSA.
> [...] attachment deletion
> and/or externally autosaving attachments is a feature in some
> widely-used e-mail clients (mutt, Eudora, and Outlook; and Pegasus
> offers some form of this). So what do these e-mail clients do with
> their handling of removing attachments that apparently (?) is accepted
> as an adequate solution?
Outlook doesn't use MBOX. I'm not sure about the other three. [...]
> Eudora does not change the message id when they externally autosave
> the attachments. Eudora replaces the attachment section with text
> like the following:
> Attachment Converted: "c:\program files\eudora\attach\filename.ext"
>
> Is this a case of an mbox violation?
IMO it is
> If so, is there adequate documentation indicating that the violation
> occurred, such that the violation itself wouldn't pose any legal risk
> to individuals if they had to provide such an e-mail in court?
> (Although the seal has been broken,
> the documentation (the "Attachment Converted..." text) openly declares
> that the seal was broken, and it openly states what was removed from
> the e-mail.)
right, i go to court and present a signed letter indicating that I've
perjured myself. Courts love that. <me> Hi judge, here's the original
message that i received from <sender>. <judge> oh look there's a note
certifying that this isn't an original message. Here's your
additional fine sir.
What happens if someone sends me a letter with Attachment Converted:
"c:\windows\desktop\attach.ext"
If i try to go to a judge and claim that I never received the attachment
because look, this is what i got. The judge asks if my machine could
have ripped off the attachment and replaced it with that text? I gulp
and quietly say 'yes sir'. The judge throws the book at me.
[...]
what happens if someone wants to remove an attachment from a signed or
encrypted message? [Please take your time considering your answer to
this question. You may need to design a complicated set of dialogs and
other things.]
> What's your recommendation on this [the option to disable deletion
> of attachments]? A user pref that can only be
> changed if the preferences file is edited by hand?
well, corps are generally screwed if they try to set any policy, but
that's not for us to declare. if we do declare it, then they can set a
policy forbidding use of mozilla and its derivatives, we don't want that...
one thing they could do (if you had a pref) is use a <appname>.cfg file
to lock the pref. This means that if the pref is locked and set to
disallow, there should be absolutely no UI and no way for the user to
change it.
(please ignore any mechanisms a person could use to circumvent this
lock, that's really not our problem)
as for the visible pref bit, i still don't see much use in a visible
pref. more interesting questions do arise.
Suppose a person had two mailnews accounts: 1. "Personal", 2.
"Business". Suppose that the business had a policy prohibiting the
deletion of attachments. The user or employer might want to configure
the second mailbox not to support deleting attachments. Perhaps the user
does want to be able to delete attachments from the other mailbox. OK,
so one could argue for a visible (per account) pref. Well, supposing
one did that, what happens if the user drags a message from 2 to 1,
deletes an attachment and drags it back from 1 to 2?
The simplest solution for everyone involved is to force the user to have
two distinct profiles. IMO it's even the correct solution.
(I say business, but in fact government is perhaps an even likelier
candidate, there are all sorts of record keeping laws for USG employees,
it's fun)
---
Message-ID: <3D5D8E87...@sp4m-less.forestpath.org>
[...]
re: Not storing attachments in sent, same issue but in reverse. Suppose
i send a reply with the contract signed but my sent folder doesn't keep
the attachment, instead it references a local version of the file.
perhaps about the time i send the message i save an annotation to the
contract. Judge, i think this is the file i sent and the message.
<landlord> this is what i received, it's all right here intact.
I think that I somewhere mentioned a nifty mime-type which probably
works nicely for this stuff, i know i ran across it.
---
Message-ID: <3D5DC3D3...@sp4m-less.forestpath.org>
heh, the ability to delete a deleted attachment. yeah that's rich. then
you can shift when an attachment was deleted.
if we use the mime-type i was thinking about, we'd probably want to
special case deleting a deleted attachment so that you just can't do it.
There's really very little harm in that.
Imagine that you had an attachment text/plain (and that you were on
windows), you delete it, and the attachment window changes the icon from
the notepad icon to the notepad icon with the cute red x overlaid. From
the end user's perspective, it isn't unreasonable that they can't delete
it a second time. (Remove/Delete, whatever both get confusing when you
try and operate again)
---
Message-ID: <3D5FD91A...@forestpath.org>
yeah that poster was right to worry about forging attachment removed, i
mentioned that in the body of this message. the concern is mainly 3
[Sending a new message that's been doctored to look like an attachment
was deleted (an attachment that was never present in the message) -Matt].
SMTP is an open protocol, so i can pretty much send you any content i
like, it's very easy for me to include random stuff in the body
especially an attachment that says it had a deleted attachment.
mozilla as an SMTP client is considerably better behaved than me+telnet
as an SMTP client -- we're just an evil pair.
as for message headers, mailers can/do defensively munge them, i think
it was mutt which had nice documentation indicating that it would drop
headers that looked like its own which it didn't create when it received
a message.
the problem is that in IMAP other than the _reallyNew_ flag [i don't
speak IMAP so I'm not sure under what circumstances that's set, and I'm
trying to get this message sent to you instead of reading the IMAP
RFCs], it's a bit hard to know that you a message is new and that now
is the time to sanitize it. supposing you took that approach, would
dropping a message from localmail into the IMAP box mark it as really new?
(i can't remember if it would). that'd be a bad loop :)
[end of comments from timeless -Matt]
====================================================================
As stated earlier, the indented "> " portions of the above text are my
own comments from the previous post. The non-indented portions are by
timeless.
Here are some hyperlinks for the newsgroup thread in question. Not sure
if I know the right way to add hyperlinks in a message, so I'll try a
couple different ways. I'm posting this message with Mozilla set to
wrap lines at 72 characters (don't know if this will break long
hyperlinks, or if long hyperlinks get broken no matter what).
n.p.m.mail-news
[Bug 2920] Discussion about deleting attachments
---------------
news://news.mozilla.org/netscape.public.mozilla.mail-news
<a
href='news://news.mozilla.org/netscape.public.mozilla.mail-news'>netscape.public.mozilla.mail-news</a>
Google Groups, the specific thread
----------------------------------
http://groups.google.com/groups?dq=&hl=en&lr=&ie=UTF-8&threadm=ajqpn3%24ctb1%40ripley.netscape.com&prev=/groups%3Fhl%3Den%26lr%3D%26ie%3DUTF-8%26group%3Dnetscape.public.mozilla.mail-news
<a
href='http://groups.google.com/groups?dq=&hl=en&lr=&ie=UTF-8&threadm=ajqpn3%24ctb1%40ripley.netscape.com&prev=/groups%3Fhl%3Den%26lr%3D%26ie%3DUTF-8%26group%3Dnetscape.public.mozilla.mail-news
'>Google Groups: thread "[Bug 2920] Discussion about deleting
attachments"</a>
Matt Coughlin
Are the legal risks you've been describing (from presenting an altered
e-mail in court) specific to RFC822, or do they apply to e-mail stored
in any format? Does Mozilla's intended compliance with RFC822 put users
at a greater legal risk, in the event of having to present altered
e-mail, than if their e-mail was stored in a different or less formal
format?
Would a deliberate invalidation of the RFC822 format by Mozilla for an
altered e-mail, such that the e-mail is no longer in fact being stored
in RFC822 format, free up the user from any such legal risk?
What would constitute the basis for the sort of hypothetical perjury
you've described in the past? Does Mozilla state somewhere that all
mail is stored in RFC822 format, such that whether or not that's the
case in practice, the court would expect any mail from a Mozilla mailbox
to be in RFC822 format, with the user being legally at fault if it's
not?
Is there something in an individual e-mail that legally asserts,
independently of the contents of the other e-mails and independently of
what Mozilla does or does not state about mail storage in general, that
that that particular e-mail is stored in RFC822 format?
Are there laws that state, for a particular mail storage format or for
any form of mail storage, that an e-mail cannot legally be altered once
it has been received. Are there legal precedents that assert that under
no circumstances can a person present an altered e-mail in court without
being guilty of perjury?
Is this an appropriate parallel: A person, after being sworn in at
court, says "The next thing I am about to say is not a truthful
statement", and then says an untruthful statement. Would that still be
an example of perjury? Is that the sort of a situation you're trying to
describe in the context of altered e-mails, in which it's not possible
to avoid perjury if the e-mail has been altered?
If a user presents an e-mail in court, an e-mail that clearly states
that it has been altered, and if the user states to the court before
handing it over that the e-mail has been altered, would they still be
guilty of perjury? Is it perjury when any false information is
presented, regardless of whether the person identifies the information
as being false or not, or is it only perjury when a person supplies
false information that they presented as being supposedly truthful?
Aside from any legal risks, how imperative is it to avoid any violation
of RFC822 in Mozilla? Is it a matter of striving for general compliance
with standards? Has Mozilla openly stated that its mission is full
compliance with standards like RFC822? Are there already some
violations of RFC822 in Mozilla? (is Mozilla truly 100% RFC822 compliant?)
Is the core Mozilla team legally required to comply with RFC822?
Matt Coughlin
posting this with the permission of timeless. This is (still) an
awkward way to continue the discussion, but for the moment it (still)
can't be helped.
The indented "> " portions of the e-mail are my own comments from the
previous post. The non-indented portions are by timeless.
===============================================================
IANAL. I just like thinking about problems that way. so don't take my
advice as legal advice.
> Are the legal risks you've been describing (from presenting an altered
> e-mail in court) specific to RFC822, or do they apply to e-mail stored
> in any format?
hrm, well for one it would probably depend on whether a court even
accepted digital data as evidence (these days they probably do).
assuming they do then it would probably only apply to a format which had
a standard. So if i just scribbled a copy of a contract onto a napkin
and didn't have the other party sign it then they'd ignore my napkin. If
my napkin had a signature from the other party then my napkin was a
contract which could be introduced into court (wills and bills of sale
have been done on napkins). The napkin can be equated to me taking notes
in notepad (which is a text editor and has no way to store most digital
signatures). So if we had a mailbox format which didn't say that its
contents are stored copies of what they received and for example the
mailbox format was actually known to shred digital signatures (i think
some webmailers do that) then in all likelihood it wouldn't be
admissible. If another format OTOH preserved the message body and said
it would, then it would probably be as admissible as the RFC822 message
from the MBOX. email has been used in cases (including USG v. ms) so i
guess we have to accept that it can be admissible.
> Does Mozilla's intended compliance with RFC822 put users
> at a greater legal risk, in the event of having to present altered
> e-mail, than if their e-mail was stored in a different or less formal
> format?
heh. IANAL (reiterated because this is an even bigger question which
really should be put to a lawyer, or ignored -- probably ignored). I'd
simply say 'no'. Supposing we did decide to munge an RFC822 message, it
would only put you at risk if you foolishly (* users are fools, and it's
impossible to make software foolproof. someone will just invent a better
idiot) presented the email as an original. Well, that's the basic case.
I'm not quite sure what would happen if someone subpoenaed your copy.
Supposing that you had since migrated to another mailer and the mailer
didn't show any sign that mozilla had tampered with the message, then
you might have problems -- this can mostly be fixed by making sure that
sane mailers show evidence that the message is modified, it might also
be an argument against simply using header flags.
> Would a deliberate invalidation of the RFC822 format by Mozilla for an
> altered e-mail, such that the e-mail is no longer in fact being stored
> in RFC822 format, free up the user from any such legal risk?
dialog
victim comes to lawyer and asks what to do
lawyer: do you have a copy of the email?
victim: yes, it's in my mailbox
lawyer: what program do you use?
victim: Netscape7.5
lawyer: oh, i think that's MBOX based, it should have the full message,
we can use that in court.
victim + lawyer go to court and make a big mistake.
you could try very hard to publicize that you're abandoning MBOX but
you'd get a whole lot of criticism and the people you're targeting
wouldn't hear you anyway. the result is a lot of negative publicity and
no protection for your intended audience.
at this point, mozilla doesn't need to be seen as breaking standards.
> What would constitute the basis for the sort of hypothetical perjury
> you've described in the past?
i take my email from the mailbox and swear to the judge that it's an
original.
> Does Mozilla state somewhere that all mail is stored in RFC822 format,
> such that whether or not that's the case in practice,
it might, or it might just be a well known fact
> the court would expect any mail from a Mozilla mailbox to be in RFC822
> format,
the court/attorneys would make that expectation
> with the user being legally at fault if it's not?
with the user being legally at fault for presenting it as an original if
it was modified. if the user didn't present it then it wouldn't be an
issue (unless something interesting happens as the email is subpoenaed -- I
suppose a problem could arise if the user forwarded the modified email
to someone).
> Is there something in an individual e-mail that legally asserts,
> independently of the contents of the other e-mails and independently
> of what Mozilla does or does not state about mail storage in general,
> that that particular e-mail is stored in RFC822 format?
no. emails can actually be in other formats. i think exchange,
firstclass, bbs's, and various older systems had other formats.
perhaps ideally the laws for email should be limited to signed and
encrypted messages, but if they did that then people couldn't be held
accountable for unsigned emails...
> Are there legal precedents that assert that under no circumstances can
> a person present an altered e-mail in court without being guilty of
> perjury?
presenting a contract which you have modified as an original will
certainly get you convicted of forgery/perjury (or perhaps something
else). so there's definitely a precedent from the real world.
if you present something and say "this is what we found on *his*
computer, it's not an original which looks like _this_", you're
certainly not going to be guilty of perjury (you're probably the other
attorney [prosecutor / plaintiff] complaining about the defendant)
> Is this an appropriate parallel: A person, after being sworn in at
> court, says "The next thing I am about to say is not a truthful
> statement", and then says an untruthful statement. Would that still
> be an example of perjury?
I'd hope not :-). OTOH, in court you'd rarely be given a chance to say
that. when you're sworn in, it's a question and answer session. you're
only supposed to give answers. so it should be rare that you are setup
(by the questioner) to be able to say something like that.
> If a user presents an e-mail in court, an e-mail that clearly states
> that it has been altered, and if the user states to the court before
> handing it over that the e-mail has been altered, would they still be
> guilty of perjury?
hopefully not
> Is it perjury when any false information is
> presented, regardless of whether the person identifies the information
> as being false or not, or is it only perjury when a person supplies
> false information that they presented as being supposedly truthful?
hopefully the latter.
One entry found for perjury.
Main Entry: per·ju·ry
Pronunciation: 'p&r-j&-rE, 'p&rj-rE
Function: noun
Date: 14th century
: the voluntary violation of an oath or vow either by swearing to what
is untrue or by omission to do what has been promised under oath
: false swearing
> Aside from any legal risks, how imperative is it to avoid any
> violation of RFC822 in Mozilla?
the legal risks are probably secondary to getting in trouble with the
developer community for violating the standard. Companies or groups
which broke MBOX by introducing the length field didn't fair well (well,
they could have done worse by being remembered forever for their poor
behavior, instead they're just forgotten).
> Is it a matter of striving for general compliance with standards?
yes, or perhaps for not breaking compliance with standards. it's OK for
us to work gradually to compliance with a *new* standard. but it's not
OK for us to work away from compliance with an old standard with which
we used to be compliant. I'm opposed to even temporary deviations, but
the fact is that if we chose a deviation, it would be because we found
no way to do it right (which i believe we can do), so it wouldn't ever
be a temporary deviation.
> Has Mozilla openly stated that its mission is full
> compliance with standards like RFC822?
http://www.mozilla.org/mozorg.html
Mozilla is an open-source web browser, designed for standards
compliance, performance and portability.
> Are there already some violations of RFC822 in Mozilla?
I'm presuming there aren't, or that if there are they're bugs which
people are working to fix
> (is Mozilla truly 100% RFC822 compliant?)
i don't know
> Is the core Mozilla team legally required to comply with RFC822?
no
[...]
ah. i was hoping to find a nice place to work back in digitally signed
emails, but i didn't really find one, so here goes (quickly):
there are a few ways to sign emails, the preferred way is mime encoded.
i don't know much about them, but I'd expect that they sign the entire
message (body and all attachments up to but excluding themselves).
assuming that a message works that way, then deleting an attachment from
such a message would definitely break the signature, and might even
render the message unencrypted/undecryptable. this is certainly food for
thought. how do you handle that?
certainly the messages a court would prefer to deal in are the signed
versions and if one side presents a signed version and the other side
doesn't have one then the other side would be wise not to mention their
copy (no problem, unless they can't read the copy they have). OTOH if
you make the mistake of presenting your copy (which has lost its
signature validity) and the judge asks about it, and the other side
says, hey we have a signed version of that.. you're in trouble.
certainly you/your attorney should be wise enough to notice the broken
signature and not present it but.. that's based on the assumption that
a signature wraps an entire message. but there are two other ways i can
imagine a signature working. one is wrapping each part, and the other is
signing each part and wrapping the whole. in the wrapping each part
case you might easily make the mistake of not seeing (esp if you aren't
using mozilla) the missing attachment. and actually that'd probably be
the case for the signing each part and wrapping the whole case in
another app. probably the best approach for signing is to make sure that
there's a replacement part. then at least most UIs would show it as
broken signature or missing signature.
[end of comments from timeless -Matt]
====================================================================
As stated earlier, the indented "> " portions of the above text are my
own comments from the previous post. The non-indented portions are by
timeless.
Matt Coughlin
> Supposing that you had since migrated to another mailer and the
> mailer didn't show any sign that mozilla had tampered with the
> message, then you might have problems -- this can mostly be fixed
> by making sure that sane mailers show evidence that the message
> is modified, it might also be an argument against simply using
> header flags.
I agree. I wouldn't want the visible evidence of alteration to be
entirely dependent on special functionality in Mozilla, such that if a
user migrates to a different e-mail client they likely wouldn't be able
to tell which e-mails were altered. Usage of header flags alone clearly
wouldn't be adequate.
Personally, if header flags do get added, I'd want it to be only for
posterity, as an internal, non-visible record of what happened, and not
something that Mozilla makes use of afterwards (since other e-mail
clients likely wouldn't make use of it either).
Do you see any benefit from or need for adding header flags? It might
be a bit redundant or excessive, if there's always a reasonable amount
of visible info added describing the alteration. I might lean towards
adding a header flag only if it was both very useful and safe.
Along a similar line, I'm hesitant to consider replacing the data of an
attachment with text info (as opposed to removing the attachment
altogether), because it would either be awkward (if the user can delete
or save the text-info replacement) or there'd be special functionality
(for distinguishing between original attachments and text-info
replacements) that other e-mail clients likely wouldn't support.
Plus, for any e-mail clients that don't display text attachments inline
(if such clients exist), the user wouldn't have any visible indication
of the e-mail having been altered.
At the moment, I'd favor removing the attachments altogether, appending
text info to the end of each alternate version of the message body, and,
if needed, adding a header flag that's added and forgotten.
What arguments would you have against this approach or in favor of a
somewhat different approach? (aside from the general issue of mbox
violations, which might apply equally to any of the discussed approaches)
> I suppose a problem could arise if the user forwarded the modified
> email to someone).
Are there additional concerns for forwarding or replying to an altered
e-mail, concerns that don't apply to storing an altered e-mail?
> it's OK for us to work gradually to compliance with a *new*
> standard. but it's not OK for us to work away from compliance
> with an old standard with which we used to be compliant.
The approach that I presented above might be a deviation from standards
technically speaking, but the deviation might not have any real-world
impact (as long as there's an obvious visible indication of the e-mail
having been altered).
In a practical, debatable sense, the resulting altered e-mail would be
in compliance with the mbox format, since it could still be read by any
reasonable mbox-supported e-mail client; as such, it wouldn't affect
mailbox portability. The deviation would be the fact that it was
altered at all, not (at least in this case) the way in which it was
altered. The distinction between the two might not be very significant
in a theoretical sense, but in real-world situations I'm hoping it'd be
significant enough to make the above approach, or a comparable one,
acceptable.
> I'm opposed to even temporary deviations, but the fact is
> that if we chose a deviation, it would be because we found
> no way to do it right (which i believe we can do), so it
> wouldn't ever be a temporary deviation.
What do you see being necessary to allow for deletion of attachments
without deviating from compliance with formal standards?
I'd be opposed to non-deviation (other than doing nothing), if it would
take a year or more before it could be implemented (say perhaps if a
formal superset of or an alternative to RFC822 had to be created), and
if deviation was possible in a way that didn't have a real-world impact.
To put it another way, I might not be willing to commit to a longer
time frame than half a year for the patch.
Do you see a potential, non-deviation solution that could be resolved
within the space of half a year? What steps would need to be taken to
accomplish such a solution?
I'm starting to wonder how practical it would be to append text info to
the end of each alternate version of the message body. I expect it'd be
trivial for plain text at least.
It might be a little tricky for html - at the very least it would
require a more intelligent insertion algorithm. I expect it wouldn't be
practical to use code from the html editor to read and parse the
structure of the html, insert text in the appropriate place in the data
structures, and save the html back to the e-mail; since there'd likely
be formatting differences in the resulting html, and possibly even loss
of data (if there was some very non-standard content), versus how it was
originally.
The simplest method that comes to mind for html is to search for the
"</body>" tag and insert the text info (formatted as necessary for html)
just before the tag. Hopefully that would be reasonably reliable.
But I'd wonder about any other present or future alternate versions
besides plain text and html, perhaps xml or formats that haven't even
been thought up yet. Any other types of alternate versions would
presumably need to have the text info appended to them also. What if
the formatting of some versions was very non-trivial, perhaps encrypted?
I'm wondering if this approach would open up a big, ugly can of worms,
one that might not be obvious at first.
The only safeguard I can think of for this offhand, is to disable
deletion of attachments for e-mail that has alternate versions that the
deletion algorithm doesn't support at the time. That could be extended
further to disable the deletion functionality for any situation that the
overall deletion algorithm can't handle, though it could be confusing to
the user as to why the functionality is sometimes unavailable.
In some ways it'd be cleaner and simpler to keep the attachment and just
replace the attachment data with text info, but, as described [earlier],
I have some concerns about that approach too. Maybe just about any
approach would be inherently awkward (other than doing nothing of course).
One important thing to note with appending text info to the end of each
alternate version of the message body, is that if it was implemented and
it later became necessary to remove or replace that functionality, it
wouldn't pose a problem for any e-mails that had already been altered;
they could still be accessed just as well, with the visibile indication
of the alteration still being just as obvious. This approach doesn't
depend on maintaining any special functionality for the e-mail to be
adequately accessible after it's been altered; it's a "set it and forget
it" approach, as far as the e-mail client is concerned, and a "set it
and permanently remind them" approach, as far as the user goes,
regardless of which e-mail client or which version of Mozilla the user
uses in the future.
Any thoughts about this?
Garth Wallace
>
> Along a similar line, I'm hesitant to consider replacing the data of an
> attachment with text info (as opposed to removing the attachment
> altogether), because it would either be awkward (if the user can delete
> or save the text-info replacement) or there'd be special functionality
> (for distinguishing between original attachments and text-info
> replacements) that other e-mail clients likely wouldn't support.
>
> Plus, for any e-mail clients that don't display text attachments inline
> (if such clients exist), the user wouldn't have any visible indication
> of the e-mail having been altered.
>
> At the moment, I'd favor removing the attachments altogether, appending
> text info to the end of each alternate version of the message body, and,
> if needed, adding a header flag that's added and forgotten.
How about replacing the attachment with an attachment of type
message/external-body with an access-type of "LOCAL-FILE", as defined in
RFC 2046
<http://www.nacs.uci.edu/indiv/ehood/MIME/2046/rfc2046.html#5.2.3>? That
could contain all of the necessary information (a short disclaimer could
go in the attachment's "phantom body").
--
Mozilla 1.0 Guide: http://www.mozilla.org/start/1.0/guide/
Mozilla 1.0 FAQ: http://www.mozilla.org/start/1.0/faq/
End-user discussion and peer support:
snews://secnews.netscape.com:563/netscape.mozilla.user.general
snews://secnews.netscape.com:563/netscape.mozilla.user.win32
snews://secnews.netscape.com:563/netscape.mozilla.user.mac
snews://secnews.netscape.com:563/netscape.mozilla.user.unix
Matt Coughlin
The indented "> " portions of the e-mail are my own comments from the
previous post. The non-indented portions are by timeless.
===============================================================
> I'm starting to wonder how practical it would be to append text info
> to the end of each alternate version of the message body.
imo it isn't.
> I expect it'd be trivial for plain text at least.
Agreed
> It might be a little tricky for html
i'd go with impossible. there could be css, dhtml, comments, parsing
quirks, ...
> at the very least it would require a more intelligent insertion
> algorithm.
> I expect it wouldn't be
> practical to use code from the html editor to read and parse the
> structure of the html, insert text in the appropriate place in the
> data structures, and save the html back to the e-mail; since there'd
> likely be formatting differences in the resulting html, and possibly
> even loss of data (if there was some very non-standard content),
> versus how it was originally.
>
> The simplest method that comes to mind for html is to search for the
> "</body>" tag and insert the text info (formatted as necessary for
> html) just before the tag. Hopefully that would be reasonably
> reliable.
it isn't.
> But I'd wonder about any other present or future alternate versions
> besides plain text and html, perhaps xml or formats that haven't even
> been thought up yet. Any other types of alternate versions would
> presumably need to have the text info appended to them also. What if
> the formatting of some versions was very non-trivial, perhaps
> encrypted?
or signed :-)
> I'm wondering if this approach would open up a big, ugly can of
> worms, one that might not be obvious at first.
i think we can predict 20% of the worms, and they're all really nasty.
the rest will be worse :-(
> The only safeguard I can think of for this offhand, is to disable
> deletion of attachments for e-mail that has alternate versions that
> the deletion algorithm doesn't support at the time.
sometimes doing things halfway works, but more often than not you end up
regretting them because you get many complaints saying that what you did
was broken...
> That could be extended
> further to disable the deletion functionality for any situation that
> the overall deletion algorithm can't handle, though it could be
> confusing to the user as to why the functionality is sometimes
> unavailable.
yeah confusing and likely to get more bugs filed complaining that it
doesn't work.
> In some ways it'd be cleaner and simpler to keep the attachment and
> just replace the attachment data with text info, but, as described
> [earlier], I have some concerns about that approach too.
i know that i've been leaning towards this approach, i know there can be
complications but i think it's better than the alternatives.
> One important thing to note with appending text info to the end of
> each alternate version of the message body, is that if it was
> implemented and it later became necessary to remove or replace that
> functionality, it wouldn't pose a problem for any e-mails that had
> already been altered; they could still be accessed just as well, with
> the visibile indication of the alteration still being just as
> obvious. This approach doesn't depend on maintaining any special
> functionality for the e-mail to be adequately accessible after it's
> been altered; it's a "set it and forget it" approach, as far as the
> e-mail client is concerned, and a "set it and permanently remind them"
> approach, as far as the user goes, regardless of which e-mail client
> or which version of Mozilla the user uses in the future.
I think any approach we [hah, we=you :-)] take has to be able to work
pretty well even if the feature ceases to exist.
[end of comments from timeless -Matt]
====================================================================
As stated earlier, the indented "> " portions of the above text are my
own comments from the previous post. The non-indented portions are by
timeless.
Jorge Fuente-Alba
What i do is edit message as new, then delete the attachment, and then
save in drafts, and move back to the inbox.
Of course, composer re-writes the header, and all sender data is
therefore lost. But the email looks just like the original without the
attachment.
Could it be parsed by the email composer? <--- (i dont know what im
talking about here)
---8<--*snip* *snip*---8<---
> > I expect it wouldn't be
> > practical to use code from the html editor to read and parse the
> > structure of the html, insert text in the appropriate place in the
---8<--*snip* *snip*---8<---
Matt Coughlin
> What i do is edit message as new, then delete the attachment, and then
> save in drafts, and move back to the inbox.
> Of course, composer re-writes the header, and all sender data is
> therefore lost. But the email looks just like the original without the
> attachment.
Yep, that's the currently supported option for people who don't mind if
they lose the header info - including the message id, which causes a
loss of threading.
The point of bug 2920 is to have a way to accomplish this without losing
any of the header info. The only existing way to do it is to edit the
mailbox file by hand, and delete the corresponding .msf file, which
isn't convenient and which most users wouldn't willing to do; it also
carries the risk of corrupting the mailbox, if someone edits the file
improperly.
> Could it be parsed by the email composer? <--- (i dont know what im
> talking about here)
>
> ---8<--*snip* *snip*---8<---
>
>> > I expect it wouldn't be
>> > practical to use code from the html editor to read and parse the
>> > structure of the html, insert text in the appropriate place in the
>
> ---8<--*snip* *snip*---8<---
AFAIK, the same code used for editing and viewing web pages is also used
for editing and viewing html e-mail. So having it be parsed by the
e-mail composer would be pretty much the same as having it parsed by the
html editor (I think). At this point, neither one appears to be
practical or reliable for appending text info to existing messages.
But it might not matter much because I'm starting to lean away from
appending text info to the message body. That approach may be too
problematic, with the potential of it being totally unworkable (as far
as appending text info to non-plain-text versions of the message body -
html or otherwise).
The leading candidate right now is to replace the attachment with a
text-info attachment. This might be awkward, but it may be less
problematic to deal with. It still needs to be investigated thoroughly,
and everything for bug 2920 is generally up in the air.
Matt Coughlin
> message/external-body with an access-type of "LOCAL-FILE", as defined in
> RFC 2046
> <http://www.nacs.uci.edu/indiv/ehood/MIME/2046/rfc2046.html#5.2.3>? That
> could contain all of the necessary information (a short disclaimer could
> go in the attachment's "phantom body").
This looks promising, Timeless seems to favor using
message/external-body as well; however, the first stage of work, the
work specifically covered by bug 2920, doesn't cover saving and linking
to attachments (detaching files from a message), it only covers deleting
attachments.
But for future reference, to help coordinate some of the discussions
here with later work done on saving and linking to attachments, and for
clarification...
Are attachments part of the body of a message, or are they separate from
the body? Can message/external-body be applied to an attachment? Would
message/external-body only be useful for saving and linking to
attachments (detaching files), or would it also be useful for deleting
attachments?
Thanks for the input
Garth Wallace
>> How about replacing the attachment with an attachment of type
>> message/external-body with an access-type of "LOCAL-FILE", as defined
>> in RFC 2046
>> <http://www.nacs.uci.edu/indiv/ehood/MIME/2046/rfc2046.html#5.2.3>?
>> That could contain all of the necessary information (a short
>> disclaimer could go in the attachment's "phantom body").
>
>
> This looks promising, Timeless seems to favor using
> message/external-body as well; however, the first stage of work, the
> work specifically covered by bug 2920, doesn't cover saving and linking
> to attachments (detaching files from a message), it only covers deleting
> attachments.
Hmm...true. There isn't really a "null" MIME-type.
> But for future reference, to help coordinate some of the discussions
> here with later work done on saving and linking to attachments, and for
> clarification...
>
> Are attachments part of the body of a message, or are they separate from
> the body?
Attachments are part of the body of a message. A message with
attachments is of type multipart/related. In a multipart MIME message,
the body is a series of "parts", which have their own MIME types.
> Can message/external-body be applied to an attachment?
Yes, definitely. Parts of a multipart message can have any MIME type.
The MIME types RFC (RFC 2046) even mentions this specifically,
suggesting the use of multipart/alternative with message/external-body
parts giving different access-types or locations for the file...a sort
of MIME mirroring system.
> Would
> message/external-body only be useful for saving and linking to
> attachments (detaching files), or would it also be useful for deleting
> attachments?
Not with the currently defined access-types, but Mozilla could create
its own "experimental" access-type to mean external-bodies that can't be
retrieved. Call it X-DELETED, X-REMOVED, X-UNAVAILABLE, or something. A
little weird, but legal, and it would work (to the best of my knowledge).
Matt Coughlin
the discussion, at a slow but steady pace.
>> Are attachments part of the body of a message, or are they separate
>> from the body?
>
> Attachments are part of the body of a message. A message with
> attachments is of type multipart/related. In a multipart MIME message,
> the body is a series of "parts", which have their own MIME types.
Thanks for the clarification. This was causing a lot of confusion when
reading through some of the RFCs and reading through people's suggestions.
What do you call the header section as a whole, and each individual
entry in the header section? Are they called "header" and "header tag",
respectively, or something similar to that?
>> Would message/external-body [...] also be useful for deleting
>> attachments?
>
> Not with the currently defined access-types, but Mozilla could create
> its own "experimental" access-type to mean external-bodies that can't be
> retrieved. Call it X-DELETED, X-REMOVED, X-UNAVAILABLE, or something. A
> little weird, but legal, and it would work (to the best of my knowledge).
If it helps any to know, the current approach favored for deletion of
attachments (though it's still up in the air) is to replace the
attachment with a text-info attachment that describes the deleted
attachment (and the fact that it was deleted), as well as possibly
adding a header entry for posterity which states that the message was
altered.
There's bound to be some awkwardness with using text-info replacements,
but the alternatives (other than doing nothing at all) are much worse
(as previously discussed with timeless in the thread).
With this approach in mind, does any particular access-type, MIME
entity, or header entry come to mind, either an existing one or one we
could create, that would be especially useful (perhaps along the lines
of what you were just suggesting)?
--------------------------------------------------------------
One of the other questions relates to the following portion of RFC2046:
'As with "message/partial", MIME entities of type
"message/external-body" MUST have a content-transfer-encoding of 7bit
(the default). In particular, even in environments that support binary
or 8bit transport, the use of a content- transfer-encoding of "8bit" or
"binary" is explicitly prohibited for entities of type
"message/external-body".'
AFAICT (please correct me if I'm wrong on this) message/external-body
can only be used to reference encoded attachments (in this case 7bit
encoding). You can't detach a file in decoded form, to save space and
so it can be easily accessed outside of the e-mail client as well as
through the e-mail client, and still reference that copy of the
attachment using message/external-body.
This would create some redundancy and a waste of space in situations
where a user needs to both save the attachments in decoded form and
access them through the e-mail client (either in the message or as
detached files). It wouldn't be feasible to combine save and detach as
a single operation, because the user would mistakenly assume that the
decoded saved file was the one being linked to in the e-mail, they
wouldn't realize that there was still an encoded copy of the attachment
that was taking up space. And it wouldn't be possible to access,
through an e-mail, attachments in the smaller decoded form (though that
factor might not be significant enough to matter much).
For the most part, the encoded detached files would be of internal use
only, they wouldn't be of any direct use to the user. It might simplify
things for Mozilla, versus linking to decoded files, for managing where
to store files and what filename to use, as well as restoring
attachments to a message (if that's ever needed), but it might not be
nearly as useful to the user.
Do you see any clear benefits to storing detached files in decoded form,
perhaps related to security, ease-of-management in Mozilla, or
compliance with standards? Do you think the benefits of this would
outweigh the disadvantages?
Thanks again for the help :-)
Matt
Garth Wallace
> Thanks for the feedback so far. I can see some progress being made in
> the discussion, at a slow but steady pace.
>
>>> Are attachments part of the body of a message, or are they separate
>>> from the body?
>>
>>
>> Attachments are part of the body of a message. A message with
>> attachments is of type multipart/related. In a multipart MIME message,
>> the body is a series of "parts", which have their own MIME types.
>
>
> Thanks for the clarification. This was causing a lot of confusion when
> reading through some of the RFCs and reading through people's suggestions.
>
> What do you call the header section as a whole, and each individual
> entry in the header section? Are they called "header" and "header tag",
> respectively, or something similar to that?
The header section is just called the "header section" or the "headers".
Each individual entry is just called a "header".
>>> Would message/external-body [...] also be useful for deleting
>>> attachments?
>>
>>
>> Not with the currently defined access-types, but Mozilla could create
>> its own "experimental" access-type to mean external-bodies that can't
>> be retrieved. Call it X-DELETED, X-REMOVED, X-UNAVAILABLE, or
>> something. A little weird, but legal, and it would work (to the best
>> of my knowledge).
>
>
> If it helps any to know, the current approach favored for deletion of
> attachments (though it's still up in the air) is to replace the
> attachment with a text-info attachment that describes the deleted
> attachment (and the fact that it was deleted), as well as possibly
> adding a header entry for posterity which states that the message was
> altered.
>
> There's bound to be some awkwardness with using text-info replacements,
> but the alternatives (other than doing nothing at all) are much worse
> (as previously discussed with timeless in the thread).
Yeah, that'd probably be more reasonable.
> With this approach in mind, does any particular access-type, MIME
> entity, or header entry come to mind, either an existing one or one we
> could create, that would be especially useful (perhaps along the lines
> of what you were just suggesting)?
You'd just be replacing whatever section you were "deleting" with a
section with the text/plain content type.
> One of the other questions relates to the following portion of RFC2046:
> 'As with "message/partial", MIME entities of type
> "message/external-body" MUST have a content-transfer-encoding of 7bit
> (the default). In particular, even in environments that support binary
> or 8bit transport, the use of a content- transfer-encoding of "8bit" or
> "binary" is explicitly prohibited for entities of type
> "message/external-body".'
>
> AFAICT (please correct me if I'm wrong on this) message/external-body
> can only be used to reference encoded attachments (in this case 7bit
> encoding). You can't detach a file in decoded form, to save space and
> so it can be easily accessed outside of the e-mail client as well as
> through the e-mail client, and still reference that copy of the
> attachment using message/external-body.
Nope. As I read it, that restriction only applies to the
message/external-body entity itself, *not* the file being referenced
from it. That file can be in whatever formats and encodings are
supported by the access-type.
The "in environments that support binary or 8-bit transport" clause
refers to the fact that MIME is used in some protocols that aren't 8-bit
clean, and some that are. Somebody correct me if I'm wrong about this.
Matt Coughlin
>> can only be used to reference encoded attachments (in this case 7bit
>> encoding). You can't detach a file in decoded form, to save space and
>> so it can be easily accessed outside of the e-mail client as well as
>> through the e-mail client, and still reference that copy of the
>> attachment using message/external-body.
>
> Nope. As I read it, that restriction only applies to the
> message/external-body entity itself, *not* the file being referenced
> from it. That file can be in whatever formats and encodings are
> supported by the access-type.
If so, that's very good news. That would mean message/external-body is
a very real possibility for eventually saving and linking to attachments
(detaching). At the very least it'd be a fall-back option in the event
that no other approach was found to be reasonable, and it may be the
preferred option.