Line Wrapping Question
Sukvinder Singh Gill Exchange
using our product have commented on.
Currently, when sending Internet Mail with long lines, we
default to using QP encoding so that MIME aware clients
can unwrap the lines, and display as required on their
viewers. (We use 140chars as the threshold)
We have received numerous complaints (most people assumed
that this was a bug) from people that use non-MIME aware
apps about the equal signs on every line.
I thought about this for awhile, and hoped the issue may
be resolved by using a prologue informing people that this
message was a MIME message, and they may see some
irregular things in it. Therefore every message that we
send, is sent using the multipart type, so that we could
use the prologue.
This is catch22, since people complained about the use of
multipart, even when we could have sent just text/plain
messages. (They also complained about the contents of the
prologue, but thats another issue)
I see the choices for our product as follows, and would
like your feedback on what you consider the right choice,
or suggestions on doing something better.
a)We provide an option to hard wrap lines, therefore we
don't use QP encoding to deal with the line wrapping issue
at all.
i.e. CTE is 7-bit unless QP is required for some other
reason
b) We just hard wrap without an option even when using QP
(An option is probably a good thing to turn this to use QP
wrapping)
c) We leave things the way they are.
d) We don't send the prologue, and hope that the world
will move to MIME, and this becomes a non-issue.
I really only consider option a, or b as viable for our
user population, but I'll wait to hear back from people
before I commit.
Thanks,
-Suki
Sukvinder S. Gill
Microsoft Corporation
E-Mail - su...@microsoft.com
Tel: 206-936-9761
Donald E. Eastlake 3rd
On Wed, 7 Feb 1996, Sukvinder Singh Gill (Exchange) wrote:
> Date: Wed, 7 Feb 1996 08:31:18 -0800
> From: Sukvinder Singh Gill (Exchange) <su...@wspu.MICROSOFT.com>
> To: "ietf...@list.cren.net" <ietf...@list.cren.net>
> Subject: Line Wrapping Question
>
> I'd like to get some advice on a problem that many people
> using our product have commented on.
>
> Currently, when sending Internet Mail with long lines, we
> default to using QP encoding so that MIME aware clients
> can unwrap the lines, and display as required on their
> viewers. (We use 140chars as the threshold)
You don't say much about the environment you are opperating in. Why are
you encountering very long lines? If it is because the message is the
output of a text editing system that assumes soft wrapping and where a
real new line character represents a paragraph, that's one thing. If the
problem is that most of the stuff is short but you have one or two lines
at the begining of the message that are unwrapped 822 headers from a
forwarded message, that's a completely different story.
> We have received numerous complaints (most people assumed
> that this was a bug) from people that use non-MIME aware
> apps about the equal signs on every line.
>
> I thought about this for awhile, and hoped the issue may
> be resolved by using a prologue informing people that this
> message was a MIME message, and they may see some
> irregular things in it. Therefore every message that we
> send, is sent using the multipart type, so that we could
> use the prologue.
I would like to be able to recommend that you just go with MIME but if
the basic problem is lack of MIME support, then using another MIME
feature (multi-part in addition to QP) probably won't solve your problem
of perception.
> This is catch22, since people complained about the use of
> multipart, even when we could have sent just text/plain
> messages. (They also complained about the contents of the
> prologue, but thats another issue)
? Surely you could make the use of the multipart also conditional on
there being long lines. The prolog is good in that you can give a
message saying you are using MIME that won't be seen or get in the way
for MIME capable MUAs.
> I see the choices for our product as follows, and would
> like your feedback on what you consider the right choice,
> or suggestions on doing something better.
>
> a)We provide an option to hard wrap lines, therefore we
> don't use QP encoding to deal with the line wrapping issue
> at all.
> i.e. CTE is 7-bit unless QP is required for some other
> reason
Options are good. An option to hard wrap seems useful. (Although if
there are only one or two long lines out of many, QP should not be
objectionable...)
> b) We just hard wrap without an option even when using QP
> (An option is probably a good thing to turn this to use QP
> wrapping)
>
> c) We leave things the way they are.
>
> d) We don't send the prologue, and hope that the world
> will move to MIME, and this becomes a non-issue.
I think the world is moving to MIME, although gradually, so it will
eventually become a non-issue, but your customers are complaining now.
The user based and organization environments probably vary so much that
one or more options is probably the only way to do.
> I really only consider option a, or b as viable for our
> user population, but I'll wait to hear back from people
> before I commit.
>
> Thanks,
> -Suki
>
> Sukvinder S. Gill
> Microsoft Corporation
> E-Mail - su...@microsoft.com
> Tel: 206-936-9761
Donald
=====================================================================
Donald E. Eastlake 3rd +1 508-287-4877(tel) d...@cybercash.com
318 Acton Street +1 508-371-7148(fax) d...@world.std.com
Carlisle, MA 01741 USA +1 703-620-4200(main office, Reston, VA)
http://www.cybercash.com http://www.eff.org/blueribbon.html
Ned Freed
> > Currently, when sending Internet Mail with long lines, we
> > default to using QP encoding so that MIME aware clients
> > can unwrap the lines, and display as required on their
> > viewers. (We use 140chars as the threshold)
> Umm.. this *is* using text/enriched and not text/plain, correct?
> If it isn't, that may be the solution you need....
No. They use text/plain. Bad choice. I pointed out that text/enriched is
a viable option in my previous note on the subject.
Ned
Sukvinder Singh Gill Exchange
paragraph case.
-Suki
>----------
>From: Donald E. Eastlake 3rd[SMTP:d...@cybercash.com]
>Sent: Wednesday, February 07, 1996 10:52 AM
>To: Sukvinder Singh Gill (Exchange)
>Cc: ietf...@list.cren.net
>Subject: Re: Line Wrapping Question
>
>Hi,
>
>On Wed, 7 Feb 1996, Sukvinder Singh Gill (Exchange)
>wrote:
>> Date: Wed, 7 Feb 1996 08:31:18 -0800
>> From: Sukvinder Singh Gill (Exchange) <su...@wspu.MICROSOFT.com>
>> To: "ietf...@list.cren.net" <ietf...@list.cren.net>
>> Subject: Line Wrapping Question
>>
>> I'd like to get some advice on a problem that many people
>> using our product have commented on.
>>
>> Currently, when sending Internet Mail with long lines, we
>> default to using QP encoding so that MIME aware clients
>> can unwrap the lines, and display as required on their
>> viewers. (We use 140chars as the threshold)
>
Valdis.K...@vt.edu
> Currently, when sending Internet Mail with long lines, we
> default to using QP encoding so that MIME aware clients
> can unwrap the lines, and display as required on their
> viewers. (We use 140chars as the threshold)
Umm.. this *is* using text/enriched and not text/plain, correct?
If it isn't, that may be the solution you need....
Valdis Kletnieks
Computer Systems Engineer
Virginia Tech
Ned Freed
> using our product have commented on.
FYI, this is about the tenth time I've gone other this set of issues with
either a Microsoft customer or someone from Microsoft. I hope this pass will be
more effective than the previous ones have been.
> Currently, when sending Internet Mail with long lines, we
> default to using QP encoding so that MIME aware clients
> can unwrap the lines, and display as required on their
> viewers. (We use 140chars as the threshold)
The problem in a nutshell is simple: You don't understand the intent
and operation of MIME.
More specifically, quoted-printable is *NOT* a presentation mechanism. It is a
transport mechanism. As such, you must not assume that the line wrapping
imposed by quoted-printable encoding takes care of your presentation problems
in any way, shape, or form on the systems that receive your messages.
There is confusion in this area because quoted-printable is actually designed
to be a kind of half-assed presentation mechanism for non-MIME-capable readers.
But this *ONLY* applies to non-MIME readers. MIME-capable readers remove the
presentation aspects of quoted-prinable before displaying the result.
This means that the material inside of the quoted-printable encoding has to be
in a form that is suitable for presentation. Nothing more and nothing less.
Text/plain material is material formatted for direct presentation to the user.
It isn't supposed to be line-wrapped by the user agent. Indeed, if it is
wrapped willy-nilly many common usage elements of text/plain will be lost. This
message is a case in point: I have used, as is my custom, a "> " quoting
convention to mark passages from your original message. Line wrap this message
and this effect will be trashed.
> We have received numerous complaints (most people assumed
> that this was a bug) from people that use non-MIME aware
> apps about the equal signs on every line.
The only assumption people are making is that your intent is that your messages
will be presented in a reasonable way by MIME-compliant agents And since they
aren't, it's therefore a bug in your implementation.
You are operating under the belief that text/plain material under the
quoted-printable encoding will be line-wrapped on presentation. And this is not
the case. It is not supposed to be line-wrapped by receiving agents. If you
want your text to be line-wrapped, either use text/enriched or define a new
subtype of text for this purpose.
> I thought about this for awhile, and hoped the issue may
> be resolved by using a prologue informing people that this
> message was a MIME message, and they may see some
> irregular things in it. Therefore every message that we
> send, is sent using the multipart type, so that we could
> use the prologue.
You're now trying to solve the problem in the one case where it doesn't exist.
A non-MIME reader is the only one that will see this prologue. But it is also
the only one that necessarily will see a the line-wrapped quoted-printable!
In other words, you have provided the information in a way that guarantees that
the people who need it won't see it, and the people who will see it won't need
it.
> This is catch22, since people complained about the use of
> multipart, even when we could have sent just text/plain
> messages. (They also complained about the contents of the
> prologue, but thats another issue)
Of course they did -- they didn't need the prologue since it doesn't help then
in any way. And the ones who might be helped by the prologue never see it.
> I see the choices for our product as follows, and would
> like your feedback on what you consider the right choice,
> or suggestions on doing something better.
> a)We provide an option to hard wrap lines, therefore we
> don't use QP encoding to deal with the line wrapping issue
> at all.
> i.e. CTE is 7-bit unless QP is required for some other
> reason
Like it or not, this is an option you are going to have to provide.
> b) We just hard wrap without an option even when using QP
> (An option is probably a good thing to turn this to use QP
> wrapping)
This is just a variant of the first approach, i.e. make it non-optional.
I don't really count it as a separate approach to the problem.
> c) We leave things the way they are.
This will lead to other implementors having to provide facilities to
line-wrap text/plain material. We'll have to work around your
bug, in other words.
In fact I already see this as a foregone conclusion -- we've already set up
workarounds for this Microsoft problem at a number of sites.
> d) We don't send the prologue, and hope that the world
> will move to MIME, and this becomes a non-issue.
The fact that you say this demonstrates that you do not have a clear
understanding of the problem.
The world moving to MIME makes this much worse, not better. People without MIME
readers are inconvenienced by the tacky appearance of the quoted-printable
encoding, but aside from that they have nothing to complain about. It's people
who *HAVE* *STANDARDS-COMPLIANT* *MIME* *IMPLEMENTATIONS* that bear the brunt
of your current practices. They are the ones who often as not have to manually
wrap lines in replies to messages you send.
There are also some options you have not considered:
(e) Use text/enriched. The characteristics of this type are such that you
could build material that looks decent on MIME readers and non-MIME
readers at the same time.
(f) Define a new subtype of text that specifically allows for
> I really only consider option a, or b as viable for our
> user population, but I'll wait to hear back from people
> before I commit.
I would recommend a combination of (a) and (e). Provide an option to present
text material in an unencoded form that looks reasonable when it is displayed.
And also provide an option to present text material in a more sophisticated
form. Neither of these necessarily should be encoded by default. The latter
form must be carefullly implemented so that it looks decent on non-MIME readers
but provides whatever presentation elements that lie in the intersection of
text/enriched and your internal format.
Ned
Ned Freed
> I have no argument that QP is not a presentation encoding, it's a transfer
> encoding. However, QP is specifically designed to support sending of long
> lines. Most implementors of MIME readers have recognized this and made sure
> that they wrap long lines when presenting them to the user (usually at the
> window width at presentation time, not to some fixed width).
And when they do this often as not they turn legitimate material into garbage.
I receive legitimate material all the time as text/plain containing long lines
that absolutely cannot be wrapped. In fact I would say that this amounts to a
high percentage of the messages I receive.
> I agree, text/plain is defined as "unformatted" text. But wrapping unformatted
> text is a very reasonable presentation thing to do (perhaps as a configurable
> user option).
I disagree. It may or may not be reasonable, depending on what the original
message had in it. The problem is that if you wrap blindly you stand an
excellent chance of making legitimate content unusable.
> From your rather vociferous reply, I take it your mail client or gateway didn't
> wrap text/plain by default. Given that, you've taken the world view that "just
> send QP" is a radical misunderstanding of the difference between transfer
> encoding and presentation encoding. But most MIME implementors have found that
> sending QP results in decent interoperability of automatically wrapped text and
> therefore took that approach. They knew that any mail client that supported
> MIME, supported QP, but couldn't guarantee that it would support text/enriched.
> So "just send QP" was a more conservative thing to do.
I'm not sure how you arrived at any of this. It is absolutely wrong in almost
every particular.
First of all, the client I happen to use most of the time wraps long lines just
fine if and when I want it to. I have no problem getting this effect when I
need it. However, I keep it turned off most of the time because most of the
messages I receive would be turned into trash by line wrapping. And again, this
message is a case in point, as are most of the messages I send and receive
every day.
Second, Innosoft is in the business of selling MTA/gateway functionality, not
email client software. We include some clients in our packages (including the
one I happen to use) but as I say there are no problems wrappping lines in any
of our clients if that's the effect users want. But this misses the point --
MTA/gateway functional is our focus and in general it is not the business of
either MTAs or gateways to change the presentation of the material passing
through it, so the wrapping problem is (or should be) a non-issue for us.
The problem is that there are vast number of clients, both MIME-capable and
non-MIME-capable, in widespread use that cannot do wrapping. These clients were
written with the expectation that plain text material they are given are
already in a format suitable for presentation. This is how RFC822 message text
is defined to work, and MIME's text/plain is simply a way of tacking a label on
what has been standard practice for over 13 years. To say that such clients are
broken and have to change is simply not acceptable -- we have rules about not
invalidating old practices.
These are the clients I have to support. They are fully standards-compliant,
and even if they weren't I doubt I could get a significant number of them to
change. As such, unless Microsoft and the others indulging in this practice
change their behavior the only solution I can implement is to perform line
wrapping on text/plain object in the context of an MTA. (This is not limited to
gateways.) This is at best a serious layering violation. At worst it ends up
trashing many messages, since there is in general no way I can tell the
difference between, say, a quoted section that cannot be wrapped and a section
that can.
Does all this lead me to say that Microsoft isn't playing by the rules? You bet
it does. But I'm not saying this because I have clients that lack
functionality. I don't, and even if I did it would not excuse Microsoft's
behavior. I say it because I believe that this practice is wrong. Period.
Ned
Terry Crowley
I know I shouldn't jump into this argument, but I can't resist the opportunity.
I have no argument that QP is not a presentation encoding, it's a transfer
encoding. However, QP is specifically designed to support sending of long
lines. Most implementors of MIME readers have recognized this and made sure
that they wrap long lines when presenting them to the user (usually at the
window width at presentation time, not to some fixed width).
I agree, text/plain is defined as "unformatted" text. But wrapping unformatted
text is a very reasonable presentation thing to do (perhaps as a configurable
user option).
I'm sure if text/enriched had seemed like more of an integral part of the MIME
spec, and if there hadn't been confusion introduced by QP's handling of long
lines and text/enriched's handling of long lines, more implementors would have
decided that when deciding to send formatted text, to send QP-encoded
text/enriched rather than "just send QP". But that's not how it happened.
>From your rather vociferous reply, I take it your mail client or gateway didn't
wrap text/plain by default. Given that, you've taken the world view that "just
send QP" is a radical misunderstanding of the difference between transfer
encoding and presentation encoding. But most MIME implementors have found that
sending QP results in decent interoperability of automatically wrapped text and
therefore took that approach. They knew that any mail client that supported
MIME, supported QP, but couldn't guarantee that it would support text/enriched.
So "just send QP" was a more conservative thing to do.
Yes, you really need a presentation semantics to cleanly handle quoting of
wrapped text. But that's a distinct issue.
Terry
Ned Freed
> support of MIME or any aspect of Internet Mail, and I will
> make sure that these issues get handled correctly. I
> apologize if you have been frustrated by many people
> asking the same question on this topic.
I appreciate the pointer, and I will try to cc you on any issues that arise in
this area.
> Ned, I believe that your take on QP line wrapping is never
> use the line wrapping for text/plain message bodies on
> outbound messages, but do so for attachments right?
Sort of. Whether or not to do QP encoding (it's best not to call it line
wrapping as this confuses its function with its form) is is a function of what
transport you are using and only that. It is not a function of what
presentation you expect or want.
If, for example, you are using HTTP as your transport, it is downright silly to
ever use QP or BASE64 since the transport is capable of handling raw binary.
Just use binary and be done with it.
If you're going out over old-style 7bit SMTP you'd pick QP or BASE64 for sure
if there's any 8bit content or the content isn't textual in form or has lines >
998 characters in length. If the content is short lines of 7bit text you'd have
the option of using QP or BASE64 if you don't trust the transport to deliver
things intact, but otherwise there is no requirement that you use anything
other than 7bit.
Note that the concept of attachments isn't something MIME recognizes as part of
message structure. The attachment/inline distinction is drawn by the
content-disposition header (RFC1806).
> Therefore when a UA generates long lines with extended
> characters in a message. Lets say the length of the line
> is 87 characters, and you do not support text/enriched.
> You should use option b) below.
SMTP is supposed to guarantee you 1000 character lines. Lines longer than 1000
characters have to be encoded with either QP or BASE64, but an 87 character
line should be fine. However, this presumes that you really want the line
to be 87 characters long when it is displayed. If you want it wrapped when
it is displayed you should wrap it before you encode it and send it out.
> a) Use QP and therefore wrap lines using the QP method
> b) Use QP and hard wrap lines at <= 76 characters
You still seem to be confusing QP encoding with presentation.
> The problem with option a is that it could force the
> unwrapping at the receiving end, and subsequently could
> require re-wrapping for UAs not capable of presenting
> lines longer than say 80 chars for sake of a better
> example
> The problem we originally saw with option b is that jagged
> lines get presented to the recipient
Sounds to me like your hard wrapping option wasn't implemented properly.
If you know something is a wrappable entity you can do all sorts of
things to make it display better on a fixed width display.
> For option b we also considered the following issue when
> we did the QP style line wrapping -
> A UA generates a message, with extended characters, and
> all lines were <= to 76 characters long, therefore the QP
> encoding for long lines was not an issue. When the
> recipient replies to the message, they prefixed a "> "
> combination on the lines, therefore taking some of the
> line lengths over 76 characters. Now the reply is sent
> with a jagged line.
This is part of the reason why you need to wrap before you encode.
> We are considering hard wrapping at <= 76 characters even
> if QP is used for the moment, but really would like your
> feedback on the right thing to do. i.e. is jagged lines
> something you believe is acceptable in Internet Mail for
> these cases?
Right. This is the approach you need to use by default.
> We are looking at using a lower limit for the wrapping
> than 76, and using an upper threshold to decide when to
> wrap. Have you used heard of people doing this, and what
> upper threshold did they use. We would use the lower limit
> to deal with cases where replies to mail originated in our
> systems, could be replied to and we could respond back
> without forcing jagged lines for those messages atleast,
> but this reasoning is still being thought through.
When I think about it I use a threshold of 80 and wrap at 76. This lets someone
put "> " in front and not have stuff fall off the edge.
> RE: Text/Enriched (I am investigating what supporting that
> means right now by the way, but I am concerned with the
> interoperability cases since not every MIME aware
> application may have implemented downgrading to text/plain
> if they don't support text/enriched.)
Generating text/enriched that looks good on a non-MIME UA *and* works
correctly when viewed on a MIME UA that supports enriched is a nice
clean approach. The downside is that this is a very difficult thing
to get right.
Ned
Pete Resnick
>Ned, I believe that your take on QP line wrapping is never
>use the line wrapping for text/plain message bodies on
>outbound messages, but do so for attachments right?
I hope that's not what Ned meant. If a user wants to send long lines in
text/plain, QP is a good thing. But I don't think that's the central issue
here.
Let's back up a bit and define the problem. The basic thing that you want
is to deal with auto-wrapped text that the user is giving you. The user is
typing with a composition tool which auto-wraps and they desire the same
effect on the remote end. That is, they type a single pargraph without
hitting the return key, and you want the same or similar auto-wrapping
behavior to occur on receipt of such a message. Specifically, the sender
doesn't care particularly how the line breaks appear on the screen of the
recipient, so long as the text is "wrapped nicely".
There is no way to represent the desire to have auto-wrapped text in
text/plain. Text/plain makes no indication about presentation. And since QP
line breaking is not a presentation mechanism either, that doesn't help
you. So, in order to get the desired behavior, you need to go to another
MIME type.
Text/enriched is a good choice here. For users who have a text/enriched
aware agent, the lines will have any inserted CRLFs replaced by spaces, and
text/enriched specifies that lines should be presented properly wrapped.
For those who don't have text/enriched, it is not a problem: If you get a
subtype of text that you don't understand, you're supposed to treat it like
text/plain. That means in the worst case, a user may get the text with
CRLFs where the sender intended just the continuation of the line which
would be wrapped. But since the text/enriched generator is responsible for
inserting those CRLFs, they can be put in a reasonable place, somewhere
less than 80 characters per line.
For text that should not be unwrapped and autowrapped, you simply mark it
as <nofill> in text/enriched.
You could also use a proprietary format which indicates auto-wrapping
presentation. But since there are people who have MIME programs which know
how to do text/enriched, and all that you care about is the line wrapping,
that wouldn't be terribly friendly.
Take a look at the text/enriched draft that Amanda Walker and I wrote which
is about to be released as an RFC. You can find it as:
<ftp://ds.internic.net/internet-drafts/draft-resnick-text-enriched-03.txt>
or an HTML version of the doc at:
<http://www.qualcomm.com/People/presnick/textenriched.html>
It really looks like this is what you want to do.
pr
--
Pete Resnick <mailto:pres...@qualcomm.com>
QUALCOMM Incorporated
Home: (217)337-1905 / Fax: (217)337-1980
John W. Noerenberg
>I'd like to get some advice on a problem that many people
>using our product have commented on.
>
>default to using QP encoding so that MIME aware clients
>can unwrap the lines, and display as required on their
>viewers. (We use 140chars as the threshold)
Hello, Mr. Exchange :-)!
I'm confused about your 140 character threshold. 140 is the threshold of
what? Your QP-encoder should be emitting lines of text no longer than 76
characters.
>We have received numerous complaints (most people assumed
>that this was a bug) from people that use non-MIME aware
>apps about the equal signs on every line.
We also have encountered these complaints.
There is no perfect solution. QP is intended to convey information to a
MIME-aware UA. Someone reading a message containing QP with a UA that
doesn't follow MIME rules is gonna see stuff they think looks funny.
There's no 2 ways about that.
We wrestled with this in Eudora's design literally for years. The problem
predates QP. Without MIME a UA designer has to make assumptions about the
nature of both sender and receiver's display. But windowing systems
invalidate the assumptions.
The problem you have to solve is facilitating communication between people
whose systems are built under conflicting assumptions about what can be
displayed and how it is displayed.
What to do about those awful equal signs is putting the emphasis in the
wrong place.
QP line-wrapping permits the sender to signal where line breaks should
occur and where line breaks can be ignored. QP also deals with 8-bit
character codes which can be rendered as printable glyphs assuming your
display is not limited to USASCII. But only MIME-aware UAs are prepared to
deal with the encoding. Over the years there have been Luddites who have
loudly proclaimed that QP is wrong because it makes the text worse.
As a consequence, Eudora goes to extrodinary means to estimate what is a
reasonable presentation of a message without QP to permit those who are
forced to live with the Luddites among them to be able to lead a reasonably
tranquil life. Users have to have to be able to choose whether or not to
use the capability, because only they can decide what is required for the
message they are trying to express. And their UA must arm them with enough
information so they can choose wisely. But it cannot make the choice for
them.
Eventually the benefits of richer character sets overwhelm even the most
ardent Luddite. Or they die. One or the other is guaranteed.
Of your choices, we are closest to a). However, Eudora encourages the use
of QP. If other MIME-compliant UAs do likewise, eventually d) will come to
pass.
john noerenberg
<mailto:jw...@qualcomm.com>
----------------------------------------------------------------------
Be sure of this, O young ambition, all mortal
greatness is but disease.
-- Herman Melville, "Moby-Dick", 1851
----------------------------------------------------------------------
Jamie Zawinski
>
> >Currently, when sending Internet Mail with long lines, we
> >default to using QP encoding so that MIME aware clients
> >can unwrap the lines, and display as required on their
> >viewers. (We use 140chars as the threshold)
>
> what? Your QP-encoder should be emitting lines of text no longer than 76
> characters.
There are two issues: how long QP-encoded lines should be (and that
should be 76) and *whether* to use QP at all. Since SMTP says that
lines may be up to 1000 characters long, one needn't necessarily
encode lines which aren't longer than that. I presume that this
is the threshold he's talking about.
The MIME spec does not say that a legal MIME message must have lines
less than 76. It says that *if* a part is QP-encoded, *then* it must
follow that line-length constraint.
> There is no perfect solution. QP is intended to convey information to a
> MIME-aware UA. Someone reading a message containing QP with a UA that
> doesn't follow MIME rules is gonna see stuff they think looks funny.
> There's no 2 ways about that.
Well that's true, but I would phrase it as, "QP is an alternate way of
representing data".
> We wrestled with this in Eudora's design literally for years. The problem
> predates QP. Without MIME a UA designer has to make assumptions about the
> nature of both sender and receiver's display. But windowing systems
> invalidate the assumptions.
But QP has nothing to do with the sender's or receiver's display.
All it has to do wit is whether both support MIME. QP is about
*transport*, not about *presentation*. QP is exactly like Base64
in this respect, or uuencode. It is "ASCII armor" to prevent data
from being mangled along the way. Nothing more.
> The problem you have to solve is facilitating communication between people
> whose systems are built under conflicting assumptions about what can be
> displayed and how it is displayed.
>
> What to do about those awful equal signs is putting the emphasis in the
> wrong place.
True.
> QP line-wrapping permits the sender to signal where line breaks should
> occur and where line breaks can be ignored.
Phrasing it this way bothers me, since it speaks to it from a
presentation view again, rather than properly disconnecting the
representation of the data from the raw data itself.
> QP also deals with 8-bit
> character codes which can be rendered as printable glyphs assuming your
> display is not limited to USASCII. But only MIME-aware UAs are prepared to
> deal with the encoding. Over the years there have been Luddites who have
> loudly proclaimed that QP is wrong because it makes the text worse.
I don't think this is a fair characterization of that side of the
argument; the argument is more along the lines of, "I'd rather take my
chances that my transport is 8-bit-clean than assume my recipients can
do MIME." In some situations, that's completely the right thing to do
(depending on your audience, and what you know about the paths between
you and them.)
> As a consequence, Eudora goes to extrodinary means to estimate what is a
> reasonable presentation of a message without QP to permit those who are
> forced to live with the Luddites among them to be able to lead a reasonably
> tranquil life. Users have to have to be able to choose whether or not to
> use the capability, because only they can decide what is required for the
> message they are trying to express. And their UA must arm them with enough
> information so they can choose wisely. But it cannot make the choice for
> them.
Can you tell us what Eudora's rules are?
> Eventually the benefits of richer character sets overwhelm even the most
> ardent Luddite. Or they die. One or the other is guaranteed.
>
> Of your choices, we are closest to a). However, Eudora encourages the use
> of QP. If other MIME-compliant UAs do likewise, eventually d) will come to
> pass.
In Netscape, there is a preference toggle between "Allow 8-bit" and
"MIME Compliant (Quoted-Printable)". It defaults to 8-bit now, since
there was great resistence to the use of QP when we made that the
default in an earlier beta. And this resistance was, in fact, from
the very community that QP was designed to help: those with non-7bit
languages. Many of our European users told us that the chance of their
recipients having only a 7-bit path was far, far less likely than the
chance that their recipients were able to decode MIME messages. Sad
but true...
In Netscape, a MIME part (either the main text, or an "attachment")
will be encoded if:
- it is of a non-text type; or
- the "use QP" option has been selected -and- high-bit characters
exist in the document; or
- any NULLs exist in the document; or
- any line is longer than 900 bytes (SMTP requires 1000, but we
picked 900 to allow some "slop".)
If we have decided that a document should be encoded, we must then
decide what style of encoding to use:
- If more than 10% of the document consists of bytes outside of
the printable-7bit-ASCII range, then we always use base64 instead
of QP, under the assumption that this is a "binary" file of some
kind.
- Base64 is always used for non-text documents (image/*, audio/*,
etc) on the off chance that a GIF file (for example) might contain
primarily bytes in the ASCII range, thus failing the 10% test. In
this case, using the quoted-printable representation might cause
corruption due to the translation of CR or LF to CRLF. So, when we
don't know that the document is of a type that has "lines", we
don't use QP.
So this means, roughly:
- 7-bit text files with lines <900 will always be sent as-is;
- text files with lines >900 will always be sent in QP;
- 8-bit text files will be sent as-is or with QP, at the user's
choice;
- files which are clearly not text will be sent in base64.
--
Jamie Zawinski j...@netscape.com http://www.netscape.com/people/jwz/
``A signature isn't a return address, it is the ASCII equivalent of a
black velvet clown painting; it's a rectangle of carets surrounding
a quote from a literary giant of weeniedom like Heinlein or Dr. Who.''
-- Chris Maeda
Terry Crowley
1. Yes, QP is simply an encoding mechanism.
2. Yes, it shouldn't be used as a presentation mechanism (e.g assuming long
lines wrap).
But, THIS WAS NOT CLEAR. It really wasn't. Trust me.
The whole discussion of QP in the MIME spec discusses "hard" and "soft"
carriage returns, and the examples show lines that are wrapped by QP, being
rewrapped when decoded (partially an unfortunate consequence of the ASCII only
presentation medium). The discussion doesn't center on long lined data that
needs to get through wrapping transports/gateways. It talks about hard and
soft CR's, a word-processing notion.
Ned, step back for a moment and recognize that your world view is not everyone
elses. For example, you talk about "wrapping blindly". But if I'm presenting
the text in a windowed screen, I can "wrap blindly" and allow the user to
resize the window if they don't like the way it wrapped at that size. You're
writing gateways, so you're forced to make a hard and fast decision that isn't
revokable. In your case, wrapping would be wrapping blindly. In my case it's
not.
But this is history. The bottom line is, there are still MIME capable
mailers/gateways that do things in incompatible ways because of confusion in
reading/interpreting/using the MIME spec. The programmers that are reading
that spec are not stupid. It just wasn't clear.
Here's a clear statement:
1. For maximum interoperability of text composed by the user in an
automatically wrapping text editor, send text/plain with short, hard-wrapped
lines. Wrap at something less that 80 characters (72 or 76 are good choices)
to allow for quoting on reply. If possible, send as 7-bit. If your data
contains 8-bit characters, you'll probably have to provide a user option about
whether they want to encode as QP in this case or "just send 8-bit".
2. If you want to send automatically wrapped text, use text/enriched, encoded
using QP.
If you're a vendor of MIME-capable software, support for text/enriched is a
must.
Terry
Jim Conklin
see lots of things for in your description that I've wished on occasion.
Jim
-------------------------------------------------------------
Jim Conklin,
Director, BITNET/CREN Network Information Center (BITNIC)
con...@info.cren.net
(202) 331-5367
Harald.T....@uninett.no
you say:
>In Netscape, there is a preference toggle between "Allow 8-bit" and
>"MIME Compliant (Quoted-Printable)". It defaults to 8-bit now, since
>there was great resistence to the use of QP when we made that the
>default in an earlier beta. And this resistance was, in fact, from
>the very community that QP was designed to help: those with non-7bit
>languages. Many of our European users told us that the chance of their
>recipients having only a 7-bit path was far, far less likely than the
>chance that their recipients were able to decode MIME messages. Sad
>but true...
I was in the middle of some of the heated exchanges on this in Norway,
a typical (?) 8-bit market.
The problem with the Netscape toggle was that it was the same for Mail
*and for News*, at a time when Netscape was just about the only
newsreader that handled Quoted-Printable. (or "quoted-unreadable", as
one rather vociferous opponent names it).
(There now exists a second, called "knews", and I believe some of the
newer Gnus versions do it too).
We had the same sort of flak when we started using Q-P in mail 2 years
ago, and for almost a year we hacked mailers into using theoretically illegal
"content-transfer-encoding: 8bit", but when we recently went back
to using Q-P in mail, we didn't get *any* objection; the market for
E-mail had accepted MIME.
I believe having separate toggles for defaulting Q-P in Email and in
News would be good for the users.
Harald A
Ned Freed
> 1. Yes, QP is simply an encoding mechanism.
> 2. Yes, it shouldn't be used as a presentation mechanism (e.g assuming long
> lines wrap).
> But, THIS WAS NOT CLEAR. It really wasn't. Trust me.
I disagree. I think it is quite clear. The specification of text/plain says
right at the beginning:
text -- textual information. The subtype "plain" in particular indicates plain
(unformatted) text. No special software is required to get the full meaning of
the text, aside from support for the indicated character set.
And later on it says:
NOTE: The proper interpretation of line breaks when a body is displayed depends
on the media type. In particular, while it is appropriate to treat a line break
as a transition to a new line when displaying a text/plain body, this treatment
is actually incorrect for other subtypes of text like text/enriched [RFC-1563].
In other words, line breaks in text/plain are to be treated as line breaks, and
the text is unformatted and is to be treated as such.
About the only thing missing from this description is an explicit declaration
that it should not be necessary to reformat text/plain by adding line breaks to
display it properly. I will add some wording to that effect to the next
iteration of these documents, not because I think it is necessary but because
it will be an improvement.
> The whole discussion of QP in the MIME spec discusses "hard" and "soft"
> carriage returns, and the examples show lines that are wrapped by QP, being
> rewrapped when decoded (partially an unfortunate consequence of the ASCII only
> presentation medium). The discussion doesn't center on long lined data that
> needs to get through wrapping transports/gateways. It talks about hard and
> soft CR's, a word-processing notion.
Sure it does. But you are ignoring the context in which this discussion occurs.
This discussion doesn't appear in the section on media types. It appears in the
section on transport encodings. The discussion of hard and soft line breaks
appears in the discussion of the sort of output the encoding produces, not in
any disussion of the input to the encoding process.
Furthermore, the preceeding sections explain what the purpose of transport
encodings is, and the stated purpose does not include control of presentation
to the user in any case other than that of quoted-printable being somewhat
kind to non-MIME-capable agents.
> Ned, step back for a moment and recognize that your world view is not everyone
> elses. For example, you talk about "wrapping blindly". But if I'm presenting
> the text in a windowed screen, I can "wrap blindly" and allow the user to
> resize the window if they don't like the way it wrapped at that size. You're
> writing gateways, so you're forced to make a hard and fast decision that isn't
> revokable. In your case, wrapping would be wrapping blindly. In my case it's
> not.
Of course my world view isn't universal. It would be silly to assume so.
However, the point you seem to be consistently missing is that my position is
not based on my own biases as to how things should work. I have never made any
statements as to how I think things should work. As it happens my personal view
as to how it should have been done differs in substantial ways from the views I
have presented in this discussion, and you have no business making assumptions
about my personal view in any case. I quite frankly don't know why you insist
this being a personal issue for me -- you started by trying to make it out to
be somehow related to the limitations of my user agent, and now you're making
it out to be personal bias. It is neither.
My intent is to state how the *standards* *say* things *are* *required* to work
and that software agents exist that operate on the assumption that the
standards are to be obeyed in this regard.
You claim the standards are not clear in this regard. I claim they are, and
that the only reading that could arrive at any other conclusion is one that
totally ignores the context of the some statements and ignores any number of
other statements completely.
It is flat-out impossible to write useful standards documents without assuming
some amount of information is provided by context. Nor can we add text to every
section of the document under the assumption that other sections will not be
read. Were we to do so the document would literally explode in size and the
result would be unusable.
Your philosophy, on the other hand, seems to be the same as the one that now
appears to govern the installation of traffic lights and signs on the roads in
this country: When someone screws up and there's an accident, it must be
because things weren't marked clearly, so the solution to the problem is to
install some additional lights and signs to make things even clearer. Every
misinterpretation requires clarification. Gone is the very real possibility
that some individual or individuals simply messed up.
I claim that this is what happened. Microsoft messed up. They are not immune.
Nobody is. Maybe some other people did so too, but if so I haven't seen or
heard about it. (I have seen Mac gateways get this wrong as well, but once I
talked with the gateway providers it became clear that they knew full well what
they were doing was wrong.)
I refuse to accept as given that any time someone misreads something it is
necessarily the material that they read that is in the wrong. The burden of
proof in order to get me to change something is a lot more than this.
> But this is history. The bottom line is, there are still MIME capable
> mailers/gateways that do things in incompatible ways because of confusion in
> reading/interpreting/using the MIME spec. The programmers that are reading
> that spec are not stupid. It just wasn't clear.
And once again I disagree. The specification is clear on this point. I will
take steps to make it even more clear, but I maintain that it is clear enough
already. I have cited the places where it discusses this aspect of text/plain.
To say that line wrapping should be done on text/plain requires that you simply
ignore what the specification does in fact already say.
The most you have been able to do is say that the talk about soft and hard line
breaks in the section on quoted-printable encoding is confusing. My response is
that this is confusing only if you don't take the context in which the
discusssion appears into account. And this is something I am not able to
fix -- if you take something out of context you're in the wrong, period.
> Here's a clear statement:
> 1. For maximum interoperability of text composed by the user in an
> automatically wrapping text editor, send text/plain with short, hard-wrapped
> lines. Wrap at something less that 80 characters (72 or 76 are good choices)
> to allow for quoting on reply. If possible, send as 7-bit. If your data
> contains 8-bit characters, you'll probably have to provide a user option about
> whether they want to encode as QP in this case or "just send 8-bit".
This is not a statement that is appropriate to put in MIME. MIME has no
business saying what constitutes maximum interoperability of text. The closest
MIME comes to this is it's discussion of transport limitations and what should
be done to assure maximum interoperability across transports. This is entirely
within MIME's province as it is a specification of an on-wire format.
In fact you seem to be missing the entire point with the prose you have
proposed. It is perfectly permissible to have long lines in text/plain material
-- if you want long lines to appear at the other end that's exactly what you
you should be putting in there! The problem is that this is NOT the effect that
Microsoft wanted: They wanted those lines to be wrapped. Their error consisted
of assuming that quoted-printable encoding would in some way accomplish this
goal. It won't.
> 2. If you want to send automatically wrapped text, use text/enriched, encoded
> using QP.
> If you're a vendor of MIME-capable software, support for text/enriched is a
> must.
Such a normative reference to another standard that's behind MIME on the
standards track is entirely out of order and cannot appear.
I'll think about adding a note about not using QP as a means to control the
display of material on MIME readers. I object to it on principle and think
it's unnecessary, but I'll think about it adding it.
Ned
John W. Noerenberg
>
>1. For maximum interoperability of text composed by the user in an
>automatically wrapping text editor, send text/plain with short, hard-wrapped
>lines. Wrap at something less that 80 characters (72 or 76 are good choices)
>to allow for quoting on reply. If possible, send as 7-bit. If your data
>contains 8-bit characters, you'll probably have to provide a user option about
>whether they want to encode as QP in this case or "just send 8-bit".
>
>using QP.
>
>If you're a vendor of MIME-capable software, support for text/enriched is a
>must.
>
Bravo! Well said, Terry.
Barton E. Schaefer
} Subject: Re: Line Wrapping Question
}
} readers.
And I'm probably the single largest contributor to the same support in
Z-Mail, though with greatest emphasis on the UNIX side of the UI.
} Microsoft is generating messages with single-line-paragraphs with
} content-type: text/plain and content-transfer-encoding: quoted-printable,
} such that the on-the-wire (encoded) message doesn't have long lines (thus
} not technically violating any RFCs) but which, when decoded, would become
} unreadable on a system which didn't wrap lines.
}
} Netscape 2.0 is such a system: when confronted with long lines, Netscape
} puts up a horizontal scrollbar.
Z-Mail makes this optional whenever possible; the character-terminal
version (Lite) can toggle between line wrapping or horizontal scrolling
at display time; the Motif version has an X resource for same (and allows
stretching the window as well, of course). The Mac and Windows versions
always wrap, but allow streching the window.
(This is on message receipt, of course; the following discusses
composition/sending.)
} I believe that we've got a good, interoperable approach in Netscape 2.0,
} so I'll describe that. I'd be interested in any comments/criticisms of
} it.
It's remarkably similar to the approach that has been working well in
Z-Mail for years. However, Z-Mail is a bit less consistent accross
platforms about the details than is Netscape.
} So, the Netscape composition window takes the following approach: let the
} platform-specific text editor do display-time word-wrapping. If a paragraph
} is typed, and then a word is inserted in the middle of that paragraph, the
} rest of the paragraph should re-wrap. Hitting return is a "hard" line break
} that will never be auto-filled.
Z-Mail provides a run-time toggle to switch between this behavior and
explicit carriage returns after each line. In the UNIX versions, the
UI actually changes behavior to reflect this switch; on Mac and Windows
the toggle has an affect only at send time.
} However, when the time comes to deliver the message, Netscape takes all of
} the "implicit" line breaks (where words have wrapped because they reached the
} right edge of the window) and turns them into "explicit" line breaks.
This is what Z-Mail does when automatic word-wrapping is selected.
When automatic wrapping is not selected, it does no extra processing.
} There are several problems with this.
}
} The first is that, when quoting messages using the conventional USENET style
} (to set off each quoted line by preceeding it with "> ") the line lengths
} tend to get longer with each subsequent quote.
}
} > All work and no play makes Jack a dull boy. All work and no play makes
} Jack a
}
} It would be horrible were Netscape to send out messages that looked like
} this. Therefore, there is an additional hack in the formatting code which
} is that we never wrap lines which have ">" as their first character.
I'm curious how you accomplished this in the Motif text widget. Or
do you apply this only after the fact?
Z-Mail does occasionally send out messages that look bad in the way
you describe. Netscape won't do very well with this message, for
example, because I don't use ">" to indent quotations. (I found a
long time ago that I got mis-attributed less this way.) Z-Mail does
provide an explicit edit-time formatting operation that identifies
and strips off leading indentation, re-fills the text at a narrower
margin, and then puts back the indentation.
} Now, the next problem with this is that we wrap lines based on where the
} system's built-in text editor has chosen to do display-time word-wrapping.
} This, in turn, is based on the size of the window.
An additional complication with this for Z-Mail is that the user may
select a proportional font for the editor. At send time, howver, we
do all line wrapping based character counts, not extents. This means
that the editor is not quite WYSIWYG.
This sort of thing gets even worse when you get into Asian languages
like Chinese or Japanese.
} We create all composition windows 72 characters wide, on all platforms.
} [...] The problem is this: if the user drags the window wider, we will
} then wrap at the current width of the window [...]
} One solution to this, which we rejected, would be to disallow the window from
} being made wider than 79 columns. That would be bad, because there are
} situations when it is necessary and appropriate to send messages with long
} lines: when formatting a table, for example.
Z-Mail gives the user an option for the character position at which
wrapping should take place; this determines how wide the window should
be. UNIX and Windows versions then fix the window at that size; the
Mac version permits the window to be wider or narrower, but always
re-fills to the preset wrap position at send.
Long lines can be entered by toggling off word wrapping; we provide an
edit-time fill-paragraph action that inserts hard breaks at the wrap
position, in case you want to mix filled text and long lines.
} Another attractive and often-suggested solution is to cause the editor to
} always do word-wrapping somewhere before 79 columns, regardless of the width
} of the window.
Our users ask for this all the time, too; we haven't done it yet mostly
for the same reasons you listed.
} A simpler version of this hypothetical "more sophisticated" editor would be
} one which inserted "hard" newlines when the user typed a word which passed
} the right edge. In this editor, typing a paragraph and then inserting a word
} in the middle of it would leave the paragraph with ragged margins. It would
} then need to provide a "refill paragraph" command to discard and recompute
} the line breaks of that paragraph.
We also rejected this solution, though we went ahead and provided the
"refill paragraph" action.
} Note also, that all of these problems/solutions apply only to the case
} of generating messages of type text/plain, which is all that the Netscape
} mail composition window can directly do at this time.
One additional problem that we frequently encounter is that people
sending messages to recipients who have older mail clients tend to
do things like load uuencoded text directly into the text/plain
message body. Then they send it without turning off word wrap, and
if there happen to be any space characters in the uuencoded block
(Sun's uuencode is notorious this way) ... well, you get the idea.
--
Bart Schaefer Vice President, Technology, Z-Code Software
scha...@z-code.com Division of NCD Software Corporation
http://www.well.com/www/barts
Larry Masinter
> must.
Sorry to ask a stupid question, but why not use text/html instead of
text/enriched?
Lennart Lovstrand
> However, the point you seem to be consistently missing is that my position
> is not based on my own biases as to how things should work.
YES THEY ARE! Oh dear, now I've done it. I've been trying for the longest
time to avoid getting involved in an argument that I don't think will lead
anywhere, but I can't take it anymore. I've been working on email software
for 10+ years and a (mostly lurking) member of the ietf-822 mailing list for
four of them. What really irks me is when certain members of this group take
on this "I know better than you" attitude and swiftly dismiss other peoples'
objections or ideas because they don't match with their own views of the
world. I'm sorry, but I don't believe that you -- or anyone else, myself
included -- have a guaranteed hotline to the Truth and The One Way Things
Really Work. This state is commonly known as hubris, and usually leads to a
rude awakening down the line.
> My intent is to state how the *standards* *say* things *are* *required* to
> work and that software agents exist that operate on the assumption that the
> standards are to be obeyed in this regard.
Who makes the standards? I think your name is on the top of the MIME spec.
Does that make you an impartial observer? I don't think so.
As for what this particular standard specifies, I haven't seen any
statements (your quoted ones included) that has made me believe that
transmitting paragraph-length lines was a bad thing and against the spec. On
the contrary, by their very existence I've always thought of Q-P's "soft"
line breaks as a convenient way to let non-MIME recipients get a message that
would be readable on dumb terminals, while MIME UAs would rejoin the lines
and presumably present them in a reasonable way. I have no illusions of Q-P
implying a particular presentation, but until now, I did believe that any
modern MIME-capable mail UA would have automatic line breaking built in. I
also had no idea that this could be a controversial issue. Regrettably, I
was apparently mistaken on both counts.
All I'm trying to say is that I too -- in good faith, I assure you -- went
along and used Q-P as a way to encode paragraphs in NeXT's Mail 3.3 program.
Is this bad? I wouldn't had thought so, but apparently we disagree on that.
What should be done? Well, it looks like we have three choices:
1. Explicitly disallow (or at least recommend against) sending
paragraph-long lines with text/plain.
2. Explicitly allow long lines with a recommendation that they may have to
be wrapped for a reasonable presentation.
3. Find some other way to represent paragraph oriented text.
Simply saying that all text should be "hardwrapped" I don't see as an option
since it's akin to throwing out the baby with the bathwater. Some of you
may not care much about it, but after having lived for many years with
different email systems that do handle this (eg. Xerox Lafite, Andrew, NeXT
Mail), I have no intention of going back to hardwrapped text.
Using text/enriched instead of text/plain like you suggested earlier is
indeed an option, although it has its own disadvantages that makes it
arguably less attractive for non-MIME recipients (ie. the doubling of hard
newlines and less-than signs). I also fear that it is so little supported
that few recipients will ever see the benefits of the intended presentation.
Yes, if their MIME UAs are properly implemented, the body should at least be
treated as text/plain, but that would be as bad as not having MIME at all.
I frankly don't think much of the future of text/enriched, but I hope that
text/html will soon become popular enough for most of these "Luddite" (or, if
you prefer, "legacy") problems to go away. Still, to get there we first
need an agreement of how to represent the external parts of html, ie. images
and links. I have seen a lot of discussion on the topic, but little attempt
at finding a consensus. Perhaps we will have to wait until some strong party
goes ahead and simply makes a convincing implementation, and then swallow
that wholemeal. I have to admit that I don't really care much about how it's
done, but I do want it.
In the mean time, I'll look into what it would mean for us to switch our
implementation from using long Q-P-encoded text/plain lines to "plain"
text/enriched. We already support both sending and receiving text/enriched,
so it won't be a very long step to take. I only hope that other MIME UAs
will treat it reasonably if we start using it more.
Finally, please forgive me for being a bit moralizing for a moment.
Progress is a painful process, but as long as we can be civil to each other,
there is hope for the future. In this particular case, I saw someone from
Microsoft post a perfectly reasonable question only to have his head chopped
off in response. I don't think it matters who does it, it's just as rude
every time. Worse, I fear that I may be the victim next time. This makes me
less willing to participate in a public forum. This, I believe, is
unfortunate. Do you agree?
Regards,
Lennart Lovstrand
NeXT Software Engineering
(BTW, the Content-Transfer-Encoding: header below in the message I received
is broken. I don't know if it was originally created that way or if it got
damaged on the way, but I'm assuming that someone want to know about it.)
Message-Id: <01I0XK2QD...@INNOSOFT.COM>
Date: Wed, 07 Feb 1996 12:28:20 -0800 (PST)
From: Ned Freed <N...@innosoft.com>
To: Valdis.K...@vt.edu
Cc: "Sukvinder Singh Gill (Exchange)" <su...@wspu.MICROSOFT.com>,
"ietf...@list.cren.net" <ietf...@list.cren.net>
Subject: Re: Line Wrapping Question
Mime-Version: 1.0
Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
Content-Transfer-Encoding: 7BIT
<c=US%a=_%p=Microsoft%l=DABONE9602070...@yuri.microsoft.com>
<c=US%a=_%p=Microsoft%l=DABONE9602070...@yuri.microsoft.com>
[...]
Larry Masinter
> need an agreement of how to represent the external parts of html, ie. images
> and links. I have seen a lot of discussion on the topic, but little attempt
> at finding a consensus.
There's a group that's been discussing this; see, for example,
http://www.ccs.org/validate/wwwspec6.htmlm and
draft-palme-text-html-00.txt in your friendly internet drafts group.
In addition, the mimesgml group catalog proposal seems like it might
be a solution to some of the problems.
Ned Freed
Larry, it is far from a stupid question, as I'm sure you already know. There
are lots of issues surrounding what approach is best for formatted text
in email, and I for one would welcome a discussion of them.
Ned
Pete Resnick
>> If you're a vendor of MIME-capable software, support for text/enriched is a
>> must.
>
>Sorry to ask a stupid question, but why not use text/html instead of
>text/enriched?
There's two questions here: Why not generate text/html instead and why not
interpret text/html instead?
The answer to the first is easy:
1. Lots of mail packages now interpret text/enriched as a matter of course,
even though some simply unwrap the lines, remove the directives in '<>',
and get rid of the doubled '<' characters. In any event, that means that
you can pretty safely send text/enriched without too many users yelping as
loadly about text/enriched directives in the way they yelled about '=' in
QP. With text/html, most mail packages nowadays will just dump everything
to the screen since the only behavior for unknown text subtypes is to treat
it like text/plain. Users become pissy.
2. Text/enriched has some advantages over HTML, at insofar as RFC 1866 is
concerned. First, it also has no requirement for a <p> or <br> directive to
introduce a hard line break, which means that you can generate a nice plain
text message with no text/enriched markup at all and have it look fine on a
non-text/enriched viewer. Also, it has straight presentation markup, like
an <underline> directive and paragraph indentation, which e-mail users find
important; <blockquote> is not necessarily the markup you want when
something is indented. Newer HTML may address this, but that's not quite
here yet.
***HOWEVER***
Even *I* know that generating text/enriched is only a stop-gap. HTML is
coming at us like a fast moving train and we're going to have to deal with
it (and probably generate it) in the long run. So, the answer to the second
question is a rephrasing of Terry's original claim:
If you're a vendor of MIME-capable software, support for interpreting
text/html is must. Get started now. We certainly are.
Ned Freed
> > However, the point you seem to be consistently missing is that my position
> > is not based on my own biases as to how things should work.
> YES THEY ARE! Oh dear, now I've done it. I've been trying for the longest
> time to avoid getting involved in an argument that I don't think will lead
> anywhere, but I can't take it anymore. I've been working on email software
> for 10+ years and a (mostly lurking) member of the ietf-822 mailing list for
> four of them. What really irks me is when certain members of this group take
> on this "I know better than you" attitude and swiftly dismiss other peoples'
> objections or ideas because they don't match with their own views of the
> world. I'm sorry, but I don't believe that you -- or anyone else, myself
> included -- have a guaranteed hotline to the Truth and The One Way Things
> Really Work. This state is commonly known as hubris, and usually leads to a
> rude awakening down the line.
Geez. Did I say anything like this? I don't think so... The only "suggestion"
that has been made here as far as I know was to add some prose to the MIME
specification. I did object to the prose, but I certainly did not dismiss it
out of hand. On the contrary, I spent considerable time looking into it and
formulating a detailed response explaining why I thought it was not appropriate
to include the prose in the MIME document.
I try to take all suggestions for changes to MIME seriously and answer
them fully and completely. If I screw up sometimes I certainly apologize
for it.
> > My intent is to state how the *standards* *say* things *are* *required* to
> > work and that software agents exist that operate on the assumption that the
> > standards are to be obeyed in this regard.
> Who makes the standards? I think your name is on the top of the MIME spec.
> Does that make you an impartial observer? I don't think so.
Of course I'm not an impartial observer. Nor did I ever claim to be one. My
participation in this discussion makes it obvious that I'm nothing of the sort.
If I were an impartial observer by definition I would not be a participant.
> As for what this particular standard specifies, I haven't seen any
> statements (your quoted ones included) that has made me believe that
> transmitting paragraph-length lines was a bad thing and against the spec.
I certainly hope not, because it is not a bad thing and it is entirely in
accordance with the specification to do it. My objection to the suggested prose
was exactly this, in fact -- the suggested prose recommended banning (or at
least deprecating) doing things like this. Many systems currently depend on the
ability to send long lines in text and have them arrive intact.
You are absolutely allowed to transmit long lines of text if you want to do so.
The specification specifically allows for it. The problem lies in how receiving
agents are expected to handle such material. Microsoft apparently assumed that
such material would be wrapped automatically, and also assumed that using
quoted-printable would in some way provide a guarantee of this. And such
assumptions are contrary to the MIME specification.
> On
> the contrary, by their very existence I've always thought of Q-P's "soft"
> line breaks as a convenient way to let non-MIME recipients get a message that
> would be readable on dumb terminals, while MIME UAs would rejoin the lines
> and presumably present them in a reasonable way. I have no illusions of Q-P
> implying a particular presentation, but until now, I did believe that any
> modern MIME-capable mail UA would have automatic line breaking built in. I
> also had no idea that this could be a controversial issue. Regrettably, I
> was apparently mistaken on both counts.
Misreading the specifications is one thing. Having expectations that have
nothing whatsoever to back them up in the specification is a different
situation. There is nothing in the MIME conformance document that says that
line wrapping services must be provided by receiving user agents.
> All I'm trying to say is that I too -- in good faith, I assure you -- went
> along and used Q-P as a way to encode paragraphs in NeXT's Mail 3.3 program.
Using QP is perfectly fine, and I applaud you for having used it. Expecting
MIME-capable agents to have line wrapping facilities available is a different
(and separate) matter.
> Is this bad? I wouldn't had thought so, but apparently we disagree on that.
I'm not sure about that. If the material was properly formatted underneath the
QP for whatever agent you expected to send to I see no problems with your
approach. If, on the other hand, you expected all agents in the world to have
the ability to line wrap, a capability nowhere required by MIME, I would have
to say that your expectations were a overly hopeful. And if you expected QP
encoding to somehow imply to MIME agents that line wrapping should be done,
then I would say that your usage is contrary to the specifications.
> What should be done? Well, it looks like we have three choices:
> 1. Explicitly disallow (or at least recommend against) sending
> paragraph-long lines with text/plain.
This is a bit simplistic. What if I want to have really long lines in the
material I send? How am I supposed to accomplish this in the face of such a
recommendation?
The point here is not to send material that isn't suitable for display "as-is".
Not to avoid long lines altogether. Not to ban such usage outright.
> 2. Explicitly allow long lines with a recommendation that they may have to
> be wrapped for a reasonable presentation.
What consitutes "reasonable presentation"? I deal with material all the time
that cannot tolerate being line-wrapped, and I suspect lots of other people do
as well.
If you want lines wrapped you can either use text/plain and do it yourself or
use a format that let's you tell the receiver when it's appropriate to do it.
> 3. Find some other way to represent paragraph oriented text.
Presumably by using one of the other formats we have already defined for
this purpose.
> Simply saying that all text should be "hardwrapped" I don't see as an option
> since it's akin to throwing out the baby with the bathwater. Some of you
> may not care much about it, but after having lived for many years with
> different email systems that do handle this (eg. Xerox Lafite, Andrew, NeXT
> Mail), I have no intention of going back to hardwrapped text.
Nor do I. It's probably obvious from the messages I send that the agent I use
does line wrapping for me. I certainly don't bother to do the wrapping
manually!
> Using text/enriched instead of text/plain like you suggested earlier is
> indeed an option, although it has its own disadvantages that makes it
> arguably less attractive for non-MIME recipients (ie. the doubling of hard
> newlines and less-than signs). I also fear that it is so little supported
> that few recipients will ever see the benefits of the intended presentation.
> Yes, if their MIME UAs are properly implemented, the body should at least be
> treated as text/plain, but that would be as bad as not having MIME at all.
Speaking as an early proponent of text/richtext back in the days when Nathaniel
and I fought to keep it in the MIME specification (and eventually lost), I must
say I used to be very disappointed as what I saw as an outright rejection by
most developers of text/enriched. But recently the good folks at Qualcomm and
Intercon have opened my eyes: There is an amazing amount of support for
text/enriched out there already. Far from being rejected, it actually has
caught on in a fairly major way. It just took longer than we expected, and it
also happened without a lot of trumpets and sundry fanfare.
> I frankly don't think much of the future of text/enriched, but I hope that
> text/html will soon become popular enough for most of these "Luddite" (or, if
> you prefer, "legacy") problems to go away.
I don't think the text/enriched proponents disagree with you, as Pete Resnick
has already illustrated in his recent posting. text/enriched is primarily seen
as a step on the road to text/html in email. Pete's discussion of the
various pros and cons was so good that I see no reason to repeat any of
it here.
> Finally, please forgive me for being a bit moralizing for a moment.
> Progress is a painful process, but as long as we can be civil to each other,
> there is hope for the future. In this particular case, I saw someone from
> Microsoft post a perfectly reasonable question only to have his head chopped
> off in response.
I admit to considerable frustration about this, as I believe I pointed out in
my original response. I have been over this ground with Microsoft both directly
and indirectly through various Exchange beta sites a large number of times now,
and up until now I've gotten nowhere. I'm sure I was a lot nicer the first half
a dozen times it came up in other forums.
I don't think it matters who does it, it's just as rude
> every time. Worse, I fear that I may be the victim next time. This makes me
> less willing to participate in a public forum. This, I believe, is
> unfortunate. Do you agree?
Well, it now seems that the folks at Microsoft might actually do something
about this. So maybe "chopping their heads off", to use your phrase, was a good
idea after all ;-)
> (BTW, the Content-Transfer-Encoding: header below in the message I received
> is broken. I don't know if it was originally created that way or if it got
> damaged on the way, but I'm assuming that someone want to know about it.)
I checked -- it is not what left my system nor was it corrupted in what I
received from the list. I don't know where the problem arose for you, but I
didn't see any sign of it here.
Ned
Terry Crowley
> text/enriched?
Larry,
Here's my take on it. Millions of people are now using mail composition and
reading tools that support automatically wrapped text with very simple markup
(usually fonts and color, sometimes simple margins, alignment). Text/enriched
is a good representation for that content and presentation material (personally
I wish it had been less schizoid with respect to procedural/semantic markup,
ie. had accepted that what was needed with precise procedural markup and had
actually provided precise margins, tabs, etc, but that's water under the
bridge).
text/html provides no special benefits for this type of content and is
significantly more complex to parse and layout. It's therefore much less
likely that all mail readers/gateways etc will do something "useful" with it
(that is, convert it to a readable plain text representation).
That said, text/html is obviously going to be a major content type in MIME
mail, and mail vendors would find it well worth it to make the presentation of
text/html natural and seamless. But I don't think it makes sense to take your
normal casual email note and convert it to text/html.
One other point: one of text/enriched's goals was to be an "acceptable" format,
even for non MIME mail readers. While text/enriched is readable, I think must
vendors have found it is not "acceptable". That is, if our customers are
sending mail to recipients without MIME software, they don't want to send
text/enriched, in the same way that they don't want to send QP - any oddities
make the sender look bad. That's too bad, because it means slowly growing the
community to be distributing text/enriched rather than text/plain (when
text/enriched is the appropriate form for the presentation and content) has
proven difficult. Not sure what to do about that, but if we keep on urging
vendors, including gateways, to support text/enriched, than eventually we'll
get to a point where it is acceptable to send since few recipients will
actually see any oddities.
Terry
Jim Conklin
got blocked by one of our loop-detection algorithms.
>Date: Fri, 9 Feb 1996 09:28:48 +0100
>To: <ietf...@list.cren.net>
>From: font...@sri.ucl.ac.be (Alain Fontaine)
>Subject: Re: Line Wrapping Question
>
>At 10:22 96/8/02, Ned Freed wrote:
>> text -- textual information. The subtype "plain" in particular
>>indicates plain
>> (unformatted) text. No special software is required to get the full
>>meaning of
>> the text, aside from support for the indicated character set.
>
>Maybe the confusion comes from the use of 'unformatted'. text/plain is
>not unformatted ; on the contrary, it is very rigidly formatted by the
>line breaks it contains.
>
> /AF
>
>
>
Ned Freed
> even for non MIME mail readers. While text/enriched is readable, I think must
> vendors have found it is not "acceptable". That is, if our customers are
> sending mail to recipients without MIME software, they don't want to send
> text/enriched, in the same way that they don't want to send QP - any oddities
> make the sender look bad. That's too bad, because it means slowly growing the
> community to be distributing text/enriched rather than text/plain (when
> text/enriched is the appropriate form for the presentation and content) has
> proven difficult. Not sure what to do about that, but if we keep on urging
> vendors, including gateways, to support text/enriched, than eventually we'll
> get to a point where it is acceptable to send since few recipients will
> actually see any oddities.
Part of the problem has been that some of the original programs generating
text/richtext and subsequently text/enriched did not pay any attention to how
their results looked on non-MIME-capable agents. And the results these agents
produced were pretty grotty, to put it mildly -- they were loaded with totally
unnecessary formatting options, line breaks were awful (I remember one that put
each word on a separate line), and so on. This has created an impression in
some quarters that text/enriched cannot be readable in and of itself, and this
really isn't true.
In many cases this wasn't really something these agents could control, as most
of them were inserts into existing backend output frameworks that were never
designed to cater to presentation of raw material containing formatting
commands.
Newer agents are much more careful about this and produce much more readable
output. In fact it may almost be too good -- I've been tricked into thinking I
was editing text/plain when in fact I was editing text/enriched, and I've seen
others fall into the same trap as well. (I guess there's always a downside...)
This doesn't mean that there are absolutely no problems with presenting
text/enriched on non-MIME-capable agents. There are. People will always find
things to complain about, like the customer who complained that a random
MIME-boundary generator had managed to produce something that looked vaguely
like profanity. And text/enriched is still a lot friendlier than text/html in
this regard.
Ned
Ned Freed
> got blocked by one of our loop-detection algorithms.
> > Maybe the confusion comes from the use of 'unformatted'. text/plain is
> > not unformatted ; on the contrary, it is very rigidly formatted by the
> > line breaks it contains.
As a matter of fact I noticed this myself late last night. You're absolutely
right -- "unformatted" here means "there are no formatting commands in the
text" and this is easily confused with "this isn't formatted now and needs to
be formatted before it is displayed".
I will change this in the next iteration of the specification and make the
meaning clearer.
Ned
Terry Crowley
<nofill> issue. Although these mail composers we've been talking about provide
for automatic wrapping, not everyone uses it. If they actually type the line
breaks themselves, their message comes out with a blank line separating every
line, which is annoying.
On the other hand, if they do that than the text/plain version might also have
ragged <long line>, <short line> pairs (if their lines are longer than the hard
break limit, which is easy to do with a proportional font and a wide window),
so maybe I've been too hard on text/enriched, when generated with as little
markup as possible.
A really smart system might notice whether text/plain or text/enriched would
result in a "prettier" uninterpreted output, and pick that. Ah, that might be
too smart, users also like predictability....
Terry
Terry Crowley
I played a significant role in getting richtext dropped from the original MIME
spec. While working on BBN/Slate, a multimedia mail system of CMU's Andrew's
ilk, I tried to implement text/richtext input/output converters. I found that
1) it had many Andrew-based predjudices in its text model that did not jibe
with the model in either Slate or your typical Mac/Windows simple text
processor and 2) there was an amazing amount of implied semantics that came out
of Andrew's handling of formatting that was not in any way part of the spec.
In addition, it had "the kitchen sink" including
headers/footers/footnotes/floating figures/keep with next, etc. that was
entirely inappropriate for a common denominator rich text format.
text/enriched is much more tightly specified and much more constrained in the
kind of markup it is trying to provide.
Terry
Dirk Fieldhouse
>...>
>
>The problem is this: if the user drags the window wider, we will then wrap at
>
>One solution to this, which we rejected, would be to disallow the window from
>being made wider than 79 columns. That would be bad, because there are
>situations when it is necessary and appropriate to send messages with long
>lines: when formatting a table, for example.
>
>window is too wide (probably just before the message is sent) which offered
>to shrink it and allow the paragraphs to refill. We may yet do this, but
>rejected it initially as too intrusive and annoying.
As a matter of interest, the WinVN freeware Newsreader does do this and it is
not annoying to my mind. The check appears when the Post/Send Article option
is selected, not before. It advises you to resize the window and you can then
avoid the reappearance of the dialog by saving the window positions.
Implementation hints: make sure the dialog box explains the problem clearly -
this to get through to the sort of user who would treat your guideline rule as
a bug - and ideally (unlike WinVN) have a button that resizes the composition
window to an acceptable width. You also need a facility to set the initial
composition window width permanently.
Thus the options when posting with >80 (78?) column text should be:
- OK: post anyway with my non-standard line length
- Cancel: let me think about the width and perhaps resize the window
myself
- Resize: resize the composition window to wrap at column n (default
n=76, could be a control to allow values 72<=n<=80)
- and a sub-option on Resize to set the default width.
--
Dirk Fieldhouse Logica UK Limited
field...@logica.com 75 Hampstead Road
c=gb;a=tmailuk;p=logica; London NW1 2PL
o=lg;ou1=lgwct;s=fieldhouse UK
+44 (171) 637 9111
Lennart Lovstrand
> ilk, I tried to implement text/richtext input/output converters. I found
> that 1) it had many Andrew-based predjudices in its text model that did
> not jibe with the model in either Slate or your typical Mac/Windows simple
> text processor and 2) there was an amazing amount of implied semantics
> that came out of Andrew's handling of formatting that was not in any way
> part of the spec.
> text/enriched is much more tightly specified and much more constrained in
> the kind of markup it is trying to provide.
Amen. I had exactly the same reflection when writing the text/richtext and
text/enriched converters for NeXT's (RTF-based) mail UA. The updated
text/enriched spec is much clearer and one thing I'm particularly happy to
see explicitly stated is the implied line breaks around paragraph formatting
commands like <center>.
<nofill> still seems a bit loose, though. In particular, it's not clear to
me exactly what the extent of the affected text really is supposed to be.
>From the draft-03 spec, it sounds like it will begin immediately after the
right bracket in "<nofill>" and continue to immediately before the left
bracket in "</nofill>". However, this means that an example like this:
--------------------------------
<nofill>
aaa
bbb
</nofill>
<nofill>
xxx
yyy
</nofill>
--------------------------------
will generate the somewhat surprising:
--------------------------------
aaa
bbb
xxx
yyy
--------------------------------
since each nofill block will include the CRLF just after <nofill>, the one
just before </nofill>, and the double CRLF between the two blocks will
generate an extra CRLF too.
HTML's <pre> command has a special rule for this. It dictates that any
directly adjoining newline to the <pre> command is to be excluded from the
affected text. With that rule in effect, you'd get this instead:
--------------------------------
aaa
bbb
xxx
yyy
--------------------------------
Which is more in tune with the other commands. For example, if you have:
--------------------------------
<center>
aaa
bbb
</center>
<center>
xxx
yyy
</center>
--------------------------------
you currently get:
--------------------------------
aaa bbb
xxx yyy
--------------------------------
if I read the spec right.
On a related topic, the rule that makes a single CRLFs turn into a space
seems a bit simplistic. For example, this:
--------------------------------
first
<flushleft>
<bold>
second
</bold>
</flushleft>
third
--------------------------------
will generate this:
--------------------------------
first
second
third
--------------------------------
That is, there will be an extra space before both "second" and "third" and
maybe one after "first" and "second" too. It would probably be better to
borrow an idea from the paragraph commands and say something like "a single
CRLF will cause a space to be produced unless it would mean that it would be
generated immediately next to another space or newline".
My apologies if I misread something in the spec.
--Lennart
Pete Resnick
>Amen. I had exactly the same reflection when writing the text/richtext and
>text/enriched converters for NeXT's (RTF-based) mail UA. The updated
>text/enriched spec is much clearer and one thing I'm particularly happy to
>see explicitly stated is the implied line breaks around paragraph formatting
>commands like <center>.
Thanks. Our main purpose was to clarify some of the things that were simply
underspecified. I'm glad you like it.
><nofill> still seems a bit loose, though. In particular, it's not clear to
>me exactly what the extent of the affected text really is supposed to be.
>From the draft-03 spec, it sounds like it will begin immediately after the
>right bracket in "<nofill>" and continue to immediately before the left
>bracket in "</nofill>".
That is exactly correct.
>However, this means that an example like this:
[...]
>since each nofill block will include the CRLF just after <nofill>, the one
>just before </nofill>, and the double CRLF between the two blocks will
>generate an extra CRLF too.
Yes, it does. However, I don't think we want to play around with this too
much. First, there are a good deal of parsers out there that do this now,
and it's probably not nice to legislate them into illegitimacy. Second, and
more importantly, the idea is that this is supposed to be a simple little
formatting language. The more exceptions we put in, the higher the
complexity. It's probably not optimal, but then again, it's not so bad.
>HTML's <pre> command has a special rule for this. It dictates that any
>directly adjoining newline to the <pre> command is to be excluded from the
>affected text.
Yes, that would be nice. But there are lots of those kinds of things that I
would not necessarily want to impose on someone writing a text/enriched
parser. Remember, it would be nice if this thing went away eventually.
> For example, if you have:
>
> --------------------------------
> <center>
> aaa
> bbb
> </center>
>
> <center>
> xxx
> yyy
> </center>
> --------------------------------
>
>you currently get:
>
> --------------------------------
> aaa bbb
>
> xxx yyy
> --------------------------------
>
>if I read the spec right.
Actually, as you have written it, you should get an extra space before the
first a and x, as well as extra spaces after the final b and y. This is
because the CRLF after and before each formatting command "counts".
>On a related topic, the rule that makes a single CRLFs turn into a space
>seems a bit simplistic. For example, this:
>
> --------------------------------
> first
> <flushleft>
> <bold>
> second
> </bold>
> </flushleft>
> third
> --------------------------------
>
>will generate this:
>
> --------------------------------
> first
> second
> third
> --------------------------------
>
>That is, there will be an extra space before both "second" and "third" and
>maybe one after "first" and "second" too.
Actually, I count two spaces after "second".
>It would probably be better to
>borrow an idea from the paragraph commands and say something like "a single
>CRLF will cause a space to be produced unless it would mean that it would be
>generated immediately next to another space or newline".
Again, we're increasing complexity. It's not that I don't think that these
would be nice ideas; it's just that we were looking to change as little as
possible and effect installed base as little as possible too.