user-editable mail headers

[Greg Skinner]

Is there any interest in Unix Mailers which support user-editable headers
like tops20 MM supports them? The new exptools version of Mail doesn't
seem to support them and I don't know of any other Unix Mailers offhand
that do.

Note that I am not talking about editing just the to, from, cc, bcc and
subject fields ... I am talking about putting anything in your header which
RFC822 will parse.
--
Hug me till you drug me, honey!

Greg Skinner (gregbo)
{allegra,cbosgd,ihnp4}!hou2e!gregbo

[bamford]

Y E S ! ! ! I am very interested in a well supported mailer program
that allows editing of the headers. I very much liked what the TOPS20
machine at Stanford had.

[Steven Bellovin]

Please note that the Rand MH system -- distributed with 4.2bsd -- has had
that ability for years. When you want to send a letter, it pops you into
the editor of your choice; when you're done, it parses the header to see
who it's addressed to.

[Ben Cutler]

Is there any interest in Unix Mailers which support user-editable headers
like tops20 MM supports them? The new exptools version of Mail doesn't
seem to support them and I don't know of any other Unix Mailers offhand
that do.

Note that I am not talking about editing just the to, from, cc, bcc and
subject fields ... I am talking about putting anything in your header which
RFC822 will parse.

I have written a portable mailer called AZ. It's very similar to Tops-20 OZ
written by John Ellis. AZ runs on Apollos, and Unix and VMS Vaxen at Yale and
at several other sites. You can edit the basic header fields (To, Cc, Bcc,
Subject) directly using simple commands or using a full screen editor (if you
have one). AZ doesn't support user-editing of the wide variety of possible
header fields, but this capability would be very easy to add.

Ben Cutler
Cut...@YALE.ARPA
decvax!yale!multiflow!cutler


-------

[Erik E. Fair]

The recmail program from the netnews distribution does similar things.
Standard procedure is to make a prototype header for the user, fire
up an editor, and when it exits, hand the text to recmail. It picks out
the To: and Cc: fields and mails off the letter with /bin/mail, which
will accept *anything* as input. Voila! Editable headers.

Erik E. Fair ucbvax!fair fa...@ucb-arpa.ARPA

dual!fa...@BERKELEY.ARPA
{ihnp4,ucbvax,hplabs,decwrl,cbosgd,sun,nsc,apple,pyramid}!dual!fair
Dual Systems Corporation, Berkeley, California

[Henry Spencer]

I have no direct experience with MH, but this is *clearly* the right way
to do letter composition. A mail system is a mail system: it should not
try to be a text editor as well. Partly because it complicates the mail
system unnecessarily, partly because it means the poor user has to learn
two different editors, but mostly because mail-system editors are lousy
editors! Text editing is a complex task; writing a good text editor is
not easy. Mail-system authors should leave this job to specialists.
--
Henry Spencer @ U of Toronto Zoology
{allegra,ihnp4,linus,decvax}!utzoo!henry

[Stephen Wolff]

Can anyone explicate an honest reason for editing the From: line of a
message? It is certainly not possible in our mailer without exercising
privilege, and I had blithely assumed that to be universally the case.

I should hate to have to regard every message as a potential forgery.

[Mark Crispin]

The TOPS-20 MM mailsystem allows editing the From: line of a message.
This is to support some of the hairier aspects of office automation;
e.g. models where some other person (e.g. a secretary) sends mail for
another. MM also has the capability to be run in a mode where you
can "become" another user for reading/sending mail, but this needs
access to that user's directory (MM will ask for the user's password
if necessary to obtain that access).

The security problems with forged mail are there to be sure, BUT...
unlike TOPS-10 or Unix, programs on TOPS-20 NEVER run with "temporary"
privileges. It is therefore impossible to enforce any security at the
level of a program run by users, because any security enforcement done
by such a program can be evaded simply by building one's private copy
of the program without the check. Even leaving aside the issue of a
patched copy of the program, you must also consider the possibility of
other message composition programs being developed to run concurrantly
on the same system; whether to evade the security check or simply to
experiment with alternative interfaces.

That leaves security enforcement to a privileged agent, that is, the
mailer. The problem here is that at the present time the TOPS-20
mailer doesn't have the slightest idea what RFC 822 is. It doesn't
deal at that level; it deals with SMTP. When RFC 1234 comes out that
completely changes the standards, not only would the message composition
program require changing but also the mailer. Ugh.

Personally, I believe that security against forged mail is a fantasy.
The best you can do is validate that a message clearly came from such-
and-such a host, or for locally-originated mail, that a message was
composed by a certain user (provided your operating system supports a
"file author" word that unprivileged users can't modify). In TOPS-20,
this is done via Received: and Mail-From: lines.
-------

[John R Ellis]

Personally, I believe that security against forged mail is a fantasy.
The best you can do is validate that a message clearly came from such-
and-such a host, or for locally-originated mail, that a message was

composed by a certain user.

It's quite easy to do much better than this for local networks, using
standard operating systems like TOPS-20 and Unix. At Yale, our Chaosnet
implementation provides a server with the user id and host of the program
at the other end of the connection. The operating systems provide this
information; user-state programs cannot forge it. (It isn't hard to modify
TOPS-20 and Unix implementations of Chaosnet to provide this capability.)
Thus our mail system knows reliably who sent local-network mail.

Of course, if someone broke into the operating systems, they could forge
the mail. So what? Computer people often talk about "security" as if
it were an all-or-nothing proposition. But as in the physical world,
there are varying degrees of computer security, depending on how much
the security is worth to you. Show me a particular computer security
method, and I'll show you a (possibly very expensive) way to circumvent
it (including non-electronic methods).

Just as most of us prefer moderately secure locks on the doors of our
homes in preference to no locks at all, most computer users would prefer
protection against easy forgery rather than no protection at all. I was
once told the government's sensible definition of security: Make it more
expensive for the spies to break security of your particular system than
it would cost them to achieve their goals by some other means.
-------

[G...@mit-xx.arpa]

From: Stephen Wolff <st...@BRL-BMD.ARPA>

I often use my nickname (gregbo) in the From: field and use my real
name in the Sender: field. It's not to be fraudulent, just to have a
little fun.

--gregbo
-------

[Kenneth Almquist]

I often use my nickname (gregbo) in the From: field and use my real
name in the Sender: field. It's not to be fraudulent, just to have a
little fun.

What you may not realize is that you are having fun in a way that
violates RFC 822. (I sometimes wonder whether anyone in Arpaland
has ever read that document.) RFC 822 requires that all addresses
contain at signs, as your mail program should know but apparently
doesn't. Therefore, when your letter arrived here, I had to manually
edit the header to replace
From: Grebo
with
From: G...@MIT-XX.ARPA
So nobody on the USENET side of the gateway got to see your little
joke.
Kenneth Almquist

[Frank J. Wancho]

Kenneth,

RFC822 says many things in different places. One of them says that
the Sender: field MUST appear if the From: field is not "correct".
Another says that the Reply-to: supercedes the From: field. So, in
spite of the "illegality" of the From: field, both the Sender: and the
Reply-to: fields were legal. Your complaint should have been made to
the maintainer of your mail handler.

--Frank

[Ron Natalie]

However, there is a difference between not being correct (having the
wrong or non-existant user name) than having the wrong syntax. People
who have illegally formatted header lines should be shot.

-Ron

[Mark Crispin]

The command which GDS uses to generate "From: Gregbo" is a power
tool in MM and not a result of any failure in it. There are
other commands which can be used to "become" some other user which
do ensure valid RFC 822 header lines. A "power tool" is something
which lets the user go beyond the program's (narrow) interpretation
of whatever standard...either to use some unimplemented or new
feature...or in this case to violate it.

Just like somebody can hurt themselves with a chain saw, one can
abuse a mailsystem power tool. But blame the user, not the tool.
-------

[Joseph I. Pallas]

"MM doesn't shoot RFC822, people shoot RFC822." Sorry, by no stretch of
the imagination can your "power tool" argument justify sending illegal
(I mean, invalid) headers out into world. MM can do whatever it wants,
but your SMTP mailer is responsible for supplying the safety glasses if
MM doesn't.

joe

[Kenneth Almquist]

Frank,
I disagree with the way you interpret standards. If you write a
program with a syntactically incorrect statement in it, would you
try to tell the compiler writer that the compiler should accept the
program anyway as long as the statement was not executed? It is true
that the message in question had a Sender: field which indicated the
real sender of the message and a Reply-to: field which indicated where
replies were to be sent to. The fact that the From: field was not
used for these purposes does not mean that it is OK for the From:
field to be syntactically incorrect.

I do not check the syntax of every entry in a mail header. However,
RFC 822 requires gateways to parse From: fields. In section 6.2.2 it
says:
This standard permits abbreviated domain specification....
When a message crosses a domain boundary, all addresses must
be specified in the full format, ending with the top-level
name-domain in the right-most field. It is the responsibility
of mail forwarding services to ensure that addresses conform
with this requirement. In the case of abbreviated addresses,
the relaying service must make the necessary expansions.

Presumably RFC 822 is intended to specify the format of ARPANET mail.
It follows that any mailer which sends out nonconforming messages
is broken. I should not be told to fix *my* software to deal with such
messages. I gather that the mailer on MIT-XX doesn't bother to check
the syntax of user specified From: lines on the grounds that users are
not supposed to make mistakes. That is not the mark of a quality piece
of software.
Kenneth Almquist

[Mark Crispin]

Whether or not software is "quality" or no depends upon whether
or not it does what it is intended to do. It is pretty silly to
expect Visicalc to execute PL/I programs.

The TOPS-20 mailer agrees to act as a mail bridge only, not as
a "mail gateway". It is the responsibility of the originating entity
to generate a message to the destination in correct format. Similar,
all entities should use the same standard for message headers. The
mailer (MMailr) knows nothing about RFC 822; its expertise is with
RFC 821 and other transport-level protocols. It is completely
separate from any message composition agent.

Any user agent which does not wish to take the responsibility
of composing a valid message for the destination agent should simply
not use a mail bridge. Most of the TOPS-20 mail bridges would be
happy to have less traffic; they provide it only as a courtesy and
not as a guaranteed service.
-------

[Ron Natalie]

Then you are saying that TOPS-20 users should either get smarter user
agents or not use MMailr. Sounds good, when is it likely to happen?

-Ron

[Greg Skinner]

From: Mark Crispin <M...@SU-SCORE.ARPA>

The command which GDS uses to generate "From: Gregbo" is a power
tool in MM and not a result of any failure in it. There are
other commands which can be used to "become" some other user which
do ensure valid RFC 822 header lines. A "power tool" is something
which lets the user go beyond the program's (narrow) interpretation
of whatever standard...either to use some unimplemented or new
feature...or in this case to violate it.

Now hold on! Just because I like to modify my headers, it doesn't
mean that I am violating the mailsystem! I've seen some really
ridiculous headers in my day, and mine is quite reasonable compared to
those!

Greg Skinner
G...@MIT-XX.ARPA
ihnp4!houxm!gregbo (UUCP)
-------

[Greg Skinner]

From: ihnp4!hou3c!ka@mit-eddie (Kenneth Almquist)

People on the more civilized side of ucbvax do read RFC 822.
I have read it also.
All you had to do was delete the From: line and take what was in the
Sender: line and manually replace it.
Since there is an authenticated Sender: field generated whenever a
From: field is munged, the message still stays within legality of
RFC822.
The only problem is readnews' and its brothers' inabilities to read
Arpanet mail. Simple fix to readnews -- just have the Sender: field
examined if the From: field is unparseable.
One more thing, in case you didn't know, I work at Bell Labs also, so
I have the experience of both ARPA and Usenet message composition.
Don't blame the user, blame the standards' (RFC850 and RFC822)
different interpretation of From:

Greg Skinner (I do read RFCs!)
G...@MIT-XX.ARPA
ihnp4!houxm!gregbo (UUCP)

p.s. perhaps mit-vax is at fault also!
-------

[Mark Crispin]

Well, in fact, I have started the specification for a
smarter user agent than MM. MM has reached the end of its useful
lifetime. There are other user agents of varying strengths and
weaknesses for TOPS-20 that interact with MMailr: Babyl and
Hermes come immediately to mind.

Many of the messages that people have complained about did
not, in fact, originate at a TOPS-20 system. Many of them did,
in fact, originate on a Unix system! You see, while there are
very good mailers on Unix there are very poor ones. Some of the
latter think it is alright to send a letter out on the Internet
simply by bouncing it to a TOPS-20 system after having appended
"@host" (where host is the name of the TOPS-20 system) to the
From: line.

I feel it violates all concepts of modularity to have the
mail delivery agent know about whatever is inside the message.
Part of the problem is that the Internet mail protocols still
insist upon having syntax-checking of this thing called a
"message header" inside the message, instead of having all that
stuff be external and/or obtained from the envelope at the
transport level where it belongs.
-------

[William C. Wells]

In reply to:

Date: Sat 25 Aug 84 09:12:21-PDT
From: Mark Crispin <M...@SU-SCORE.ARPA>
Subject: Re: user-editable mail headers

....

I feel it violates all concepts of modularity to have the
mail delivery agent know about whatever is inside the message.
Part of the problem is that the Internet mail protocols still
insist upon having syntax-checking of this thing called a
"message header" inside the message, instead of having all that
stuff be external and/or obtained from the envelope at the
transport level where it belongs.
-------

Lets be clear about whether you are talking about mail user agent
or mail transport agent functions. A message composed by a user should
have its headers validated before it is passed to a mail transport
agent to be transmitted. That is, the header should be full and
complete before it is passed to the envelope builder.

I concur that the envelope should contain the information it needs
to the job. For example, Precedence and Classification should be
part of the SMTP envelope. But since those fields are user defined
(in the DOD community), they first have to be in the "contents" header.

Within the same mail system, it should not be necessary for intermediate
mail agent program to check or modify the "contents" message header
(with the exception of adding the "Received" audit trail). However,
when the message is gatewayed into another mail system, format and
address conversion may need to be preformed.

I think some of the problems we have been having are a result of the
unofficial policy of making mail programs liberal in what they are
willing to receive and (hopefully) strict about what they transmit.

I would like to suggest that gateway mail agents enforce the rules.
If you are a gateway into the Internet mail community, you should have
a syntax and address checker between the Internet mail agent program
and the non-Internet mail system. Of course, if you want to be kind
to your non-Internet neighbor, you can put a conversion program
between the non-Internet mail system and the check program.

Bill Wells
wcw...@Berkeley.ARPA
ucbvax!wcwells
WCW...@UCBJADE.BITNET

[Mark Crispin]

Let's get some other things straight.

What is the point of a power tool to create arbitrary headers if some
process (be it composition or transport) decides to "correct" them?
It means that you cannot experiment with new header syntaxes without
writing code to do so.

I am greatly prejudiced against having mail transport agents (or even
mail bridges -- which some call "mail gateways") know about RFC 822.
Time and time again I have seen perfectly valid headers munged into
unrecognizability by such overly-"helpful" bridges.

Personally, I consider mail bridges in general to be a crock. If it
weren't for every trivial entity which strings up a network thinking
they have God's gift for designing the perfect network protocols, we
wouldn't have this plethoria of different network protocols that
require mail bridges.
-------

[John Gilmore]

Sendmail makes it relatively easy to build such mail interfaces.
It has an option (-t) which says, "Read the header and message from
standard input and decide who to send it to". This lets mh, vnews,
etc. avoid mesing with headers and just let them pass the whole message
to sendmail.

I believe it deals with the "authentication" issue by inserting a Sender:
field if the supplied From: field doesn't match the one it would have
provided (except for commentary). It probably doesn't deal with
duplicated fields (eg an existing Sender:, or some Received: lines).
It removes all Bcc: lines from the header after building the envelope.

There was a message from someone at IBM suggesting that you should be
able to edit a version of the message with groups expanded, then
for sending it should compress them out again? Why would you want to?

[Tommy_Er...@qzcom.mailnet]

(Text 66788) 84-08-22 11.39 GDS @ MIT-XX.ARPA
%Original date: Tue 21 Aug 84 19:59:48-EDT.
%FROM: Gregbo
%In-reply-to: Message from "Stephen Wolff <st...@BRL-BMD.ARPA>"
of Sat 18 Aug 84 22:12:45-EDT
%SMTP sender: @MIT-MULTICS.ARPA,@MIT-MC:G...@MIT-XX.ARPA

I often use my nickname (gregbo) in the From: field and use my real
name in the Sender: field. It's not to be fraudulent, just to have a
little fun.

In our COM computer conferencing system we want to retain as
much security and stringency as possible. Therefore we select
the sender name from the SMTP-sender field but adds a different
>From field right into the text just for user information.
I.E. with reliable (in some measure) mail agents consistency
is retained as far as possible.

Nicknames: we don't have'm. But we allow sort of fuzzy matching:
"t er q", "ericson qz" and "zeke" (just for fun) are valid local
parts.

To MRC: Yes, TOPS-20 does not allow user programs to get "temporary"
privs. But it is rather easy to install a monitor call that will
allow a certain program to access a certain file safe, we have
done this to ensure that only THE (execute-only) program may
access the message data base.

[g...@sun.uucp]

Ahem. A moment of reality, please. All this software was written
under time pressure and is still evolving. Precious little of it
was originally written or designed to do what it is currently doing.
Five years ago, few of these systems had even rudimentary capability to
hook up to foreign networks -- many did not know how to pass data with
another machine of their own kind!

Being in the "leading edge" of mail technology, being in the biggest
most advanced and interconnected networks in the world has its prices:
things in thousands of hosts scatterred all over the world do not always,
necessarily, perfectly, work right every time. I don't have time
to pop into sendmail every time someone reports a bug; my sendmail
bug folder has several hundred messages in it. But I regularly send
and receive mail with people in Europe and on corporate networks, which
I could not do two years ago.

If you have a problem with somebody's headers, please, tell us about it.
But one tortured scream, and maybe a rebuttal, should suffice.
We can hopefully expound our theories of "the way to run the mail network"
without getting our throats this hoarse.

Personally I have written many programs that would coredump when given
invalid input data, but I didn't write them as production programs, I
wrote them as hacks. In time of need, hacks are pressed into service
and fixed later. This is not a cause for alarm; the alternative to
Unreliable mail service is No mail service, which is the default when
you first apply power to your chips and can continue in force for quite
a long time if desired.

As soon as we find a reliable mail service we'll be sure to tell you,
so we can talk to you with it.

[Julian Davies]

Why don't all you ARPA people just make the change to X.400 protocols.
The rules for message envelopes are there clearly distinguished from
those for message contents. So as long as the envelopes are correct
(which is the function of the trusted mail transfer agents) people can
go and make strange message contents if they wish. The protocol even
has room for use of 'private use' types. Messages with strange
headers might not be handled by 'standard' user agents of course, but
presumably experimenters provide appropriate user agents for their
user communities.
My own interest is also in such experimentation, and using the
flexibility to try 'new media' messages for handicapped persons.
The ARPA RFC-xxx orientation to messages and envelopes all in ASCII
character sets strikes me as obsolescent.

Julian Davies
{deepthot|uwo}!julian

[Rick Lindsley]

Allowing a user to edit a From: line forces the mailer to do authentication;
tricky business at best. But since the From line is usually added by the
mailer and not the user anyway, this should not be a problem.

Rick Lindsley
richl@tektronix
..!{allegra,ihnp4,decvax}!tektronix!richl

[fai...@ucla-cs.uucp]

In the article I am commenting on, it was pointed out that TOPS-20
allows for editing of the From: line for the case where a secretary
might want to send mail in the name of (pronoun)'s boss.

You do not need to edit the ``From:'' line to do this. Arpa RFC822
supports the ``Sender:'' field to provide this functionality. Here's a
concrete example: in the mail distribution system I developed (Quadratron
System's Q-MAIL), you can send mail from another user's desk. (This
would be done by a secretary in such a situation). In this case, the
mail program is given the name of the user who the mail is ``from'';
this is used for the``From:'' line; the name associated with the current
uid is used as the sender.

As for non-ARPA systems, (or ARPA systems where you cannot change the
From line), I just put in a ``Sent-For:'' header item (ARPA: x-sent-for:)
which contains the appropriate information.

Daniel Faigin
Quadratron Systems, Encino
UCLA

{ucbvax,ihnp4,randvax,trwspp}!ucla-locus!faigin
{ucbvax,ihnp4,randvax,trwspp}!ucla-locus!quad1!faigin