Getting 2822 to Draft
Arnt Gulbrandsen
We went through this before. At the end, noone raised objections against
the insertion of a sentence such as "X-* header fields are not covered
in this document". Please, can we just insert that in the list of
changes and be done?
Quarrelling about whether X-* is useful, a mistake, neither or both is
not going to help 3822, so let's not do it.
--Arnt
Pete Resnick
On 1/2/04 at 10:02 AM -0500, Keith Moore wrote:
>why not post the revised ABNF as an I-D so that others can go over
>it with a fine tooth comb?
I sent mail to Bruce. The only copy of the grammar he wrote that I've
got has some MIME grammar mixed into it in a couple of places. I've
asked him to post one that is pure 2822 grammar.
pr
--
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102
Simon Josefsson
Pete Resnick <pres...@qualcomm.com> writes:
> Comments?
Would 2822bis obsolete 822 fully? If so, I believe it should mention
what happened to the X- header prefix that was part of 822. I don't
believe simply removing them without stating why is a good idea,
people can draw many different conclusions from that, which affect
real implementations.
Thanks,
Simon
Pete Resnick
So, for the new year I started thinking about getting 2822 to Draft.
(Also, 3822 is coming up in the RFC numbers :-) ). I've gotten
started on getting a new draft together. There are a small bunch of
nits to fix; that I can handle. There's also an implementation report
to write. On that I'd like to get some help.
The one possible big thing has to do with the ABNF in 2822. Out of
either altruism or insanity, some time ago Bruce Lilly had written up
changes to the ABNF in 2822 to do some cool things. On the plus side,
it seems to get rid of all of the [C]FWS shift-reduce conflicts, and
it is already done. On the minus side, I don't know anyone (myself
included) who has gone over it with a fine tooth comb, it is a
significant number of ABNF changes, and it therefore might recycle us
at Proposed. I am open to suggestions on this.
Comments?
Keith Moore
> Would 2822bis obsolete 822 fully? If so, I believe it should mention
> what happened to the X- header prefix that was part of 822. I don't
> believe simply removing them without stating why is a good idea,
> people can draw many different conclusions from that, which affect
> real implementations.
some people think X- headers are extremely useful, others think they
were a bad mistake that should be eradicated. maybe we need a separate
RFC about X- fields. I'd hate to see this argument block progression
of 2822 bis.
Keith Moore
> Replacing 822 and in the course silently dropping X-* would leave a lot
> of software and systems stand outside without the protective umbrella
> they had up to now; they survive but if it rains, they get wet.
X- headers aren't going to get dropped.
> RFC-1918 is a godsend for IP addressing.
I'm going to assume this is a troll, and ignore it.
Keith
Keith Moore
well, I do sympathize with the idea that being silent on the subject is
confusing. but I don't want that discussion to be critical path for 3822.
Keith Moore
> Start saying things like that and you'll only encourage me to write
> the "X- considered harmful" document. ;-)
and if you do, I'll write the "X- considered wonderful" document :)
actually I think we could probably find a compromise position that we
could agree on, as long as we don't make it part of the critical path
for more important work.
Al Costanzo
I am one of those people that think that the X-Headers are useful since
software we produce uses them also one of those peopel that think MIME has
gone amuck.
I would oppose:
1. Any RFC that completely obsoletes RFC822
2. Any RFC that remove, obsoletes, etc. the X-Headers fields.
Al
Simon Josefsson
Keith Moore <mo...@cs.utk.edu> writes:
Ignoring the problem won't make it go away, though. Consciously
removing text is driven by some motivation. Why not state that
motivation in the document? Furthermore, if 2822bis fully obsolete
822, without mentioning X- headers, then the X- headers have been
effectively obsoleted. I don't see how acknowledging that implicit
decision by stating it makes matters worse.
Thanks,
Simon
WJCarpenter
km> As a practical matter, nothing can really obsolete 822 as long as
km> people still need to read old mail messages.
I don't think this is a useful consideration, and it is perhaps even
harmful to the discussion at hand. People (hey, I'm one of those
people) need to read old email in a variety of file formats, and that
includes some that are very like RFC-822. It also includes many
formats which are widely different from RFC-822.
Who cares?
Software that used to understand RFC-822 will not suddenly stop
understanding it unless some implementer does something to make it so,
just as implementers already occasionally, silently do something which
isn't RFC-822 compliant.
In any event, people using RFC-822/2822-ish formats are an order of
magnitude less screwed than those whose archived email gets trapped
inside undocumented proprietary formats. There are a heck of a lot
more of those people these days, though they don't tend to participate
in IETF discussions.
--
bi...@carpenter.ORG (WJCarpenter) PGP 0x91865119
38 95 1B 69 C9 C6 3D 25 73 46 32 04 69 D6 ED F3
Pete Resnick
On 1/2/04 at 3:54 PM -0500, Keith Moore wrote:
>>If we can get 2822bis to full Standard (which is the desired
>>intent), it should obsolete 822 in that it should contain all of
>>the grammar to read old messages. That was the intent of section 4
>>of 2822.
>
>yeah, I know. I'm sort of a purist though - I think if you really
>want to read 822 messages you should consult 822 as the definitive
>spec.
It will always have the informational references to 822.
>>I don't think anyone would have a problem with a small note in the
>>"Changes to previous standards" section which says, "Some people
>>think X-* fields are bad, some think they are good, they were
>>removed from this document."
>
>and for the time being 822 is still the definitive spec on X- fields.
Start saying things like that and you'll only encourage me to write
the "X- considered harmful" document. ;-)
pr
Pete Resnick
On 1/2/04 at 11:14 AM -0500, Keith Moore wrote:
>>>On 1/2/04 at 1:34 PM +0100, Simon Josefsson wrote:
>>>
>>>>Would 2822bis obsolete 822 fully?
>
>2822 bis will not obsolete 822 in any event - 822 is still a full
>Standard, while 2822 bis would only be Draft Standard. As a
>practical matter, nothing can really obsolete 822 as long as people
>still need to read old mail messages.
Uh, no. If we can get 2822bis to full Standard (which is the desired
intent), it should obsolete 822 in that it should contain all of the
grammar to read old messages. That was the intent of section 4 of
2822.
>>>>If so, I believe it should mention what happened to the X- header
>>>>prefix that was part of 822.
I don't think anyone would have a problem with a small note in the
"Changes to previous standards" section which says, "Some people
think X-* fields are bad, some think they are good, they were removed
from this document." For a fuller discussion, I think Keith is right
that we might consider a separate informational document.
Keith Moore
> >> Would 2822bis obsolete 822 fully? If so, I believe it should
> >mention> what happened to the X- header prefix that was part of 822.
> >I don't> believe simply removing them without stating why is a good
> >idea,> people can draw many different conclusions from that, which
> >affect> real implementations.
> >
> > some people think X- headers are extremely useful, others think they
> > were a bad mistake that should be eradicated. maybe we need a
> > separate RFC about X- fields. I'd hate to see this argument block
> > progression of 2822 bis.
>
> Ignoring the problem won't make it go away, though. Consciously
> removing text is driven by some motivation.
IIRC it was driven by the inability to get agreement on the topic within
the DRUMS WG.
Keith Moore
> The one possible big thing has to do with the ABNF in 2822. Out of
> either altruism or insanity, some time ago Bruce Lilly had written up
> changes to the ABNF in 2822 to do some cool things. On the plus side,
> it seems to get rid of all of the [C]FWS shift-reduce conflicts, and
> it is already done. On the minus side, I don't know anyone (myself
> included) who has gone over it with a fine tooth comb, it is a
> significant number of ABNF changes, and it therefore might recycle us
> at Proposed. I am open to suggestions on this.
if the new grammar accepted the same language as the old one, I'd
consider it nothing more than a clarification of the spec, and a reset
to Proposed would be unnecessary.
or if the only differences could be seen as minor bug fixes to the old
grammar, I'd consider it a minor bug fix to the spec.
IMHO simplifying the grammar and getting rid of parsing conflicts would
both be highly desirable improvements. if it then becomes possible to
feed the ABNF to a parser generator and generate a verifier for 2822
messages, that's a huge win.
why not post the revised ABNF as an I-D so that others can go over it
with a fine tooth comb? or has this already been done, and I missed
it?
Keith Moore
>>>>> Would 2822bis obsolete 822 fully?
>>
>> 2822 bis will not obsolete 822 in any event - 822 is still a full
>> Standard, while 2822 bis would only be Draft Standard. As a
>> practical matter, nothing can really obsolete 822 as long as people
>> still need to read old mail messages.
>
> Uh, no. If we can get 2822bis to full Standard (which is the desired
> intent), it should obsolete 822 in that it should contain all of the
> grammar to read old messages. That was the intent of section 4 of
> 2822.
yeah, I know. I'm sort of a purist though - I think if you really want
to read 822 messages you should consult 822 as the definitive spec.
>>>>> If so, I believe it should mention what happened to the X- header
>>>>> prefix that was part of 822.
>
> I don't think anyone would have a problem with a small note in the
> "Changes to previous standards" section which says, "Some people think
> X-* fields are bad, some think they are good, they were removed from
> this document."
and for the time being 822 is still the definitive spec on X- fields.
Al Costanzo
If they are not part of this document it cannot replace/obsolete 822.
Al Costanzo
----- Original Message -----
From: "Pete Resnick" <pres...@qualcomm.com>
To: "Keith Moore" <mo...@cs.utk.edu>
Cc: "Simon Josefsson" <j...@extundo.com>; <mo...@cs.utk.edu>;
<ietf...@imc.org>
Sent: Friday, January 02, 2004 3:37 PM
Subject: Re: Getting 2822 to Draft
>
Kurt Keller
Replacing 822 and in the course silently dropping X-* would leave a lot
of software and systems stand outside without the protective umbrella
they had up to now; they survive but if it rains, they get wet.
Some grepping over a couple of inboxes showed that in 650 messages there
were 1232 X-* headers. Many anti-spam and anti-virus systems use such
headers, as does some mailing list software etc, even the messages from
this ietf-822 list do have a X-Authentication-Warning: header.
RFC-1918 is a godsend for IP addressing. X-* headers could be handled in
just a paragraph or two and can be a godsend in many situations as well.
Users and systems which need to use some proprietary header and can do
with an X-* won't need to bloat the standards with their specialized
extensions.
I think there definitely should be either a note in 3822, and a separate
specification, or (preferrably and simpler) a short definition directly
in 3822.
Pete Resnick
On 1/3/04 at 11:00 AM -0500, Al Costanzo wrote:
>>I don't think anyone would have a problem with a small note in the
>>"Changes to previous standards" section which says, "Some people
>>think X-* fields are bad, some think they are good, they were
>>removed from this document." For a fuller discussion, I think Keith
>>is right that we might consider a separate informational document.
>
>If they are not part of this document it cannot replace/obsolete 822.
I'm sorry, but I think this is rubbish. X-* fields are perfectly
legal in 2822 (they fall under "optional-field"); they simply are not
given the special treatment of 822, that they won't ever be official
published extensions to the standard. The only difference then
between 822 and 2822 in this respect is that 822 gives publication
guidelines for extensions where 2822 does not. This has absolutely
*no* effect on implementations of the protocol. I defy anyone to
explain to me why this should have any effect on 2822bis obsoleting
822.
Now, let me use this opportunity to explain *why* I don't think X-*
fields should be in the standard in the first place:
The idea behind X-* fields (what 822 called user-defined fields) was
so that folks could use new field names without worrying that they
would interfere with the standard fields. That way, you didn't have
to go through the trouble of publishing an RFC just to use a field
that was only going to be internal to your particular system. The
downside was that someone else could come along and potentially use
the same field name in a completely incompatible way, but that was
the risk that you took.
But since then, two things have made it clear that such user-defined
fields are unnecessary and problematic:
1. Unnecessary: We now have IANA registration services. We could very
simply create an IANA registry of e-mail field names. Then, if you
wanted to create a new field, you just fill out the registration form
and you're not only guaranteed that you won't interfere with
published standard fields, you're also guaranteed that nobody else
would use the field you chose in an incompatible way. And you
wouldn't have to have a name that started with "X-".
2. Problematic: Some folks thought it was nice to have "X-*" fields
for completely private use. But history teaches us that inevitably
"X-*" fields *will* leak out onto the rest of the Internet. As
several people in this thread have pointed out, we now have some
"X-*" fields that are in widespread use. Sometimes, people use them
very interoperably and they have served a fabulous purpose.
Sometimes, they don't get used interoperably. It would be great if we
could publish some standards to make sure that they do get used
interoperably. But guess what: We can't! It is forbidden by RFC 822
to publish any standard for a field name that beings with "X-". No
matter how widespread the use, no matter how useful the field is, it
can not be standardized *by definition*. That decreases
interoperability on the Internet.
Now, personally, I would love the document to say, "There is no good
reason to use fields that begin with 'X-' as defined by RFC 822; any
name that does not interfere with a currently registered or published
field name is fine, and using an 'X-' field is discouraged because
they can't ever be standardized." But there was widespread
non-consensus in the DRUMS Working Group when this was discussed.
2822 does say "Extension header fields no longer specifically called
out." In 2822bis, I think it would be fine to clarify that and say
"User-defined header fields (those starting with 'X-') no longer
specifically discussed", and even have an informational reference to
a document (or 2) on why it was pulled. But I see absolutely no
reason that this should stop 2822bis from moving to full Standard and
obsoleting 822.
Dave Crocker
Folks
> 2822 allows X-* fields and always has. Any future document will
> allow them. What 2822 doesn't do is give special meaning to them
> other than as optional-fields. -- Pete Resnick
> <http://www.qualcomm.com/~presnick/>
> QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-
> 1102
1. If X- fields are the only technical aspect of 2822 that folks have
any comments about, then the doc is clearly is really excellent shape.
2. In spite of originally thinking that X- fields were a really
excellent idea, I am now a vigorous opponent to the use of X- fields.
However I think that any significant effort to focus on their use is a
waste of time, at best, and more likely will just confuse things. The
labels exist. They are part of the usage model. The damage they cause
might be real but it would be difficult to say that it has really caused
major damage to the Internet. Adding a discussion paragraph to 2822 is
as much as might be useful.
d/
--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <http://brandenburg.com>
Pete Resnick
On 1/3/04 at 1:24 AM +0100, Kurt Keller wrote:
>Replacing 822 and in the course silently dropping X-* would leave a lot
>of software and systems stand outside without the protective umbrella
>they had up to now; they survive but if it rains, they get wet.
2822 allows X-* fields and always has. Any future document will allow
ned+ie...@mrochek.com
> On 1/3/04 at 11:00 AM -0500, Al Costanzo wrote:
> >>I don't think anyone would have a problem with a small note in the
> >>"Changes to previous standards" section which says, "Some people
> >>think X-* fields are bad, some think they are good, they were
> >>removed from this document." For a fuller discussion, I think Keith
> >>is right that we might consider a separate informational document.
> >
> >If they are not part of this document it cannot replace/obsolete 822.
> I'm sorry, but I think this is rubbish. X-* fields are perfectly
> legal in 2822 (they fall under "optional-field"); they simply are not
> given the special treatment of 822, that they won't ever be official
> published extensions to the standard. The only difference then
> between 822 and 2822 in this respect is that 822 gives publication
> guidelines for extensions where 2822 does not. This has absolutely
> *no* effect on implementations of the protocol. I defy anyone to
> explain to me why this should have any effect on 2822bis obsoleting
> 822.
Pete, I am in complete agreement with your analysis here. Additionally, I will
point out that even had 2822 done something else with X- fields, such as
deprecating their use, even that would be no impediment to 2822 specification
eventually obsoleting 822. There is certainly no requirement that new
specifications retain all the features of old ones they obsolete. Indeed, when
advancing on the standards track, the one change we do allow along the way is
for features to be dropped.
> Now, let me use this opportunity to explain *why* I don't think X-*
> fields should be in the standard in the first place:
> The idea behind X-* fields (what 822 called user-defined fields) was
> so that folks could use new field names without worrying that they
> would interfere with the standard fields. That way, you didn't have
> to go through the trouble of publishing an RFC just to use a field
> that was only going to be internal to your particular system. The
> downside was that someone else could come along and potentially use
> the same field name in a completely incompatible way, but that was
> the risk that you took.
> But since then, two things have made it clear that such user-defined
> fields are unnecessary and problematic:
> 1. Unnecessary: We now have IANA registration services. We could very
> simply create an IANA registry of e-mail field names.
This is already underway and the document specifying it -
draft-klyne-msghdr-registry-06.txt - has been discussed on this list. I note in
passing that it doesn't distinguish X- fields as being in any way special
either.
An interesting question is whether or not 2822bis should reference this
registry, assuming the specification for it is approved in the near future
(which I hope it will be).
> Then, if you
> wanted to create a new field, you just fill out the registration form
> and you're not only guaranteed that you won't interfere with
> published standard fields, you're also guaranteed that nobody else
> would use the field you chose in an incompatible way. And you
> wouldn't have to have a name that started with "X-".
> 2. Problematic: Some folks thought it was nice to have "X-*" fields
> for completely private use. But history teaches us that inevitably
> "X-*" fields *will* leak out onto the rest of the Internet. As
> several people in this thread have pointed out, we now have some
> "X-*" fields that are in widespread use. Sometimes, people use them
> very interoperably and they have served a fabulous purpose.
> Sometimes, they don't get used interoperably. It would be great if we
> could publish some standards to make sure that they do get used
> interoperably. But guess what: We can't! It is forbidden by RFC 822
> to publish any standard for a field name that beings with "X-". No
> matter how widespread the use, no matter how useful the field is, it
> can not be standardized *by definition*. That decreases
> interoperability on the Internet.
I agree with this as well.
> Now, personally, I would love the document to say, "There is no good
> reason to use fields that begin with 'X-' as defined by RFC 822; any
> name that does not interfere with a currently registered or published
> field name is fine, and using an 'X-' field is discouraged because
> they can't ever be standardized." But there was widespread
> non-consensus in the DRUMS Working Group when this was discussed.
Yep.
> 2822 does say "Extension header fields no longer specifically called
> out." In 2822bis, I think it would be fine to clarify that and say
> "User-defined header fields (those starting with 'X-') no longer
> specifically discussed", and even have an informational reference to
I'd say "defined or discussed" rather than simply "discussed", but
other than that I think this is fine.
> a document (or 2) on why it was pulled. But I see absolutely no
> reason that this should stop 2822bis from moving to full Standard and
> obsoleting 822.
Nor do I.
Ned
Bruce Lilly
Keith Moore wrote:
>
>> The one possible big thing has to do with the ABNF in 2822. Out of
>> either altruism or insanity, some time ago Bruce Lilly had written up
>> changes to the ABNF in 2822 to do some cool things. On the plus side,
>> it seems to get rid of all of the [C]FWS shift-reduce conflicts, and
>> it is already done. On the minus side, I don't know anyone (myself
>> included) who has gone over it with a fine tooth comb, it is a
>> significant number of ABNF changes, and it therefore might recycle us
>> at Proposed. I am open to suggestions on this.
>
I can't comment on insanity (catch-22). I decided to build a
grammar-based parser because
I was tired of writing (and rewriting) application-specific 822
parsers. I started a couple
of months after 2822 was published, which was a big help primarily
because of the formal
and explicit use of FWS and CFWS vs. 822's rather vague text.
> if the new grammar accepted the same language as the old one, I'd
> consider it nothing more than a clarification of the spec, and a reset
> to Proposed would be unnecessary.
>
> or if the only differences could be seen as minor bug fixes to the old
> grammar, I'd consider it a minor bug fix to the spec.
The intent is to accept the same message text as the existing 2822
grammar, modulo some minor
glitches in 2822.
> IMHO simplifying the grammar and getting rid of parsing conflicts
> would both be highly desirable improvements. if it then becomes
> possible to feed the ABNF to a parser generator and generate a
> verifier for 2822 messages, that's a huge win.
Whether it's "simpler" is perhaps debatable. Shift/reduce conflicts are
eliminated. Generating
a verifier for 2822 [822/733/724/561/MIME/etc.] isn't quite as easy as
feeding the ABNF to
a parser generator, but it is possible to feed something quite close to
the ABNF to a parser
generator (http://users.erols.com/blilly/mparse). And that parser
generator can be used to build
a message verifier. The complete grammar (2822/822/MIME/various
extensions/most of 733,
724, 561) has no unresolved shift/reduce conflicts and a single
reduce/reduce conflict (due to
changing grammar of the MIXER Supersedes (formerly Obsoletes) field,
resolved in favor of
the current (RFC 2156) definition).
> why not post the revised ABNF as an I-D so that others can go over it
> with a fine tooth comb? or has this already been done, and I missed it?
An earlier version of the revised ABNF has been posted to ietf-822 and
the full version
(with encoded-word grammar) is available at the above URI. Let the
combing begin.
Bruce Lilly
Pete Resnick wrote:
> X-* fields are perfectly legal in 2822 (they fall under
> "optional-field"); they simply are not given the special treatment of
> 822, that they won't ever be official published extensions to the
> standard. The only difference then between 822 and 2822 in this
> respect is that 822 gives publication guidelines for extensions where
> 2822 does not. This has absolutely *no* effect on implementations of
> the protocol.
There is a minor effect. A user-defined field (per 822 definition) can
be recognized as such by examining the
first two octets of the field name, which is quite efficient. While
there exist efficient methods of identifying
fields (e.g. http://www.gnu.org/directory/gperf.html), they are not
quite *as* efficient as a two-octet (case-
insensitive) comparison. Moreover, "X-" serves to differentiate
user-defined fields from non-defined
fields (i.e. those field names for which there is no IETF published
definition or which the implementation
does not recognize).
> Now, let me use this opportunity to explain *why* I don't think X-*
> fields should be in the standard in the first place:
>
> The idea behind X-* fields (what 822 called user-defined fields) was
> so that folks could use new field names without worrying that they
> would interfere with the standard fields. That way, you didn't have to
> go through the trouble of publishing an RFC just to use a field that
> was only going to be internal to your particular system. The downside
> was that someone else could come along and potentially use the same
> field name in a completely incompatible way, but that was the risk
> that you took.
>
> But since then, two things have made it clear that such user-defined
> fields are unnecessary and problematic:
>
> 1. Unnecessary: We now have IANA registration services. We could very
> simply create an IANA registry of e-mail field names. Then, if you
> wanted to create a new field, you just fill out the registration form
> and you're not only guaranteed that you won't interfere with published
> standard fields, you're also guaranteed that nobody else would use the
> field you chose in an incompatible way. And you wouldn't have to have
> a name that started with "X-".
Yes, but we don't yet have such a field name registry (and obviously we
didn't have one in 2001 when
2822 was published). In order to be at all useful to implementors,
registration would have to be contingent
upon the existence of a stable, formal, public definition of the
proposed field's syntax (with ABNF) and
semantics. That differs from publication via RFC regarding review and
comment; and I'm not convinced
that foregoing such review and comment would be a good thing.
> 2. Problematic: Some folks thought it was nice to have "X-*" fields
> for completely private use. But history teaches us that inevitably
> "X-*" fields *will* leak out onto the rest of the Internet. As several
> people in this thread have pointed out, we now have some "X-*" fields
> that are in widespread use. Sometimes, people use them very
> interoperably and they have served a fabulous purpose. Sometimes, they
> don't get used interoperably. It would be great if we could publish
> some standards to make sure that they do get used interoperably. But
> guess what: We can't! It is forbidden by RFC 822 to publish any
> standard for a field name that beings with "X-". No matter how
> widespread the use, no matter how useful the field is, it can not be
> standardized *by definition*. That decreases interoperability on the
> Internet.
It's difficult to see how such problems are either unique to X- or are
more of a hindrance to
interoperability than for other fields. As an example, consider "Status"
which was in "private" use
(I believe) by BSD "mailx" decades ago, and which is currently in use by
several other MUAs,
and which *does* leak out; there is also a formal definition of a
"Status" field -- incompatible
with the private usage -- defined as one of the delivery status
notification fields (RFC 3464).
So neither leakage nor incompatibility seem to be unique to X- fields.
Note that if BSD mailx'
author(s) had used X-Status for private use, there would be no conflict
with the formal DSN
Status field.
> Now, personally, I would love the document to say, "There is no good
> reason to use fields that begin with 'X-' as defined by RFC 822; any
> name that does not interfere with a currently registered or published
> field name is fine, and using an 'X-' field is discouraged because
> they can't ever be standardized." But there was widespread
> non-consensus in the DRUMS Working Group when this was discussed. 2822
> does say "Extension header fields no longer specifically called out."
> In 2822bis, I think it would be fine to clarify that and say
> "User-defined header fields (those starting with 'X-') no longer
> specifically discussed", and even have an informational reference to a
> document (or 2) on why it was pulled. But I see absolutely no reason
> that this should stop 2822bis from moving to full Standard and
> obsoleting 822.
There is one very good reason to use X- for private or experimental use,
viz. interoperability.
Use of X- as a field name prefix for private or experimental use
guarantees that there will be
no conflict with a formal field name. Use of other names can lead to
conflicts, as in the case
of "Status", the damage in that case fortunately being limited by the
fact that in that case, DSN
fields are significant only in one part of a specific type of MIME
multipart message and the
private use occurs only in the top-level message header.
An issue not mentioned above is migration of X- fields to a formal
definition following
successful experimental use. That obviously entails a name change.
However name changes
are not uncommon, e.g. refer to the MIXER RFCs, which define a number of
fields whose
names have changed. That merely means that parsers need to recognize
one name as a
synonym for another.
I don't see the X- issue as any reason to keep 2822 from progressing;
from a practical point of
view (as noted by others), parsers will need to be able to handle the X-
fields already in use,
reading "old" messages still requires parsers to be able to handle RFC
822 syntax -- and that
will continue to be the case even after 822 is eventually obsoleted
regardless of what 2822
et succ. have to say about the matter, just as it is now necessary to
support at least some
RFC 733 and 724 constructs to be able to read really "old" messages even
though those RFCs
have been formally obsolete for decades.
Pete Resnick
On 1/4/04 at 2:54 AM -0500, Bruce Lilly wrote:
>Pete Resnick wrote:
>
>>The only difference then between 822 and 2822 in this respect is
>>that 822 gives publication guidelines for extensions where 2822
>>does not. This has absolutely *no* effect on implementations of the
>>protocol.
>
>There is a minor effect. A user-defined field (per 822 definition)
>can be recognized as such by examining the first two octets of the
>field name, which is quite efficient.
But *why* would an implementation care about the publication
guidelines for the field? That's the *only* thing "X-" tells you.
>"X-" serves to differentiate user-defined fields from non-defined
>fields (i.e. those field names for which there is no IETF published
>definition or which the implementation does not recognize).
Again, what use is there in making that distinction? Do you think the
fact that "X-Priority" and "X-Face" start with "X-" means that you
shouldn't support them in your implementation or they don't have
well-defined syntax? What about "X-Sender"? Is "List-ID" a
non-defined field if your implementation doesn't recognize it? What
useful purpose is there in differentiating fields which start with
"X-"?
>In order to be at all useful to implementors, registration would
>have to be contingent upon the existence of a stable, formal, public
>definition of the proposed field's syntax (with ABNF) and semantics.
Nonsense. It is useful to an implementation (especially an
implementation that generates fields) to know that a field exists for
a particular purpose even if its syntax and semantics have not
undergone extensive review and comment or are still under
development. It is by implementation that fields get stable and then
they can be documented as standards.
>It's difficult to see how such problems are either unique to X- or
>are more of a hindrance to interoperability than for other fields.
>As an example, consider "Status" which was in "private" use (I
>believe) by BSD "mailx" decades ago, and which is currently in use
>by several other MUAs, and which *does* leak out; there is also a
>formal definition of a "Status" field -- incompatible with the
>private usage -- defined as one of the delivery status notification
>fields (RFC 3464). So neither leakage nor incompatibility seem to be
>unique to X- fields. Note that if BSD mailx' author(s) had used
>X-Status for private use, there would be no conflict with the formal
>DSN Status field.
First of all, the "Status" field of DSN is not defined to appear in a
top-level header of a [2]822 message, so there is no
"incompatibility" between the two. But let's talk about the leakage:
Yes, the "Status" field leaks. Now, what can be done about that?
Well, it could be documented in an RFC so that everyone could know
what it means. And if turned out useful, it could be a standardized
top-level header field. But what if mailx's author had instead used
"X-Status"? In that case, it's DOA, because by definition it could
not be documented in a standard way. So there is no way for an
implementor to figure out what "X-Status" means other than by word of
mouth. That invites incompatibility. So what exactly would that "X-"
have gained you?
>There is one very good reason to use X- for private or experimental
>use, viz. interoperability. Use of X- as a field name prefix for
>private or experimental use guarantees that there will be no
>conflict with a formal field name. Use of other names can lead to
>conflicts
No it doesn't. Some "X-" field names are now just as formal as some
non-"X-" field names. And I can come up with a list of non-"X-" field
names for which it would be *much* safer to have two incompatible
implemenations than some "X-" field names. The only thing the "X-"
guarantees you is long, ongoing conflicts. Using a non-"X-" field
name means that you can document one use as *the* standard way to do
something.
>An issue not mentioned above is migration of X- fields to a formal
>definition following successful experimental use. That obviously
>entails a name change. However name changes are not uncommon, e.g.
>refer to the MIXER RFCs, which define a number of fields whose names
>have changed. That merely means that parsers need to recognize one
>name as a synonym for another.
But what about generators? Because there will be parsers out there
that will only interpret the "X-" form of the field, the generators
must continue to send the "X-" form. Furthermore, updating some
parsers is non-trivial.
pr
Adam M. Costello
Pete Resnick <pres...@qualcomm.com> wrote:
> In 2822bis, I think it would be fine to clarify that and say
> "User-defined header fields (those starting with 'X-') no longer
> specifically discussed",
I think that might be a little too terse. I think it could be a lot
clearer with slightly more explanation:
There is no longer a distinction made between User-defined header
fields (which start with "X-") and Extension header fields (which
don't). Both are now treated uniformly as Optional header fields.
AMC
Keith Moore
> But *why* would an implementation care about the publication
> guidelines for the field? That's the *only* thing "X-" tells you.
because any field that doesn't have a published definition probably
isn't something that should be implemented on a widespread basis -
since without a definition you have no reason to expect it to work
predictably from one implementation to another.
admittedly, this inference might be a bit subtle. I see no harm in
making it more explicit.
Keith
Charles Lindsey
In <p06100723bc1aa0f50ab0@[10.0.2.4]> Pete Resnick <pres...@qualcomm.com> writes:
>So, for the new year I started thinking about getting 2822 to Draft.
>(Also, 3822 is coming up in the RFC numbers :-) ). I've gotten
>started on getting a new draft together. There are a small bunch of
>nits to fix; that I can handle. There's also an implementation report
>to write. On that I'd like to get some help.
Well if we are collecting small things that need to be changed, there is
the matter of compatibility with Netnews. Usefor has tried to bring the
syntax of Netnews headers into line with RFC 2822, and has mostly
succeeded, but there are one to two nits which remain different, and where
it is plain impractical for News to change. Therefore, there are a few
features currently valid in RFC 2822 that I would like to move into the
obsolete syntax. None of them is ever encountered in current email
practice that I have seen, so no one will notice their loss.
So, in due course, I would like to start a separate thread to discuss
these.
>The one possible big thing has to do with the ABNF in 2822. Out of
>either altruism or insanity, some time ago Bruce Lilly had written up
>changes to the ABNF in 2822 to do some cool things. On the plus side,
>it seems to get rid of all of the [C]FWS shift-reduce conflicts, and
>it is already done.
One of the problems with this is likely to be that, although it is known
to be theoretically possible to construct a BNF grammar for any
context-free language, in practice it leads to humanly-unreadable grammars
in many cases, and I suspect this may be one such.
However, I see that Bruce has now posted his grammar, so I shall study it
carefully.
--
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133 Web: http://www.cs.man.ac.uk/~chl
Email: c...@clerew.man.ac.uk Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9 Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
Charles Lindsey
In <01L4Z0G3H...@mauve.mrochek.com> ned+ie...@mrochek.com writes:
>> On 1/3/04 at 11:00 AM -0500, Al Costanzo wrote:
>> 1. Unnecessary: We now have IANA registration services. We could very
>> simply create an IANA registry of e-mail field names.
>This is already underway and the document specifying it -
>draft-klyne-msghdr-registry-06.txt - has been discussed on this list. I note in
>passing that it doesn't distinguish X- fields as being in any way special
>either.
Yes, I am surprised that people have been going over issues which the
Klyne draft has already resolved. I think it will be best to get that
draft adopted as a BCP and the IANA register created before starting
serious work on 2822-bis. But I hope that will have happened within
months, if not sooner.
>An interesting question is whether or not 2822bis should reference this
>registry, assuming the specification for it is approved in the near future
>(which I hope it will be).
Yes, I think it should. There is a corresponding place in the Usefor draft
where it discusses X-headers (and when not to use them), and the wording
there would be much simpler if it could just point to the Klyne BCP.
>> Then, if you
>> wanted to create a new field, you just fill out the registration form
>> and you're not only guaranteed that you won't interfere with
>> published standard fields, you're also guaranteed that nobody else
>> would use the field you chose in an incompatible way. And you
>> wouldn't have to have a name that started with "X-".
>> 2. Problematic: Some folks thought it was nice to have "X-*" fields
>> for completely private use. But history teaches us that inevitably
>> "X-*" fields *will* leak out onto the rest of the Internet. As
>> several people in this thread have pointed out, we now have some
>> "X-*" fields that are in widespread use. Sometimes, people use them
>> very interoperably and they have served a fabulous purpose.
>> Sometimes, they don't get used interoperably. It would be great if we
>> could publish some standards to make sure that they do get used
>> interoperably. But guess what: We can't! It is forbidden by RFC 822
>> to publish any standard for a field name that beings with "X-". No
>> matter how widespread the use, no matter how useful the field is, it
>> can not be standardized *by definition*. That decreases
>> interoperability on the Internet.
>I agree with this as well.
Bur I don't. What you say is that X-headers are for human readers only -
essentially they give the possibility for creating lots of extra
Comment-headers. But you also say that they may also be used for protocol
purposes ONLY where there is prior agreement between sender and recipient.
That is not so improbable as it sounds. If my ISP is kind enough to put
all my mail through Spamassassin and record the result in the form:
X-Spamassassin-Score: sssss
then I can use that header as a filter to send high scoring messages to a
special box, or even to /dev/null. Essentially, there is an agreement
(unwritten) between the ISP and myself that the header will appear and
that I can rely on that format.
ned+ie...@mrochek.com
> In <01L4Z0G3H...@mauve.mrochek.com> ned+ie...@mrochek.com writes:
> >> On 1/3/04 at 11:00 AM -0500, Al Costanzo wrote:
> >> 1. Unnecessary: We now have IANA registration services. We could very
> >> simply create an IANA registry of e-mail field names.
> >This is already underway and the document specifying it -
> >draft-klyne-msghdr-registry-06.txt - has been discussed on this list. I note in
> >passing that it doesn't distinguish X- fields as being in any way special
> >either.
> Yes, I am surprised that people have been going over issues which the
> Klyne draft has already resolved. I think it will be best to get that
> draft adopted as a BCP and the IANA register created before starting
> serious work on 2822-bis. But I hope that will have happened within
> months, if not sooner.
I agree 100%. The current status of the registry draft is that it has
completed last call and has been discussed by the IESG. There were
two discuss votes on the specification and the authors have posted
as revised version that hopefully addresses the issues that needed to
be addressed. However, there were a couple of concerns it isn't clear
how to address so the document remains under discussion.
> > An interesting question is whether or not 2822bis should reference this
> > registry, assuming the specification for it is approved in the near future
> > (which I hope it will be).
> Yes, I think it should. There is a corresponding place in the Usefor draft
> where it discusses X-headers (and when not to use them), and the wording
> there would be much simpler if it could just point to the Klyne BCP.
OK.
> >> Then, if you
> >> wanted to create a new field, you just fill out the registration form
> >> and you're not only guaranteed that you won't interfere with
> >> published standard fields, you're also guaranteed that nobody else
> >> would use the field you chose in an incompatible way. And you
> >> wouldn't have to have a name that started with "X-".
> >> 2. Problematic: Some folks thought it was nice to have "X-*" fields
> >> for completely private use. But history teaches us that inevitably
> >> "X-*" fields *will* leak out onto the rest of the Internet. As
> >> several people in this thread have pointed out, we now have some
> >> "X-*" fields that are in widespread use. Sometimes, people use them
> >> very interoperably and they have served a fabulous purpose.
> >> Sometimes, they don't get used interoperably. It would be great if we
> >> could publish some standards to make sure that they do get used
> >> interoperably. But guess what: We can't! It is forbidden by RFC 822
> >> to publish any standard for a field name that beings with "X-". No
> >> matter how widespread the use, no matter how useful the field is, it
> >> can not be standardized *by definition*. That decreases
> >> interoperability on the Internet.
> >I agree with this as well.
> Bur I don't. What you say is that X-headers are for human readers only -
> essentially they give the possibility for creating lots of extra
> Comment-headers. But you also say that they may also be used for protocol
> purposes ONLY where there is prior agreement between sender and recipient.
I see nothing in the previous discussion having anything to do with whether
the fields are used by humans or programs. The issue of whether or not
something is standardizable has nothing to do with who the eventual
consumer is.
> That is not so improbable as it sounds. If my ISP is kind enough to put
> all my mail through Spamassassin and record the result in the form:
> X-Spamassassin-Score: sssss
> then I can use that header as a filter to send high scoring messages to a
> special box, or even to /dev/null. Essentially, there is an agreement
> (unwritten) between the ISP and myself that the header will appear and
> that I can rely on that format.
Sure, there are loads of examples of both sorts of X- fields (and both
sorts of non-X- fields for that matter).
Ned
Keith Moore
Sigh. I'd hoped we could avoid this discussion, but it seems like
people can't resist focusing on it to the detriment of attention to more
substantive issues. (Or maybe this really is the most substantive issue
in 2822, and we should regard this debate as a sign that we don't have
anything worse to worry about?)
> The idea behind X-* fields (what 822 called user-defined fields) was
> so that folks could use new field names without worrying that they
> would interfere with the standard fields. That way, you didn't have
> to go through the trouble of publishing an RFC just to use a field
> that was only going to be internal to your particular system. The
> downside was that someone else could come along and potentially use
> the same field name in a completely incompatible way, but that was
> the risk that you took.
I won't claim that I can make statements about the original idea behind
X- fields, since I wasn't in on the discussions at the time. However
there are other useful properties of X- fields that derive from the 822
specification, specifically that the use of X- serves as a notice that
the field is nonstandard (and thus it hasn't received the review and
scrutiny that a standard would have received), that it never will be a
standard, that IF it somehow becomes widely used there is a good chance
it will not be consistently used, and that you should therefore avoid
assuming that an X- field means what you think it means.
These properties *are* useful for flagging experimental or local
features. And it's useful to have such a flag precisely because such
fields will leak outside of their experimental or local environment.
The fact that these fields leak should not cause a problem as long as
(a) the extra fields don't change the meaning of other fields and (b)
the software that creates the extra fields doesn't assume that either
recipient's UAs or intermediares will understand those fields. (These
rules should generally be followed by all header fields, not just
extension fields that begin with X-.) But IF you follow the 822
specification, the fact that a field-name begins with X- is an
indication that you shouldn't expect it to work on a widespread basis,
and that's a useful indication.
But this does mean that X- fields shouldn't be used by widely-deployed
software, except perhaps for "write only" fields like X-User-Agent
that mostly serve as comments or indications to human recipients.
> But since then, two things have made it clear that such user-defined
> fields are unnecessary and problematic:
>
> 1. Unnecessary: We now have IANA registration services. We could very
> simply create an IANA registry of e-mail field names. Then, if you
> wanted to create a new field, you just fill out the registration form
> and you're not only guaranteed that you won't interfere with
> published standard fields, you're also guaranteed that nobody else
> would use the field you chose in an incompatible way. And you
> wouldn't have to have a name that started with "X-".
Even when we have header registration, this won't make X- fields
unnecessary. Sometimes it's just too much trouble to get a field name
registered, especially if it's just for some experiment or local
feature.
That, and it's too easy to just code new field names into software
without registering them. If you're writing code and you find that
you need a new field, are you really going to take the page fault
to get that field name registered, or are you just going to type
printf ("new-field-name: whatever\r\n"); and be done with it?
If we give implementors a convention that let's them say "this field
isn't intended for widespread use" then at least some of those
implementors will be responsible and use that convention, which is far
better than not having a convention at all and encouraging implementors
to allocate field names on a whim. (URI prefix space is a huge mess
because implementors have done precisely that.)
And if we really registered every field name for every
local or experimental extension, we'd soon find that the field names we
wanted to use for extensions that are worthy of standardization, were
already allocated by half-baked ideas not worthy of standardization.
Yet the half-baked ideas would have the appearance of legitimacy, since
they'd have the "better" or "more obvious" names. We'd also lose the
useful ability to mark extensions as "local" or "experimental".
(though this could be accomodated by a different naming convention)
> 2. Problematic: Some folks thought it was nice to have "X-*" fields
> for completely private use. But history teaches us that inevitably
> "X-*" fields *will* leak out onto the rest of the Internet. As
> several people in this thread have pointed out, we now have some
> "X-*" fields that are in widespread use. Sometimes, people use them
> very interoperably and they have served a fabulous purpose.
> Sometimes, they don't get used interoperably. It would be great if we
> could publish some standards to make sure that they do get used
> interoperably. But guess what: We can't! It is forbidden by RFC 822
> to publish any standard for a field name that beings with "X-". No
> matter how widespread the use, no matter how useful the field is, it
> can not be standardized *by definition*. That decreases
> interoperability on the Internet.
The cases where you really want to standardize a field whose name begins
with X- are rare, because it's rare that someone manages to define a new
field in such a way that it's suitable for widespread use without
benefit of a published spec and wide review before deployment of that
feature. But I don't read RFC 822 as preventing publication of
information about X- fields. Indeed RFC 2076 does mention some X-
fields, so clearly it's possible to publish information about X- fields
with Informational status.
For most cases where the X- field turns out to really be what we want to
standardize, what we should do is publish a new standard with a new
field name, document the previous usage in a non-normative section of
the standard, and describe how senders and receivers should cope with
the two fields.
But if there were a case where there was a compelling reason to
standardize the use of an existing X- field, then I wouldn't object to
that specific field being treated as an exceptional case. We should do
the right thing for interoperability and functionality even if it wasn't
what 822 anticipated. But we should make it clear that this is an
exceptional case and not the general rule. If we make X- fields the
same as any other fields then we are optimizing for the rare case, and
at the same time doing harm to our ability to extend the mail protocol.
Keith
Bruce Lilly
bli...@verizon.net (Bruce Lilly) wrote on 04.01.04 in <3FF7C6B...@verizon.net>:
Pete Resnick wrote:This has absolutely *no* effect on implementations of the protocol.
Is *isn't* "a reason to be for or against the concept." It is an example of anThere is a minor effect. A user-defined field (per 822 definition) can be recognized as such by examining the first two octets of the field name, which is quite efficient. While there exist efficient methods of identifying fields (e.g. http://www.gnu.org/directory/gperf.html), they are not quite *as* efficient as a two-octet (case- insensitive) comparison.This seems an incredibly silly reason to be for or against the concept.
effect on implementation to counter the claim of "absolutely *no* effect on implementations".
An issue not mentioned above is migration of X- fields to a formal definition following successful experimental use. That obviously entails a name change. However name changes are not uncommon, e.g. refer to the MIXER RFCs, which define a number of fields whose names have changed. That merely means that parsers need to recognize one name as a synonym for another.
Or in other words, it means that interoperation with the installed base is
broken. Presumably that installed base is nontrivial, or else there would
have been no need to standardize. ("merely"?!)
Obsoletes (RFC 1327) -> Supersedes (RFC 2156)
Content-Identifier (RFC 1327) -> X400-Content-Identifier (RFC 2156)
Expiry-Date (RFC 1327) -> Expires (RFC 2156)
etc., and
X-Accept-Language (MUA usage) -> Accept-Language (RFC 3066)
Bruce Lilly
Pete Resnick wrote:
>
> On 1/4/04 at 2:54 AM -0500, Bruce Lilly wrote:
>
>> Pete Resnick wrote:
>>
>>> The only difference then between 822 and 2822 in this respect is
>>> that 822 gives publication guidelines for extensions where 2822 does
>>> not. This has absolutely *no* effect on implementations of the
>>> protocol.
>>
>>
>> There is a minor effect. A user-defined field (per 822 definition)
>> can be recognized as such by examining the first two octets of the
>> field name, which is quite efficient.
>
>
> But *why* would an implementation care about the publication
> guidelines for the field? That's the *only* thing "X-" tells you.
An implementation *might* be written so as to hand off parsing of X-
fields to a separate
procedure, or ignore them entirely. And the ability to efficiently
bypass checking against
the 100+ standardized field names *might* be considered important.
[my implementation does hand off most X- fields (one notable exception
being X-Accept-Language)
and unrecognized (in this case anything not standardized in an RFC or an
Internet Draft that
seems likely to become an RFC) to optional user-supplied functions.
Separate functions may
be specified for unrecognized X- (user-defined) fields and for
unrecognized extension fields
(or the application can point to a single function). Were it not for
handling X-Accept-Language,
I might have elected to bypass lookup for standard field names (via
gperf) in the case of user-
defined fields.]
>> "X-" serves to differentiate user-defined fields from non-defined
>> fields (i.e. those field names for which there is no IETF published
>> definition or which the implementation does not recognize).
>
>
> Again, what use is there in making that distinction? Do you think the
> fact that "X-Priority" and "X-Face" start with "X-" means that you
> shouldn't support them in your implementation or they don't have
> well-defined syntax? What about "X-Sender"?
I don't support them because I haven't found a specification for them in
any RFC.
> Is "List-ID" a non-defined field if your implementation doesn't
> recognize it? What useful purpose is there in differentiating fields
> which start with "X-"?
I do recognize and support List-ID (RFC 2919). Flexibility is one
reason to differentiate;
because RFC 822 made the distinction, it is conceivable that some
message-processing
application may wish to treat user-defined and non-standard extension
fields differently.
By providing for separate function pointers for the two cases, an
application using the mparse
library can either treat them differently (using distinct function
pointers) or treat them
identically (by providing two pointers to a single function).
>> In order to be at all useful to implementors, registration would have
>> to be contingent upon the existence of a stable, formal, public
>> definition of the proposed field's syntax (with ABNF) and semantics.
>
>
> Nonsense. It is useful to an implementation (especially an
> implementation that generates fields) to know that a field exists for
> a particular purpose even if its syntax and semantics have not
> undergone extensive review and comment or are still under development.
> It is by implementation that fields get stable and then they can be
> documented as standards.
Example:
somebody registers a "Foo" field, but provides no public syntax or
semantics. As an implementor,
what am I supposed to do about "Foo"?
>> It's difficult to see how such problems are either unique to X- or
>> are more of a hindrance to interoperability than for other fields. As
>> an example, consider "Status" which was in "private" use (I believe)
>> by BSD "mailx" decades ago, and which is currently in use by several
>> other MUAs, and which *does* leak out; there is also a formal
>> definition of a "Status" field -- incompatible with the private usage
>> -- defined as one of the delivery status notification fields (RFC
>> 3464). So neither leakage nor incompatibility seem to be unique to X-
>> fields. Note that if BSD mailx' author(s) had used X-Status for
>> private use, there would be no conflict with the formal DSN Status
>> field.
>
>
> First of all, the "Status" field of DSN is not defined to appear in a
> top-level header of a [2]822 message, so there is no "incompatibility"
> between the two. But let's talk about the leakage: Yes, the "Status"
> field leaks. Now, what can be done about that? Well, it could be
> documented in an RFC so that everyone could know what it means. And if
> turned out useful, it could be a standardized top-level header field.
> But what if mailx's author had instead used "X-Status"? In that case,
> it's DOA, because by definition it could not be documented in a
> standard way. So there is no way for an implementor to figure out what
> "X-Status" means other than by word of mouth. That invites
> incompatibility. So what exactly would that "X-" have gained you?
My implementation -- and I believe that it is not unique in this respect
-- parses a field with
a given name depending on that name and not specifically on its
context. The incompatibility
is that the DSN Status field has a specific syntax for the field body
(viz. three dot-separated
numbers) which is not matched by the BSD et al usage of Status (a string
of alphabetic characters).
A single field name with two different syntax definitions (and different
semantics) would IMO
be a bad thing, even if theoretically they could be differentiated by
context; header fields do
occasionally end up in message bodies due to some software inserting an
empty line, and that
empty line may cause top-level message headers moved into the body to
appear to be MIME-part
fields (i.e. if the message header still contains an appropriate
Content-Type field). Mailx uses
Status to store state metadata about the mail store, which is a private
use. X-Status could be
documented (e.g. via an Informative RFC), but not registered with IANA
as an extension field,
though *as* private use, there would be no reason to document it (given
that X- guarantees no
collision with any future registered extension field). The only reason
for an implementor to care
about a hypothetical X-Status field is if said implementor had reason to
interact with a message
store on a system that also used mailx with that message store -- that's
a storage issue, not a message
format or transmission issue and therefore an issue outside of the scope
of IETF. So X-Status would
not "invite incompatibility" in any sense in which it matters to IETF.
>> There is one very good reason to use X- for private or experimental
>> use, viz. interoperability. Use of X- as a field name prefix for
>> private or experimental use guarantees that there will be no conflict
>> with a formal field name. Use of other names can lead to conflicts
>
>
> No it doesn't. Some "X-" field names are now just as formal as some
> non-"X-" field names.[...]
"[J]ust as formal"? Really? Which X- field names are defined in
Standards-track RFCs? For that
matter which X- field names are defined in *any* RFC? Or by "non-'X-'
field names" do you
mean something other than registered extension field names?
>> An issue not mentioned above is migration of X- fields to a formal
>> definition following successful experimental use. That obviously
>> entails a name change. However name changes are not uncommon, e.g.
>> refer to the MIXER RFCs, which define a number of fields whose names
>> have changed. That merely means that parsers need to recognize one
>> name as a synonym for another.
>
>
> But what about generators? Because there will be parsers out there
> that will only interpret the "X-" form of the field, the generators
> must continue to send the "X-" form. Furthermore, updating some
> parsers is non-trivial.
By the same argument, generators will have to generate multiple versions
with a single non-X- name
as syntax changes during experimentation. Presumably those
participating in an experiment have a
vested interest in moving forward with an official implementation at the
end of the period of experimentation.
Whether or not some parsers are difficult to update is irrelevant to the
issue; if a parser can be updated to
handle Obsoletes/Supersedes, Expiry-Date/Expires,
Content-Identifier/X400-Content-Identifier, etc., then
it can be updated to handle X-Accept-Language/Accept-Language.
Conversely if it cannot be updated
to handle X-Accept-Language/Accept-Language, then presumably it cannot
handle the other (standard)
fields whose field names have been changed.
>
> pr
Bruce Lilly
Here's another take on the subject (I hope it doesn't open up several cans
of worms, but here goes...)
X- is used as an indicator for experimental or private-use tags in many
places
in Internet message protocols other than field names. For example:
charsets (RFC 2978, sect. 3.1, also RFC 2046, sect. 4.1.2)
MIME media types (RFC 2045, sections 5 & 5.1, also RFC 2046, section 6)
and subtypes
language tags (RFC 3066, section 2.2)
disposition-notification options (RFC2298 section 2.2 )
transfer encodings (RFC 2045, sect. 6.3)
MTA-name-types (RFC 3464 sect. 2.2)
access-types (RFC 2046 sect 5.2.3.1)
address-types (RFC 2298 sect. 3.1.2, also RFC 3464 sect. 2.2)
dispositions (RFC 2183 sect 2.8)
diagnostic types (RFC 3464 sect. 2.2)
disposition modifiers (RFC 2298 sect 3.2.6.3)
And is mentioned in Keith's auto-response draft for auto-submitted types
-- and I don't recall
any discussion about that here when that draft was recently discussed.
And of course there are MDN X- fields (RFC 2298 sect. 3.3) and DSN X-
fields (RFC 3464
sect. 2.4).
So what's so abhorrent about top-level X- fields (RFC 822) that doesn't
also apply to
charsets, media types, language tags, etc.?
And yet another feature of X- fields:
RFC 2047, section 5 notes that encoded-words may appear in X- fields.
In all
other fields, it is necessary to know the syntax of the field in order
to know
whether and where encoded-words are permitted. I.e. when displaying fields,
one may decode an encoded word in an appropriate place in a field body
if the
field syntax is known, and one may decode an encoded-word in any X- field
(presumed to be unstructured), but one may not decode anything which has
the form of an encoded-word in an unrecognized field (other than an X-
field).
Paul Smith
FWIW (NAL..)
My view on X- header fields is that they're a good thing if used properly -
but there should probably be a registry for optional fields in the long term.
RFC 2822 section 3.6.8 leaves things wide open IMHO - I could legitimately,
according to RFC 2822, create my own header field called 'Content-Type' and
put anything I want into it - because the only restriction in RFC 2822 is
"The field names of any optional-field MUST NOT be identical to any field
name specified elsewhere in *this* standard." Content-Type isn't defined in
RFC 2822, so I can use it..
If I want to create my own header for my own use (which happens) and I
don't have an exhaustive knowledge of all the RFCs, it's quite possible I
could inadvertently re-use a field name which already exists in a standard.
Also, I may use a custom field name which doesn't already exist, but then
someone else comes along and makes a new standard using the same name but
in a different way.
At the moment, there's no definitive way to avoid this, other than to use
an X- header (Yes, I could use '<MTA Name>-header' - which I have no
problem with - *if* this was 'standardised' as a way of defining custom
headers)
If there was a 'registry' of headers where I could register either field
names which I think would be useful to the community in general, or an MTA
name which I could then append my own custom field names to, then I'd be happy.
But, until that happens, I think there needs to be a way you can GUARANTEE
that a field name someone invents does not clash with a standard name. The
X- headers achieve that aim IMHO.
(Also, a minor point, but possibly significant none-the-less. Moderately
technical people who aren't email experts can generally accept that 'X-'
headers are custom headers. Without this distinction they can assume
they're standard. So, if an email parser can't parse, for instance,
'Status:' or 'Delivered-To:' headers, there can be user support issues
because 'they're not custom headers, why can't your parser understand
them?', whereas if they were X- headers this misunderstanding would be less
likely, or if they are standardised properly, the parser could understand them)
JM2P
Paul VPOP3 - Internet Email Server/Gateway
sup...@pscs.co.uk http://www.pscs.co.uk/
Arnt Gulbrandsen
Keith Moore writes:
> Sigh. I'd hoped we could avoid this discussion, but it seems like
> people can't resist focusing on it to the detriment of attention to
> more substantive issues. (Or maybe this really is the most
> substantive issue in 2822, and we should regard this debate as a sign
> that we don't have anything worse to worry about?)
I don't think said substantive issues will be discussed until there is a
draft, or at least an suggested list of changes. (I saw Bruce's
grammar, and will go through it next Thursday.)
--Arnt
Keith Moore
>>> But *why* would an implementation care about the publication
>>> guidelines for the field? That's the *only* thing "X-" tells you.
>>
>> isn't something that should be implemented on a widespread basis -
>> since without a definition you have no reason to expect it to work
>> predictably from one implementation to another.
>>
>> admittedly, this inference might be a bit subtle. I see no harm in
>> making it more explicit.
>
>
> Counterexamples:
>
> X-Face: - there's every expectation of this interoperating just fine.
Note that I said "probably". The fact that there are a small number of
counterexamples doesn't make it false. Yes, it can be made to
interoperate on any platform, but if we were to standardize a field for
the author's face, we might or might not choose this particular field
name, image format, and encoding.
Now IMHO, the use of X- in X-Face shouldn't prevent us from documenting
X-Face in an RFC, nor even from making it a standard - but only if we
really had consensus to choose that name and format.
Bruce Lilly
Pete Resnick wrote:
>
> So, for the new year I started thinking about getting 2822 to Draft.
> (Also, 3822 is coming up in the RFC numbers :-) ). I've gotten started
> on getting a new draft together. There are a small bunch of nits to
> fix; that I can handle. There's also an implementation report to
> write. On that I'd like to get some help.
>
> The one possible big thing has to do with the ABNF in 2822. Out of
> either altruism or insanity, some time ago Bruce Lilly had written up
> changes to the ABNF in 2822 to do some cool things. On the plus side,
> it seems to get rid of all of the [C]FWS shift-reduce conflicts, and
> it is already done. On the minus side, I don't know anyone (myself
> included) who has gone over it with a fine tooth comb, it is a
> significant number of ABNF changes, and it therefore might recycle us
> at Proposed. I am open to suggestions on this.
>
> Comments?
>
> pr
As
a) 2821 and 2822 are closely related
and
b) there remain a few inconsistencies between 2821 and 2822
is there a draft of a 2821 successor that can be reviewed at the same time
as the draft of the 2822 successor?
#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################
Russ Allbery
Keith Moore <mo...@cs.utk.edu> writes:
> Now IMHO, the use of X- in X-Face shouldn't prevent us from documenting
> X-Face in an RFC, nor even from making it a standard - but only if we
> really had consensus to choose that name and format.
And there's actually a much nicer and more modern Face header
specification that would be more suitable for standardization. I think
X-Face is really more of an informational RFC at this point, not a
direction we'd want to move towards going forward.
--
Russ Allbery (r...@stanford.edu) <http://www.eyrie.org/~eagle/>
Keith Moore
>> Now IMHO, the use of X- in X-Face shouldn't prevent us from
>> documenting
>> X-Face in an RFC, nor even from making it a standard - but only if we
>> really had consensus to choose that name and format.
>
> And there's actually a much nicer and more modern Face header
> specification that would be more suitable for standardization. I think
> X-Face is really more of an informational RFC at this point, not a
> direction we'd want to move towards going forward.
I guess I'm still looking for that canonical example of a read-write X-
field that really, really should be standardized as-is. Or maybe I've
forgotten it.
Russ Allbery
Charles Lindsey <c...@clerew.man.ac.uk> writes:
> And also in NNTP where there are widely deployed commands such as XOVER
> and XHDR. Nevertheless, the new NNTP draft if defining new commands OVER
> and HDR even though their syntax/semantics is 99.9% identical.
In both cases, fixing some long-standing problems in the process.
John C Klensin
--On Tuesday, 06 January, 2004 09:07 -0500 Bruce Lilly
<bli...@verizon.net> wrote:
The editor of 2821 also made a new year's resolution, and
actually opened the document last week to incorporate more of
the changes that have been suggested. The bad news is that the
working copy is in Word XP. I'm not a Word fan -- all of my
other I-Ds are in XML or directly edited in ASCII-- but because,
after the experiences with DRUMS, I wanted to be sure that I
could identify, in the working text, the source and
justification for all of the requested changes if any
controversy arose. The difficulty is that converting a Word
document with as many comments, change markup, internal cross
references, etc., as that one now has is far beyond the scope of
RFC 3285 (the attempt was what produced the "RFC 3285 considered
harmful" thread on the IETF list a year or so ago). So, I'm
reluctant to go through the rather painful conversion until I
have a complete draft. If you would like to look directly at
the Word form, or a PDF display of it (our experience has been
that Word XP documents with extensive comments and markup don't
get along with Mac Word either), I think that could be arranged
within the next week or two.
john
ned+ie...@mrochek.com
X-Accept-Language comes very close IMO.
Ned
Charles Lindsey
>Here's another take on the subject (I hope it doesn't open up several cans
>of worms, but here goes...)
>X- is used as an indicator for experimental or private-use tags in many
>places
>in Internet message protocols other than field names. For example:
>charsets (RFC 2978, sect. 3.1, also RFC 2046, sect. 4.1.2)
>MIME media types (RFC 2045, sections 5 & 5.1, also RFC 2046, section 6)
>and subtypes
>language tags (RFC 3066, section 2.2)
>disposition-notification options (RFC2298 section 2.2 )
>transfer encodings (RFC 2045, sect. 6.3)
>MTA-name-types (RFC 3464 sect. 2.2)
>access-types (RFC 2046 sect 5.2.3.1)
>address-types (RFC 2298 sect. 3.1.2, also RFC 3464 sect. 2.2)
>dispositions (RFC 2183 sect 2.8)
>diagnostic types (RFC 3464 sect. 2.2)
>disposition modifiers (RFC 2298 sect 3.2.6.3)
And generally speaking for tokens to introduce new parameters in
Content-Type and other headers that use the same parameter syntax.
And also in NNTP where there are widely deployed commands such as XOVER
and XHDR. Nevertheless, the new NNTP draft if defining new commands OVER
and HDR even though their syntax/semantics is 99.9% identical.
>And is mentioned in Keith's auto-response draft for auto-submitted types
>-- and I don't recall
>any discussion about that here when that draft was recently discussed.
>And of course there are MDN X- fields (RFC 2298 sect. 3.3) and DSN X-
>fields (RFC 3464
>sect. 2.4).
>So what's so abhorrent about top-level X- fields (RFC 822) that doesn't
>also apply to
>charsets, media types, language tags, etc.?
I agree. There is a widespread understanding that these X-things don't get
standardized as such.
Pete Resnick
On 1/2/04 at 10:02 AM -0500, Keith Moore wrote:
>why not post the revised ABNF as an I-D so that others can go over
>it with a fine tooth comb?
Anyone done so yet? Keith? Charles? Anyone?
Charles Lindsey
In <p0610073bbc2636eb4ba8@[216.43.25.67]> Pete Resnick <pres...@qualcomm.com> writes:
>On 1/2/04 at 10:02 AM -0500, Keith Moore wrote:
>>why not post the revised ABNF as an I-D so that others can go over
>>it with a fine tooth comb?
>Anyone done so yet? Keith? Charles? Anyone?
Sorry. Not enough Round Tuits yet.
Arnt Gulbrandsen
Charles Lindsey writes:
>>> why not post the revised ABNF as an I-D so that others can go over
>>> it with a fine tooth comb?
>>
>> Anyone done so yet? Keith? Charles? Anyone?
>
> Sorry. Not enough Round Tuits yet.
FWIW, I plan on doing it on Wednesday.
Arnt
Bruce Lilly
Bruce Lilly wrote:
>
> Here's another take on the subject (I hope it doesn't open up several
> cans
> of worms, but here goes...)
>
> X- is used as an indicator for experimental or private-use tags in
> many places
> in Internet message protocols other than field names. For example:
[...]
> So what's so abhorrent about top-level X- fields (RFC 822) that
> doesn't also apply to
> charsets, media types, language tags, etc.?
I didn't intend that to stifle discussion, but I haven't seen any
responses from the anti-X-
community. I did intend it to be a moderately strong case in support of
X- fields (by
way of asking what's so bad about them that differs from charsets,
language tags, etc.).
I don't want to read too much into that dearth of responses -- any comments?
ned+ie...@mrochek.com
> Bruce Lilly wrote:
> [...]
In the case of media types, x- values have proved to be a very poor idea, in
that their use has in some cases prevented registration of types that really
needed to be properly registered. And note that media types are, generally
speaking, something that implementations support through the use of a
configurable table, which means the adverse impact on interoperability of
multiple labels is much, much lower than something that tends to be embedded
in program code. However, abandoning the x- distinction in the case of media
types is highly problematic at this point because of the political issues
surrounding the registration process and the history of its development. Header
fields do not have a registry at present and hence do not have this problem.
I haven't seen significant use of x- types in either charsets or language tags.
In the case of charsets the early registration of a massive number of charsets
is likely what prevented this from happening, along with the willingness,
right or wrong, to simply use non-x- names without registration. Language tags
started out with a reasonably comprehensive set of initial registrations and a
reasonably timely and fair review and approval process. (Of course there's also
the fact that defining a new language is a much bigger deal than coming up with
new item for most of of the other registries we have.)
I can argue the effects of x- tags in the content-transfer-encoding
space either way. On the one hand, the ability to use various x- tags
for various uuencode variants made it possible to use uuencode as
transitional encoding on the way to getting base64 support in place. On
the other hand, the lack of a standarized label did cause fairly
nontrivial interoperability problems along the way. Does the end justify
the means?
The bottom line is that the success of x- fields in other places has anywhere
from disasterous at worst to mixed at best. And to a large extent the lack of
harm in many spaces has come from lack of use. So if you're looking for support
for keeping the X- distinction from the successful use of X- in other contexts,
I'm afraid it is not forthcoming.
Ned
pbdl...@pinboard.com
On Sun, Jan 18, 2004 at 08:49:42AM -0800, ned+ie...@mrochek.com wrote:
> > Bruce Lilly wrote:
>
> > > X- is used as an indicator for experimental or private-use tags in
> > > many places
> > > in Internet message protocols other than field names. For example:
> > [...]
> [...]
> The bottom line is that the success of x- fields in other places has anywhere
> from disasterous at worst to mixed at best. And to a large extent the lack of
> harm in many spaces has come from lack of use. So if you're looking for support
> for keeping the X- distinction from the successful use of X- in other contexts,
> I'm afraid it is not forthcoming.
I don't quite understand why the X- headers or other similar X- stuff
must be successful. Aren't these simply meant for 'private', experimental
and test use? If I want to test out a new idea, if I want _my_ mailing
system A and _my_ mailing system B and _my friends_ mailing system C to
do something special, if I want to flag certain messages for special
handling in _my_ environment, then I'll use X- headers. All this may be
highly successful for me, but not for the world at large.
If some header-based feature or interoperation should work on a large
scale, not just in the environment I control, then I will need to define
and describe a non-X- header for it. The X- prefix only gives me
assurance that my experimental or internal stuff is not clashing with
any currently defined, widely used header, now or in the future. The X-
header will not even guarantee that it will work through externally
controlled gateways, even though most of the time it will.
Anybody who uses X- headers or tags and expects large scale
interoperation is simply naive. I can easily make a rule that in my huge
private garden cars are to drive backwards. If I want them to do the
same on public roads, well, good luck.
I like the comparison between X- and 192.168.0.x. I can create a huge
internal network with 192.168.0.x IP addresses, I can even tunnel the
network over the public internet. This can all work perfectly well. But
I can not expect it to work outside of my private network, neither is it
accessible from the internet as is, nor is it guaranteed that it can
work with another private network using the same address range.
I thought RFC822 was quite clear about this:
4.7.4
[...]
To provide
**user-defined**
fields with a measure of safety, in name selection, such
**extension-fields will never**
have names that
**begin with the string "X-".**
[...]
4.7.5
[...]
Note: The prefatory string "X-" will never be used in the
names of Extension-fields. This provides user-defined
fields with a protected set of names.
Kurt
ned+ie...@mrochek.com
> On Sun, Jan 18, 2004 at 08:49:42AM -0800, ned+ie...@mrochek.com wrote:
> > > Bruce Lilly wrote:
> >
> > > > X- is used as an indicator for experimental or private-use tags in
> > > > many places
> > > > in Internet message protocols other than field names. For example:
> > > [...]
> > [...]
> > The bottom line is that the success of x- fields in other places has anywhere
> > from disasterous at worst to mixed at best. And to a large extent the lack of
> > harm in many spaces has come from lack of use. So if you're looking for support
> > for keeping the X- distinction from the successful use of X- in other contexts,
> > I'm afraid it is not forthcoming.
> I don't quite understand why the X- headers or other similar X- stuff
> must be successful.
Think for a moment about what you're saying. You're saying that it doesn't
matter if a particular feature of a specification has caused interoperability
problems in other contexts and is known to have caused severe interoperability
problems in the present context.
I'm sorry, but the entire point of this exercise is to produce interoperable
standards. The question was whether or not X- fields have succeeded in
not causing interoperability issues in other contexts. And the answer
is that in the cases where they have seen active use they have NOT
succeeded in this regard.
> Aren't these simply meant for 'private', experimental
> and test use? If I want to test out a new idea, if I want _my_ mailing
> system A and _my_ mailing system B and _my friends_ mailing system C to
> do something special, if I want to flag certain messages for special
> handling in _my_ environment, then I'll use X- headers. All this may be
> highly successful for me, but not for the world at large.
The problem is that often as not such experiments do not stay confined,
and instead deploy more widely.
> If some header-based feature or interoperation should work on a large
> scale, not just in the environment I control, then I will need to define
> and describe a non-X- header for it.
Which doesn't happen in practice.
> The X- prefix only gives me
> assurance that my experimental or internal stuff is not clashing with
> any currently defined, widely used header, now or in the future. The X-
> header will not even guarantee that it will work through externally
> controlled gateways, even though most of the time it will.
No such guarantee is present in practice, since any number of
X- fields are now in wide use.
> Anybody who uses X- headers or tags and expects large scale
> interoperation is simply naive. I can easily make a rule that in my huge
> private garden cars are to drive backwards. If I want them to do the
> same on public roads, well, good luck.
On the contrary, the person who is being naive here is you. You're expecting
people to play by the rules when the rules require them to behave in seriously
inconvenient ways. We have ample evidence this is not how things play out. The
trick is to write rules that people will actually follow.
> I like the comparison between X- and 192.168.0.x. I can create a huge
> internal network with 192.168.0.x IP addresses, I can even tunnel the
> network over the public internet. This can all work perfectly well. But
> I can not expect it to work outside of my private network, neither is it
> accessible from the internet as is, nor is it guaranteed that it can
> work with another private network using the same address range.
First, its not a valid comparison since such addresses are unroutable and hence
are unusable on a wide scale. No such restrictions apply to X- fields; message
transfer agents do not drop them.
Second, the fact of the matter is that private use addresses of this sort
routinely leak outside of local networks and cause all sorts of problems when
they do. In particular, the number of UDP packets that arrrive at various
destinctions on the Internet with these sorts of IP addresses in the sender
field is enormous, as is the amount of bandwidth that they waste.
Ned
Keith Moore
> > I don't quite understand why the X- headers or other similar X-
> > stuff must be successful.
>
> Think for a moment about what you're saying. You're saying that it
> doesn't matter if a particular feature of a specification has caused
> interoperability problems in other contexts and is known to have
> caused severe interoperability problems in the present context.
Misuse of any protocol feature can cause interoperability problems.
That is at least as true for X- fields (which IMHO should never have
been put in shipping products except perhaps as write-only fields) as
for non X- fields (which should never be in shipping products without
public vetting).
The burden for those who want X- fields to be treated just as any other
field is to convincingly argue that there would be greater overall
interoperability by doing that. Making it easier for people to follow
the rules doesn't help if those rules don't encourage interoperability.
> I'm sorry, but the entire point of this exercise is to produce
> interoperable standards. The question was whether or not X- fields
> have succeeded in not causing interoperability issues in other
> contexts. And the answer is that in the cases where they have seen
> active use they have NOT succeeded in this regard.
A better question is whether treating X- fields differently (in that
they cannot be registered) has improved the overall level of
interoperability over what it would have been to not have them treated
differently. I don't think we can answer this with a great deal of
confidence, since we don't have a control. At one time it was widely
believed that providing a special, reserved space of protocol fields
would discourage pollution of the normal space. Personally I'm glad
that we encouraged people to use X- because I think that if those fields
didn't use X- there would be a lot more bogus and poorly-defined
fields out there and it would be more difficult to distinguish those
fields from fields that should be implemented than it is now. Which is
not to say that there aren't bogus non-X- fields out there, just that
X- helps identify _some_ of the bogus fields.
Most of the X- fields that have been deployed shouldn't have been
deployed in shipping product with or without the X-. Furthermore,
there has never been a prohibition on defining new fields without X-.
So I have a hard time understanding how writing new RFCs that change the
rules for X- helps interoperability, when the real issue is the failure
to define the fields in a way that allows them to interoperate. More
generally, when the problem is due to the failure to read existing RFCs,
I fail to see how writing new RFCs is going to help.
(Granted, it's easier to get people to follow rules that say "anything
goes", because they don't have to know the rules to follow them. But
while "anything goes" might result in more widespread compliance, it
doesn't necessarily result in greater interoperability.)
> > Aren't these simply meant for 'private', experimental
> > and test use? If I want to test out a new idea, if I want _my_
> > mailing system A and _my_ mailing system B and _my friends_ mailing
> > system C to do something special, if I want to flag certain messages
> > for special handling in _my_ environment, then I'll use X- headers.
> > All this may be highly successful for me, but not for the world at
> > large.
>
> The problem is that often as not such experiments do not stay
> confined, and instead deploy more widely.
But at least they're labelled as experimental. (or more precisely,
they're labelled as "user-defined" fields that cannot be registered.)
Even though I believe X- fields should not be used in shipping products,
labelling an unvetted field with X- seems to be at least slightly more
responsible than not labelling it in that way.
> > If some header-based feature or interoperation should work on a
> > large scale, not just in the environment I control, then I will need
> > to define and describe a non-X- header for it.
>
> Which doesn't happen in practice.
And it still won't happen if the treatment of X- fields is changed.
> > The X- prefix only gives me
> > assurance that my experimental or internal stuff is not clashing
> > with any currently defined, widely used header, now or in the
> > future. The X- header will not even guarantee that it will work
> > through externally controlled gateways, even though most of the time
> > it will.
>
> No such guarantee is present in practice, since any number of
> X- fields are now in wide use.
It is certainly true that some X- fields are in wide use. Or more
generally, some widely deployed implementations of mail tools are paying
attention to unregistered fields, some of which begin with X-.
> > Anybody who uses X- headers or tags and expects large scale
> > interoperation is simply naive. I can easily make a rule that in my
> > huge private garden cars are to drive backwards. If I want them to
> > do the same on public roads, well, good luck.
>
> On the contrary, the person who is being naive here is you.
No, he's being realistic, and more folks need to recognize this.
More specifically, anyone who expects fields that aren't well defined
and publically vetted to interoperate on a wide scale is being naive.
It might happen by accident, but not in general.
Use of X- fields is just a special case of the above.
> You're
> expecting people to play by the rules when the rules require them to
> behave in seriously inconvenient ways. We have ample evidence this is
> not how things play out. The trick is to write rules that people will
> actually follow.
Indeed. It seems that there's no getting around these two things:
1. You don't tend to get widespread interoperability without careful
design that takes into account a wide range of input (because conditions
vary widely and no single person or group appreciates the breadth of
conditions).
2. Getting that wide range of input (typically by public review and
vetting) is "seriously inconvenient" - at least, in comparison to
how much effort it takes to type
printf ("field-name: ...\r\n");
Frankly I don't think we're going to improve interoperability without
somehow encouraging more people to do things that are inherently
"seriously inconvenient". Maybe we can make them more convenient, but
it will never be anywhere nearly as easy as typing printf.
> > I like the comparison between X- and 192.168.0.x. I can create a
> > huge internal network with 192.168.0.x IP addresses, I can even
> > tunnel the network over the public internet. This can all work
> > perfectly well. But I can not expect it to work outside of my
> > private network, neither is it accessible from the internet as is,
> > nor is it guaranteed that it can work with another private network
> > using the same address range.
>
> First, its not a valid comparison since such addresses are unroutable
> and hence are unusable on a wide scale.
A better analogy would be to the new kinds of addressing being proposed
for IPv6, which _are_ globally unique (at least with high probability)
_aren't_ publically routable (since they aren't aggregatable)
but _can_ still be distinguished from publically routable addresses.
(I call them GUPIs - globally-unique provider-independent addresses,
but this name hasn't caught on.) One of the arguments that have been
made in favor of deprecating site-locals in favor of GUPIs is that,
while both kinds of addresses would leak, the GUPIs would at least be
recognizable as "addresses that don't belong here" (say in routing
advertisements) and thus you can reliably distinguish them from
legitimate traffic and (at least for registered GUPI prefixes)
you might also be able to tell where the traffic is coming from.
But this is where the analogy falls apart, because for the case of
non-local GUPIs we might want to filter them, for the case of unknown
extension headers we might want to implement that feature rather than
ignore it.
--
I really think that the X- debate is a red herring. The reason we
seem to always be at an impasse on the X- debate is that we're
dancing around the real problem, which really has not much at all
to do with X- . The problem, I suspect, is something like this:
In the marketplace, vendors distinguish between each other on features,
and some features require protocol extensions. Standardizing those
protocol extensions before deployment defeats the vendor's purpose for
deploying those extensions, which was to gain some (however limited in
time) edge over a competitor. So it's not going to happen very often.
But when such features are deployed they may not interoperate well with
other vendors' products because of inadequate design and/or lack of a
well-written published specification, and they may actually degrade
interoperability of the email service in general.
So the question for those who want to promote interoperability is:
How do we encourage experimentation with, and deployment of, new
features in email without degrading interoperability?
ned+ie...@mrochek.com
> > > I don't quite understand why the X- headers or other similar X-
> > > stuff must be successful.
> >
> > Think for a moment about what you're saying. You're saying that it
> > doesn't matter if a particular feature of a specification has caused
> > interoperability problems in other contexts and is known to have
> > caused severe interoperability problems in the present context.
> Misuse of any protocol feature can cause interoperability problems.
> That is at least as true for X- fields (which IMHO should never have
> been put in shipping products except perhaps as write-only fields) as
> for non X- fields (which should never be in shipping products without
> public vetting).
But it is far from clear that the cases where X- field use is causing interop
problems actually constitute "misuse". For example, there have been cases where
two parties opted to use the same X- mail header field for their own private,
but sematically different, purposes. (X-Sender is the example that comes to
mind.) The specification of X- in RFC 822 doesn't give you much leverage to
argue that either of them were "misusing" the field.
Another, similar, case is the X-uuencode CTE. (And before you say that the
right answer is to use base64 and not uuencode, the choice in many cases isn't
between base64 or uuencode, but between embedding uuencode material in text and
calling it out in the MIME structure. In such cases I'll take the latter every
time.)
> The burden for those who want X- fields to be treated just as any other
> field is to convincingly argue that there would be greater overall
> interoperability by doing that. Making it easier for people to follow
> the rules doesn't help if those rules don't encourage interoperability.
I see it the other way around. Given that interop problems are known
to exist the burden should instead be on the proponents of X- to
make a case for the continued inclusion of the feature being worth it.
Less is usually more as far as standards go.
> > I'm sorry, but the entire point of this exercise is to produce
> > interoperable standards. The question was whether or not X- fields
> > have succeeded in not causing interoperability issues in other
> > contexts. And the answer is that in the cases where they have seen
> > active use they have NOT succeeded in this regard.
> A better question is whether treating X- fields differently (in that
> they cannot be registered) has improved the overall level of
> interoperability over what it would have been to not have them treated
> differently. I don't think we can answer this with a great deal of
> confidence, since we don't have a control.
True enough. However, the question that started this thread was how well the
use of X- fields have worked in other contexts. And although it is far from
clear the results elsewhere are relevant, the fact is that such usage has not
turned out well.
> At one time it was widely
> believed that providing a special, reserved space of protocol fields
> would discourage pollution of the normal space. Personally I'm glad
> that we encouraged people to use X- because I think that if those fields
> didn't use X- there would be a lot more bogus and poorly-defined
> fields out there and it would be more difficult to distinguish those
> fields from fields that should be implemented than it is now. Which is
> not to say that there aren't bogus non-X- fields out there, just that
> X- helps identify _some_ of the bogus fields.
First, since there are huge numbers of bogus header fields (I have 587 on my
personal list of such things, about evenly split in their use or non-use of X-)
both with and without X-, I don't buy the argument that X- has helped in any
material way.
Second, the existance of X- has led to the use of well-defined fields in the X-
space that now cannot be standardized because of the X- rule. The best example
of this I know of is X-Accept-Language but I'm sure there are others.
I've also seen cases of fields where the contents are reasonably well defined
but the field names vary, to the detriment of interoperabilty. For example, it
is often useful for different gateways to the same type of non-Internet mail
system to cooperate in their specification of a fields used to carry
information useful to the non-Internet system they gateway to. (X-VMSMail-To:
is one example of this, there are many others.) Although there tend to be
scoping issues associated with the use of such information, the fact remains
that it would be very useful for vendors of different gateways to the same
system to be able to agree on a single field name. But the use of X- fields for
this sort of stuff (which does seem like an obvious choice) has precluded
specification of such field names in these cases.
> Most of the X- fields that have been deployed shouldn't have been
> deployed in shipping product with or without the X-.
Agreed, but this doesn't make the cases where something was improperly
deployed with X- go away.
> Furthermore,
> there has never been a prohibition on defining new fields without X-.
Which further dilutes the need for X- IMO.
> So I have a hard time understanding how writing new RFCs that change the
> rules for X- helps interoperability, when the real issue is the failure
> to define the fields in a way that allows them to interoperate. More
> generally, when the problem is due to the failure to read existing RFCs,
> I fail to see how writing new RFCs is going to help.
Sure this is a problem but this isn't the problem getting rid of X- is intended
to solve.
> (Granted, it's easier to get people to follow rules that say "anything
> goes", because they don't have to know the rules to follow them. But
> while "anything goes" might result in more widespread compliance, it
> doesn't necessarily result in greater interoperability.)
This is why we're trying the provisional registry approach.
> > > Aren't these simply meant for 'private', experimental
> > > and test use? If I want to test out a new idea, if I want _my_
> > > mailing system A and _my_ mailing system B and _my friends_ mailing
> > > system C to do something special, if I want to flag certain messages
> > > for special handling in _my_ environment, then I'll use X- headers.
> > > All this may be highly successful for me, but not for the world at
> > > large.
> > The problem is that often as not such experiments do not stay
> > confined, and instead deploy more widely.
> But at least they're labelled as experimental. (or more precisely,
> they're labelled as "user-defined" fields that cannot be registered.)
But there's often no truth in that label. This is why having a provisional
registry is a better answer.
> Even though I believe X- fields should not be used in shipping products,
> labelling an unvetted field with X- seems to be at least slightly more
> responsible than not labelling it in that way.
I'm sorry, but this simply isn't a realistic approach for vendors to take.
> > You're
> > expecting people to play by the rules when the rules require them to
> > behave in seriously inconvenient ways. We have ample evidence this is
> > not how things play out. The trick is to write rules that people will
> > actually follow.
> Indeed. It seems that there's no getting around these two things:
> 1. You don't tend to get widespread interoperability without careful
> design that takes into account a wide range of input (because conditions
> vary widely and no single person or group appreciates the breadth of
> conditions).
> 2. Getting that wide range of input (typically by public review and
> vetting) is "seriously inconvenient" - at least, in comparison to
> how much effort it takes to type
> printf ("field-name: ...\r\n");
> Frankly I don't think we're going to improve interoperability without
> somehow encouraging more people to do things that are inherently
> "seriously inconvenient". Maybe we can make them more convenient, but
> it will never be anywhere nearly as easy as typing printf.
This is precisely the problem the registry proposal is trying to address. It
also tries to address the related problem that it is far better to know what
the intent of a given field usage is, even when that intent is flawed in some
way, than it is to be forced to guess. (Again I will point out that simply
ignoring these things is just not an option for vendors -- if they adopt this
attitude their customers will simply go elsewhere for their messaging
products.)
> I really think that the X- debate is a red herring. The reason we
> seem to always be at an impasse on the X- debate is that we're
> dancing around the real problem, which really has not much at all
> to do with X- . The problem, I suspect, is something like this:
It certainly is red herring in regards to the topic you're now pursuing.
> In the marketplace, vendors distinguish between each other on features,
> and some features require protocol extensions. Standardizing those
> protocol extensions before deployment defeats the vendor's purpose for
> deploying those extensions, which was to gain some (however limited in
> time) edge over a competitor. So it's not going to happen very often.
> But when such features are deployed they may not interoperate well with
> other vendors' products because of inadequate design and/or lack of a
> well-written published specification, and they may actually degrade
> interoperability of the email service in general.
This is probably true in some cases. However, I can also think of many cases
where a vendor would have had no problem with specification of a field prior to
or coincident with deployment, but was thwarted by the daunting process of
publishing an RFC. (There have even been cases where drafts were published but
then languished because the necessary energy for pushing the document through
the process just wasn't there.) Heck, I've done this myself on several
occasions, and if I opted for this approach, what does this say about someone
who knows nothing about IETF processes?
> So the question for those who want to promote interoperability is:
> How do we encourage experimentation with, and deployment of, new
> features in email without degrading interoperability?
The best tack we've been able to come with is what's in the registry
specification. Of course it remains to be seen if it will actually help.
But I agree that the X- issue is small beer in this larger context.
Ned
Dave Crocker
>> Misuse of any protocol feature can cause interoperability problems.
ni8mc> But it is far from clear that the cases where X- field use is causing interop
ni8mc> problems actually constitute "misuse".
In fact I would say that this is an example of using the "protocol" exactly the way it
was intended, with an undesired, but entirely predictable, effect.
Alas, sometimes excellent intentions and even good designing/thinking
can produce very bad results. We simply have to acknowledge the outcome
and find a way to fix it.
When we added X-headers, I thought it was a very clever idea, indeed.
Instead it has proved both problematic and aesthetically ugly. (Since
email headers are user-visible, aesthetics matter, at least somewhat.)
My own, primary concern about them is having an "experimental" header
become a defacto standard and then having to find a way to get folks to
use a new, non-X header name for it. I believe that hasn't been
successful yet, or at least not much.
d/
d/
--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <www.brandenburg.com>
Sunnyvale, CA USA <tel:+1.408.246.8253>
ned+ie...@mrochek.com
> >> Misuse of any protocol feature can cause interoperability problems.
> ni8mc> But it is far from clear that the cases where X- field use is causing interop
> ni8mc> problems actually constitute "misuse".
> In fact I would say that this is an example of using the "protocol" exactly the way it
> was intended, with an undesired, but entirely predictable, effect.
> Alas, sometimes excellent intentions and even good designing/thinking
> can produce very bad results. We simply have to acknowledge the outcome
> and find a way to fix it.
> When we added X-headers, I thought it was a very clever idea, indeed.
> Instead it has proved both problematic and aesthetically ugly. (Since
> email headers are user-visible, aesthetics matter, at least somewhat.)
> My own, primary concern about them is having an "experimental" header
> become a defacto standard and then having to find a way to get folks to
> use a new, non-X header name for it. I believe that hasn't been
> successful yet, or at least not much.
This is my major concern with them as well.
Ned
Al Costanzo
A clean, simple, fast way to register the non X- header field would do the
trick..
Al
Keith Moore
> My own, primary concern about them is having an "experimental" header
> become a defacto standard and then having to find a way to get folks to
> use a new, non-X header name for it. I believe that hasn't been
> successful yet, or at least not much.
why does it work better in the case of Experimental vs Standard MIBs?
(or do you think that doesn't work well either?)
ned+ie...@mrochek.com
First, I don't know that it works better. I suspect it does not, and that once
an experimental MIB is in use it is very difficult to get rid of it. Indeed, if
such transitions worked well, why isn't the accepted practice for MIBs
developed in the IETF to assign an experimental OID during development and
switch to a standard one after IESG approval? It sure isn't done this way -
what happpens in practice is that there is NO OID assigned in the document
until after it is approved.
But even if this approach worked well for MIBs, the cases aren't even remotely
comparable. The difference between a MIB registered in the experimental space
and a MIB registered in the standards space is simply the OID prefix. That's
almost certainly something you can configure on a management station and in
some cases on the agent side as well. I don't believe I've ever seen a mail
system component with comparable flexibility in regards to header fields.
Additionally, nobody runs around, management station in hand, and tries to
access random agents on the network. (Well, perhaps people actually do do this,
but I'll bet that when they do they are crackers and are more interested in
exploiting one of the many ASN.1 parser buffer overrun bugs than in performing
any sort of useful management function.) Agents and managements stations tend
to be operated by people in the same administrative domain and a given agent
is likely to only be probed by a small number of management stations. (I'm
confident the most common number is 0, then 1, with numbers greater than 1 a
very distant third.) This means that OID mismatches present a trivial
problem compared to header field mismatches.
Finally, anyone who thinks SNMP isn't bothered by interop problems of various
sorts hasn't spent much time using it. As it happens I have (a) Developed
standards-track MIBs, (b) Written both agent code and agent plugin code, (c)
Written management station plugins, and (d) Use a fairly sophisticated SNMP
monitoring setup to keep tabs on my home network. And I've run into plenty of
problems along the way, including but not limited to: Lack of specificity in
widely used standard MIBs causing interop problems, experimental MIBs
constructed in extremely bizarre ways that are very difficult to build probes
for (if anyone knows how to construct a probe that's able to see Mac OS X
systems connected to an original series Apple Airport please let me know),
plugin architectures that are inherently incompatible with the design of some
MIBs, and fun and games with row creation. And then there's Agent X. You don't
want to get me started on that...
Ned
P.S. An interesting test will be to see if once CAPWAP gets going whether
it can produce standard MIBs to replace the experimental MIBs in use
in existing wireless products. I am not optimistic.
Keith Moore
>> > My own, primary concern about them is having an "experimental"
>> header
>> > become a defacto standard and then having to find a way to get
>> folks to
>> > use a new, non-X header name for it. I believe that hasn't been
>> > successful yet, or at least not much.
>
>> why does it work better in the case of Experimental vs Standard MIBs?
>> (or do you think that doesn't work well either?)
>
> First, I don't know that it works better. I suspect it does not, and
> that once
> an experimental MIB is in use it is very difficult to get rid of it.
> Indeed, if
> such transitions worked well, why isn't the accepted practice for MIBs
> developed in the IETF to assign an experimental OID during development
> and
> switch to a standard one after IESG approval? It sure isn't done this
> way -
> what happpens in practice is that there is NO OID assigned in the
> document
> until after it is approved.
I think that's a change since I was on IESG - I remember having the
discussion about having to change OIDs from the experimental branch to
the standard branch more than once. Maybe not having any OID assigned
until standardization (presumably each vendor assigns its own for
prototyping) is a newer strategy.
> The difference between a MIB registered in the experimental space
> and a MIB registered in the standards space is simply the OID prefix.
That's assuming, of course, that the MIB didn't get changed somehow
when moving to standards-track. If experimental MIBs were often good
enough to standardize as-is, I wouldn't really see the point in
changing prefixes. I don't really see why it matters which OID prefix
is used for any particular MIB, whether it's standards track or not.
But some people seemed to care.
I'm a lot more concerned about email message headers that are
human-readable, because I suspect a lot more people guess semantics
from looking at message header field names than guess semantics by
looking at MIB OID prefixes.
> Additionally, nobody runs around, management station in hand, and
> tries to
> access random agents on the network.
You mean there aren't any management stations doing autodiscovery?
It's been awhile since I looked at any SNMP work but it seems like such
features were all the rage a few years back.
> Finally, anyone who thinks SNMP isn't bothered by interop problems of
> various
> sorts hasn't spent much time using it.
I would never claim that :) I was just wondering whether experience
with SMTP moving from experimental to standards-track OID prefixes
would give some additional clue about the likely effect of moving from
X- to non-X- in email.
Paul Smith
At 01:21 23/01/2004, you wrote:
>A clean, simple, fast way to register the non X- header field would do the
>trick..
Almost certainly, yes.
The primary reason "good idea" header fields don't get made into standards
is probably that people can't register them. Trying to get a standard
approved seems to be a long, arduous task, with lots of heads meeting lots
of walls.
There are quite a few non-standard headers (X- ones as well as non-X- ones)
out there which seem to be good ideas to some people - but maybe not
everyone agrees (eg Delivered-To, Priority, X-RCPTTO etc). I'd actually bet
that trying to get those through to be standards would be a nightmare, even
though it'd be nice for many people if they were standards (without the X-
in the case of X-RCPTTO), so, people use them, and try to guess what they
should mean, and how they should be used. I've seen many different ways
that all those headers are used.
This doesn't argue against X- headers. It just argues against having useful
non-standard headers. One way to solve the problem is to stop people using
them. The other (better, IMHO) way is to make it easier for people to make
them into standard headers. If people are trying to use non-standard
headers they're not creating themselves, then it means they're useful
(otherwise people wouldn't be trying to use them)...
Paul Smith
At 17:46 22/01/2004, Keith Moore wrote:
>Misuse of any protocol feature can cause interoperability problems.
>That is at least as true for X- fields (which IMHO should never have
>been put in shipping products except perhaps as write-only fields) as
>for non X- fields (which should never be in shipping products without
>public vetting).
Hmm.
I'm one of the people who's moderately happy with X- fields when used
properly (as opposed to using a non-standard non-X- field)
But, I could very easily be convinced that X- fields should be write-only
in shipping products. But, I'd prefer to see that than people using
non-standard non-X- fields, or not being able to add any field without
public vetting.
Having a computer readable X- field is 'dangerous' IMHO - for the same
reason that using a non-standard non-X- field would be - interoperability.
Having write-only X- fields doesn't have anywhere near the same amount of risk.
So, I'd be happy to see a 'X- fields SHOULD be write-only' type clause in
the standard.
Paul Smith
At 00:43 23/01/2004, Dave Crocker wrote:
>My own, primary concern about them is having an "experimental" header
>become a defacto standard and then having to find a way to get folks to
>use a new, non-X header name for it. I believe that hasn't been
>successful yet, or at least not much.
I think it is arguable that you need some method of having 'experimental'
headers 'in the wild'.
Getting rid of X- headers and saying 'you must come up with an agreed
standard before you can add a new header of any sort' is going to inhibit
innovation.
At least if you use X- headers, and they become defacto standards, then you
can tell people off ;-)
One example I can think of of standards changing and people keeping up is
with SMTP authentication. In the initial drafts of SMTP authentication the
server would say 'AUTH=LOGIN'. In the final standard, the server says 'AUTH
LOGIN'. Lots of people used the initial draft because authentication was
needed so much, but then changed it to AUTH LOGIN (or supported both, in
many cases) when the final standard came out.
I don't see a big problem with that. Yes, it would have been nice to get it
right the first time, but if people hadn't been willing to risk getting it
wrong and then having to put it right, we might never have ended up with an
SMTP authentication standard!
If you look at other situations, eg the extra trace information that people
are discussing in the SMTP groups, it's quite possible that someone will
come up with a workable idea for this and then people may implement it, and
later problems may be found, and the information will have to change.
If that's done, then either we'd have to use a header like 'Originator',
and then risk total inoperability by changing the meaning of that header,
or make 'OriginatorV2', 'OriginatorV3' etc when the changes happen, and not
be able to use the nice 'Originator' ever again, because that's already
been used.
Or, we could use 'X-Originator', then 'X-Originator2' etc, but then, when
everyone's happy with the idea, we could use 'Originator', and deprecate
the 'X-Originator' ones. Over time people would move to the Originator
header (they may still READ the X-Originator ones, but wouldn't write them)
(This is just an example - yes, it would be nice to get it right first
time, but that doesn't always happen cf SMTP authentication).
Bruce Lilly
Al Costanzo wrote:
>A clean, simple, fast way to register the non X- header field would do the
>trick..
>
>Al
>
>
Is that possible? Ignoring "fast" for the moment, my definition of
"clean" is that the
registration requires specification of semantics and syntax, including ABNF.
That may conflict with somebody else's definition of "simple".
Bruce Lilly
ned+ie...@mrochek.com wrote:
> The bottom line is that the success of x- fields in other places has
> anywhere
> from disasterous at worst to mixed at best. And to a large extent the
> lack of
> harm in many spaces has come from lack of use. So if you're looking for
> support
> for keeping the X- distinction from the successful use of X- in other
> contexts,
> I'm afraid it is not forthcoming.
For the record, I posed the question because there was a great deal of discussion regarding
X- header fields, but I can recall no recent discussion about other use of x- for experimental
and private use; as I mentioned, there are many instances of such use in text messaging
protocols (I was a little surprised to see how many, and my list was not intended to be
exhaustive -- I simply listed the ones that I had implemented in one form or another). That
struck me as curious, hence my question about what was so unique to X- header fields that
caused so much discussion, whereas there was virtually no discussion about other use of x-
for similar purposes. And if indeed use of x- in other places has been disastrous, why isn't
there a call for its removal from those places?
Paul Smith
> >A clean, simple, fast way to register the non X- header field would do the
> >trick..
> >
>Is that possible? Ignoring "fast" for the moment, my definition of
>"clean" is that the registration requires specification of semantics and
>syntax, including ABNF.
>
>That may conflict with somebody else's definition of "simple".
Well, I'd expect a specification of semantics, syntax and an ABNF
definition. That doesn't stop it being relatively simple (with guidance notes).
What would make it slow is an extensive 'approval' system as there is now.
What I'd quite like is a way of registering a header field (including basic
semantics, syntax and an ABNF definition (for many existing X- fields this
could be trivial with some examples to work from)). This would go to a
group of 'learned volunteers' who make sure it's not abusive (ie not
'grabbing' a useful header field name for an inappropriate purpose, or not
commercially advantagous to someone to the detriment of others etc) and
that's it, registered. The 'learned volunteers' may also give feedback to
the registrant saying something like 'we think this is a really good idea,
but needs developing in such and such a way', and then they reserve the
header field name, to give the full registration time to be improved.
An automated register might then allow people to register 'interest' in a
registered header field if they are MTA/MUA developers (eg 'we implement
this', 'we plan to implement it', 'we hate it and think the author ought to
be shot', etc) so that other potential users of the field would know
whether it's worth bothering about.
JM2P
(I'm sure we've been through all this a year or two ago..)
Bruce Lilly
ned+ie...@mrochek.com wrote:
>But it is far from clear that the cases where X- field use is causing interop
>problems actually constitute "misuse". For example, there have been cases where
>two parties opted to use the same X- mail header field for their own private,
>but sematically different, purposes. (X-Sender is the example that comes to
>mind.) The specification of X- in RFC 822 doesn't give you much leverage to
>argue that either of them were "misusing" the field.
>
>
Would the situation have been any better if some other name (i.e. not
beginning
with X-) had been used w/o registration? If there is a conflict in use
of an X-
field, the responsible parties can a) work out the issue and come to common
usage or agree to change one or more of the names, or b) the conflict can
continue. With other field names and with current procedures, there is
another
option, viz. one of the parties can issue an RFC registering the name.
There's no
guarantee that the IETF will be aware of the other use, and it may well
be the
case that a more useful (unregistered) field is ignored. Moreover,
there's no
guarantee that use of the same name will not continue for the unregistered
purpose (and no official means of enforcement). And as has been noted by
others in this discussion, registration without vetting is likely to
make matters
worse, e.g. by polluting the non X- namespace with items of questionable (or
worse) value.
>I see it the other way around. Given that interop problems are known
>to exist the burden should instead be on the proponents of X- to
>make a case for the continued inclusion of the feature being worth it.
>
>
Let's keep things in perspective. Have the interoperability problems
with X- header fields been minor annoyances, or have they caused massive
disruptions? Would things have been any better without X- given the current
realities (e.g. no registration procedure other than publication of an
RFC, no
means of enforcement) and given similar circumstances (i.e. use of the same
field name for different purposes)?
>Less is usually more as far as standards go.
>
>
True, but changing horses in mid-stream is usually a bad idea as well,
in standards
as well as in other endeavors.
>>At one time it was widely
>>believed that providing a special, reserved space of protocol fields
>>would discourage pollution of the normal space. Personally I'm glad
>>that we encouraged people to use X- because I think that if those fields
>>didn't use X- there would be a lot more bogus and poorly-defined
>>fields out there and it would be more difficult to distinguish those
>>fields from fields that should be implemented than it is now. Which is
>>not to say that there aren't bogus non-X- fields out there, just that
>>X- helps identify _some_ of the bogus fields.
>>
>>
>
>First, since there are huge numbers of bogus header fields (I have 587 on my
>personal list of such things, about evenly split in their use or non-use of X-)
>both with and without X-, I don't buy the argument that X- has helped in any
>material way.
>
>
Well you really have two lists of about equal size, according to the RFC
822 rules;
one is a list of bogus (unregistered) fields and the other is a list of
experimental or
private-use (unregisterable) fields. And only one of those lists has
names that could
(under RFC 822 rules) potentially conflict with future registered names.
>Second, the existance of X- has led to the use of well-defined fields in the X-
>space that now cannot be standardized because of the X- rule. The best example
>of this I know of is X-Accept-Language but I'm sure there are others.
>
>
Such well-defined fields, if also widely used, would constitute
successful experiments.
Accept-Language has been standardized (RFC 3282). If it's useful and
widely used
(I'm not convinced of either) those who were sufficiently motivated to
participate in
the experiment are also sufficiently motivated to use the official
field. If so, supporting
that change is not materially different (for parties who elected to
support the X-
experiment) from registered header fields whose names have changed
(e.g. Obsoletes -> Supersedes). For others (who have ignored the
experiment) it's
simply another new header field. And in the specific case of
X-Accept-Language, lest
you look in this message's header and find it there; the transition
gives some time
for broken implementations (such as Mozilla's) to be repaired in the
process.
>I've also seen cases of fields where the contents are reasonably well defined
>but the field names vary, to the detriment of interoperabilty. For example, it
>is often useful for different gateways to the same type of non-Internet mail
>system to cooperate in their specification of a fields used to carry
>information useful to the non-Internet system they gateway to. (X-VMSMail-To:
>is one example of this, there are many others.) Although there tend to be
>scoping issues associated with the use of such information, the fact remains
>that it would be very useful for vendors of different gateways to the same
>system to be able to agree on a single field name. But the use of X- fields for
>this sort of stuff (which does seem like an obvious choice) has precluded
>specification of such field names in these cases.
>
>
I don't get your point. If X-Accept-Language has been sufficiently
successful to result
in a standardized version with a registered name, how do the RFC 822
provisions
preclude a similar thing from happening for gateways? Moreover, there is a
counterexample of sorts; the MIXER RFCs do standardize such things for
gateways
between two specific mail environments.
>>Most of the X- fields that have been deployed shouldn't have been
>>deployed in shipping product with or without the X-.
>>
>>
>
>Agreed, but this doesn't make the cases where something was improperly
>deployed with X- go away.
>
>
>
True, but is does prevent them from clashing with registered field
names, past,
present, and future.
>>Furthermore,
>>there has never been a prohibition on defining new fields without X-.
>>
>>
>
>Which further dilutes the need for X- IMO.
>
>
>
There are some important distinctions to be made here. Of course there's
no prohibition against
registering new field names -- indeed that is explicitly part of the
extension process
provided for by RFC 822. While there is no official means of
enforcement preventing use
of unregistered field names, RFC 822 explicitly cautions that the
implementor who does
so is crawling out onto a limb that may be officially cut off behind
him. And it provides
a guarantee that that will not happen with an X- field.
Perhaps there should have been a prohibition against use of unregistered
non-X- field
names. But such a prohibition would be meaningless in practice without an
enforcement mechanism.
>Sure this is a problem but this isn't the problem getting rid of X- is intended
>to solve.
>
>
Exactly which problem is it intended to solve, and how is it proposed
that it will
do so? Especially given that X- header fields have been around for more
than two
decades, will not magically disappear from "old" stored messages, and
lacking an
effective enforcement mechanism will never disappear from new messages.
>This is why we're trying the provisional registry approach.
>
>
[...]
>The best tack we've been able to come with is what's in the registry
>specification. Of course it remains to be seen if it will actually help.
>But I agree that the X- issue is small beer in this larger context.
>
>
Wouldn't it be preferable to retain X- while that approach is tried;
that way if the
provisional registry is a disaster or fails to achieve it's purported
goals, we'll still
have an established fallback. If X- is somehow abolished and the
provisional registry
fails, then what?
Charles Lindsey
>A clean, simple, fast way to register the non X- header field would do the
>trick..
Which is exactly what the Klyne draft provides.
It is currently stuck in last call in the IESG. They need to get it out
ASAP.
ned+ie...@mrochek.com
> ned+ie...@mrochek.com wrote:
> >But it is far from clear that the cases where X- field use is causing interop
> >problems actually constitute "misuse". For example, there have been cases where
> >two parties opted to use the same X- mail header field for their own private,
> >but sematically different, purposes. (X-Sender is the example that comes to
> >mind.) The specification of X- in RFC 822 doesn't give you much leverage to
> >argue that either of them were "misusing" the field.
> Would the situation have been any better if some other name (i.e. not
> beginning with X-) had been used w/o registration?
Quite possibly. The problem with X- is that it is easy and encourages
unilateral action. Had there been (a) No X- and (b) A simple registration
scheme it is entirely possible that the first user of such a field would have
registered it and subsequent users would have picked up the registration.
And there is precedent for this. When media types were first defined
registration, although possible, was too onerous. This led to a lot of ad-hoc
use. Subsequent revisions to the process have given us a significant rise in
the registration rate. But to the extent X- types were or are still used, they
now constitue a problem rather than a solution.
> If there is a conflict in use of an X-
> field, the responsible parties can a) work out the issue and come to common
> usage or agree to change one or more of the names, or b) the conflict can
> continue.
This assumes that the responsible parties are willing able to get together and work
such things out after the fact. This almost never happens in my
experience.
> With other field names and with current procedures, there is
> another option, viz. one of the parties can issue an RFC registering the name.
There have been attempts to do this in the past. The degree of pushback
experienced killed almost all of them.
> There's no
> guarantee that the IETF will be aware of the other use, and it may well
> be the
> case that a more useful (unregistered) field is ignored. Moreover,
> there's no
> guarantee that use of the same name will not continue for the unregistered
> purpose (and no official means of enforcement). And as has been noted by
> others in this discussion, registration without vetting is likely to make matters
> worse, e.g. by polluting the non X- namespace with items of questionable (or
> worse) value.
The incredible number of unregistered non-X- fields in wide use discredits this
assertion rather completely, I think. And again, I would much rather know how
something was designed to be used, warts and all, than have to guess.
> > I see it the other way around. Given that interop problems are known
> > to exist the burden should instead be on the proponents of X- to
> > make a case for the continued inclusion of the feature being worth it.
> Let's keep things in perspective. Have the interoperability problems
> with X- header fields been minor annoyances, or have they caused massive
> disruptions?
They have on occasion caused massive disruptions. X-PMrpc is an example of a
field that caused serious operational problems at one point; there are others
as well.
> Would things have been any better without X- given the current
> realities (e.g. no registration procedure other than publication of an RFC, no
> means of enforcement) and given similar circumstances (i.e. use of the same
> field name for different purposes)?
This is soon not to be the current reality. A registration proposal is in front
of the IESG now. And it changes things substantially. Without it I see little
point in worrying about X-. But once it is in place I think X- becomes actively
harmful and I don't want to see it reintroduced.
I really didn't intend to be drawn into this whole debate; my only purpose in
posting my initial message was to answer your request for citations of X-
experience in other areas (which we have seen does not exactly constitute a
ringing endorsement of the concept). So I'm cutting this off now and it is
unlikely I will post additional messages on this topic.
Ned
ned+ie...@mrochek.com
> In <005201c3e14f$370ef920$6401...@akc.com> "Al Costanzo" <a...@akc.com> writes:
> >A clean, simple, fast way to register the non X- header field would do the
> >trick..
> Which is exactly what the Klyne draft provides.
> It is currently stuck in last call in the IESG. They need to get it out
> ASAP.
Actually, if you bother to check the datatracker you'll see that the
document is in IESG evaluation. More specifically, a response was
sent to the two ADs with outstanding discuss votes and we're now waiting
for the two ADs to either (a) Clear their discusses or (b) Respond in some
other way.
Ned
ned+ie...@mrochek.com
(one more response since this is back on the original topic of my
message)
> ned+ie...@mrochek.com wrote:
What makes you think there hasn't been such a call? The fact of the matter is
that X- media types have long been hanging by a thread. There most certainly
was a call to take them out at the time RFC 2048 came out, citing interop
problems. They remained only because the text was redone to discourage their
use and this was deemed to be sufficient. I was seriously tempted to remove
them in the recently approved RFC 2048bis but the mandate behind revision was
to make the minimal set of changes needed to open up the IETF tree to other
standards bodies, not to clean up other aspects of the specification. Even so,
there was some mumbling that retaining them wasn't a good idea. I certainly
would not bet the farm on their continued inclusion in any RFC 2048ter,
especially if the recent DNS tree proposal gains any traction.
There have been other, similar queries as to the wisdom of X- fields in other
contexts, but in most cases the lack of use of such fields is cited as a reason
not to change the status quo. The same cannot be said of X- headers, of course.
Ned
Lyndon Nerenberg
--On 2004-1-23 2:33 PM +0000 Paul Smith <pa...@pscs.co.uk> wrote:
> What I'd quite like is a way of registering a header field (including
> basic semantics, syntax and an ABNF definition (for many existing X-
> fields this could be trivial with some examples to work from)). This
> would go to a group of 'learned volunteers' who make sure it's not
> abusive (ie not 'grabbing' a useful header field name for an
> inappropriate purpose, or not commercially advantagous to someone to
> the detriment of others etc) and that's it, registered. The 'learned
> volunteers' may also give feedback to the registrant saying something
> like 'we think this is a really good idea, but needs developing in
> such and such a way', and then they reserve the header field name, to
> give the full registration time to be improved.
"Header hijacking" is also one of my major concerns, and I'm in favour
of what Paul is suggesting.
If X-* dies I suppose the experimental use folks could use something
like a base64-encoded hash of the proposed header name (or any other
method that would generate random strings with a suitably low
probability of namespace collisions).
--lyndon
Lyndon Nerenberg
--On 2004-1-22 5:08 PM -0800 ned+ie...@mrochek.com wrote:
>> My own, primary concern about them is having an "experimental" header
>> become a defacto standard and then having to find a way to get folks
>> to use a new, non-X header name for it. I believe that hasn't been
>> successful yet, or at least not much.
>
> This is my major concern with them as well.
But how often has this even been tried? If there have been *serious*
efforts to attempt this in the past it would be useful if someone could
document in detail the process taken to make the change, and provide an
analysis of why it failed.
--lyndon
Bruce Lilly
Keith Moore wrote:
>
>> Start saying things like that and you'll only encourage me to write
>> the "X- considered harmful" document. ;-)
>
>
> and if you do, I'll write the "X- considered wonderful" document :)
>
> actually I think we could probably find a compromise position that we
> could agree on, as long as we don't make it part of the critical path
> for more important work.
I just skimmed RFC 3692, which appears to be such a document (though
not specifically addressing X- fields).
#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################