application/fs vs. fs/*
Pete Resnick
>Pete writes:
>
>>No. That's where the argument is. MIME already has a great general
>>container structure, i.e., the multipart top-level type
>
>Are we reading the same definition in the same RFC?
>
>Snipped from the RFC:
>
>multipart -- data consisting of multiple entities of independent data types.
>Four subtypes are initially defined, including the basic "mixed" subtype
>specifying a generic mixed set of parts, "alternative" for representing the
>same data in multiple formats, "parallel" for parts intended to be viewed
>simultaneously, and "digest" for multipart entities in which each part has a
>default type of "message/rfc822".
>
>
>Does not fit in my mind. Maybe I'm missing your point...
Continue reading further down. In the digest section for example:
This document defines a "digest" subtype of the "multipart" Content-
Type. This type is syntactically identical to "multipart/mixed", but
the semantics are different. In particular, in a digest, the default
Content-Type value for a body part is changed from "text/plain" to
"message/rfc822". This is done to allow a more readable digest
format that is largely compatible (except for the quoting convention)
with RFC 934.
Note: Though it is possible to specify a Content-Type value for a
body part in a digest which is other than "message/rfc822", such as a
"text/plain" part containing a description of the material in the
digest, actually doing so is undesireble. The "multipart/digest"
Content-Type is intended to be used to send collections of messages.
If a "text/plain" part is needed, it should be included as a seperate
part of a "multipart/mixed" message.
The idea here is that unlike multipart/mixed, which is a general
"collection of stuff", multipart/digest can be used for something that is a
"collection of messages." For example, when Eudora gets a multipart/digest,
it presents the user with a Eudora mailbox containing all of the messages
in the digest. Other applications show a list of the messages that appear
in the digest. All in all, multipart/digest says to the interpreter, "Here
is something that has many parts; that's why it's top-level type is
multipart. Specifically, it is a collection of messages, and should be
treated on the receiving end however it is you treat a collection of
messages."
In the same way, a multipart/fs can say to the interpreter, ""Here is
something that has many parts; that's why it's top-level type is multipart.
Specifically, it is a structured collection of files in the form of a file
system, and should be treated on the receiving end however it is you treat
a collection of files in a file system."
My guess is that Eudora would use exactly the same code it uses now to
split out files of different media types, setting up the Macintosh file
types appropriately, setting creation and modification dates, etc. But with
it marked as a multipart/fs, we would also preserve the tree structure of
the file system, apply attributes to nodes in that tree as specified by the
MIME headers, and allow the user to navigate through them.
It's a very nice mapping. I don't see where you find evidence in the RFC
that something like this wouldn't fit into the model nicely.
pr
--
Pete Resnick <mailto:pres...@qualcomm.com>
QUALCOMM Incorporated
Work: (217)337-6377 or (619)651-4478
Fax: (217)337-1980 or (619)651-1102
Pete Resnick
>Application/FS:
>
>I'd have to disagree, The very fist draft of FS defined it FS _Exactly_
>the way you are describing. I was told and told and told (you get the
>idea).... that Application was definitely the wrong place for the thing
>called "FS"
By whom? I cannot imagine that anyone in the MIME community would have said
such a thing. So far, I have only heard Nathaniel say, "If that's the goal
of the exercise -- i.e. to permit the gradual *evolution* of file system
wrappers -- then a top-level name might make sense. Otherwise, if we're
really trying to standardize on a single one for all time, I'd argue for
application/fs", but I have yet to hear anyone say that application would
be a bad idea.
>Here, I'll quote an RFC, 2046 to be specific:
>
>application -- some other kind of data, typically either uninterpreted
>binary data or information to be processed by an application.
>
>Welp (my word for Well with a sigh),
>
>It's not necessarily binary, it's specific (not some other kind) and most
>likely, if we get FS defined properly will _NOT_ be processed by an external
>application....
Note the word "typically" in the 2046 section you cite. There is nothing
that prevents MIME interpreters from also interpreting the content of
things which are subtypes of application; many mail applications in fact do
just that. But like these other application subtypes, there is no further
*MIME* information to be found in the middle of these FS parts. It is data
that is dealt with outside of the context of this being a MIME message.
Let's take a look at some of the things you say later in your message:
>FS - is like a file archive _BUT_ its not a file archive for the purpose of
>archiving a file system set of objects.
>
>It is a collections of file system objects wrapped in an "FS" container.
>This container is in itself a file system object. "somename.fs"
This is increasing support for it being an application subtype. If FS *is*
a container that is a file system object with no MIME structure in it, then
it is something to be acted upon by a piece of code that considers the
structure of the FS object, not the MIME structure of the message.
>Getting away from an external application is the whole concept of putting FS
>into MIME!
This is where the multipart stuff makes sense, but let's look at that below.
>Therefore, unless someone really convinces me otherwise, to rewrite the
>draft the way it originally was ........
>
>***Application/FS*** is not even going to be thought about (again)
You have yet to present any reasoning other than "it doesn't sound like an
application subtype according to 2046." I've given you reasoning why it
clearly is. Please give some other explanation before you dismiss this
idea. I did not see the discussion regarding the first draft, but I'd like
to hear other explantions that people gave to reject the application
subtype. According to what you've said, you thought it was a good idea too,
so I'd like to hear why you changed your mind (other than, "people said it
a bunch").
>As for Multipart, I have heard from people who want it there Multipart/FS
>and people who say it does not belong.
Again, which people have said it doesn't belong. I'd like to hear their
reasoning.
>In my mind, convince me otherwise, if FS items exist as a file system object
>they are in fact a media type, correct?
Sorry to repeat my point above, but if you are saying that there are
currently existing applications outside of the mail system which interpret
FS items as file system objects, then that justifies it as a subtype of
application, not a top-level type.
>Now, if this media type to be defined is truly going to transport entire
>file system between computers of like and unlike types....
>
>Is it too much to ask for a new type to define this thing?
A new type itself is not a problem. But there is already a great deal of
machinery within MIME to do this sort of thing. If we can reuse this
machinery, it would be much better than reinventing the wheel. Again, I'd
like to hear argument that the current machinery is insufficient for the
task at hand.
>The reason I selected FILE/FS is that the level type FILE was going to
>contain file system objects that are not defined elsewhere in MIME or
>defined well.
Let's start with an easy example: GIF images. GIF images may be stored as
file system objects, or may be dynamically displayed and never stored in a
file system. There are two mechanisms in MIME to deal with GIF images.
First, there is a Content-Type describing the type of data, which in this
case is image/gif. Then there is a Content-Disposition, which describes how
the data is to be dealt with, which might be as an file system object
(i.e., attachment), or as a dynamically displayed image (i.e., inline). It
might have file system attributes, which also appear in the
Content-Disposition, like creation date, etc. MIME deals with these objects
very nicely.
As a second example, take an HTML page with embedded GIF images. MIME has
(newly defined) machinery for this: The HTML part, and its embedded GIF
images, are put together in a MIME multipart/related object. The
multipart/related object is an object to hold related objects that will not
only be displayed together, but refer to each other within the context of
the multipart MIME object. There is a sense of tree structure when needed;
there is reference information like Content-ID and Content-Location, etc.
But notice that each object within the multipart has a type (text/html,
image/gif, etc.) that is represented by a MIME Content-Type, could have a
Content-Disposition with further information about the object, and all of
this at a MIME-interpretable level.
It seems no mental stretch at all to create a multipart container type
which, instead of "related to eachother" semantics, or "digest of messages"
semantics, had a "collection of file system objects" semantics. Each object
could have it's own *MIME-interpretable* type information, it's own
*MIME-interpretable* file system attributes, etc.
>how is an AVI file defined?
>
>A Real Media clip? Are you going to stick all these into Multipart? (yuck)
Actually, both of those, since they are normally dealt with as single
units, go under the video top-level type. And like other media types, their
Content-Disposition can be specified too.
>To quote the same RFC:
>
>multipart -- data consisting of multiple entities of independent data types.
>The two types I have just mentioned are of a mixed data type but they in
>themselves are a single media type. They are not multiple entities.
>
>Could it be, that these two types are being misused? I think so.
Nope. This is why 2046 also specifies:
A media type of "video" indicates that the body contains a time-
varying-picture image, possibly with color and coordinated sound.
The term 'video' is used in its most generic sense, rather than with
reference to any particular technology or format, and is not meant to
preclude subtypes such as animated drawings encoded compactly. The
subtype "mpeg" refers to video coded according to the MPEG standard
[MPEG].
Note that although in general this document strongly discourages the
mixing of multiple media in a single body, it is recognized that many
so-called video formats include a representation for synchronized
audio, and this is explicitly permitted for subtypes of "video".
>Ask any of the MIME RFC authors about me, I complained about MIME when it
>was introduced, I'm not complaining NOW -but- I may wonder about the MIME
>spec if its not flexible enough to handle the introduction of new types and
>things somewhat easily.
It is exactly the flexibility of the multipart top-level type that I am
defending here. It is quite flexible enough to handle the subtype of "fs".
MIME is certainly flexible enough to take new top-level types, but
hopefully it should also be flexible enough to not require a new one when
something fits very smoothly into a current top-level type.
I still see no evidence that multipart/fs is a problem.
Al Costanzo
>
>A new type itself is not a problem.
From the discussion, it seems to be....
>But there is already a great deal of
>machinery within MIME to do this sort of thing. If we can reuse this
>machinery, it would be much better than reinventing the wheel. Again, I'd
>like to hear argument that the current machinery is insufficient for the
>task at hand.
Pete,
The following email is only my opinions,
Multipart sounds better than Application _BUT_ the fact that this machinery
is in place causes me to desire a new definition. For one thing, I do not
want MIME implementations to fall back to some other multipart definition if
they do not understand fs.
I think FS could be made to fit into Multipart _BUT_ I believe FS will be
evolving.
The control elements in FS will possibly become MIME defined control
elements, it is my hope that the community will embrace FS as a way to
easily attach directory structures to email messages.
Secondly, I do not want FS to become lost in Multipart.
To this end, in my opinion, it may be better to re-invent a wheel; to
embrace the vehicle riding upon it. (Not all cars ride on the same tires :-)
I hate the idea of mailers that may not know what to do with an fs object
trying to do some Multipart 'ing fall back to it.
I can see the I-D going through many iterations, it would be nice to have
all of these FS definitions in a type all their own. I think this will be of
great use to implementers (if this is where it leads)
Thirdly, The definition of a new type give more of an 'appearance' that this
is a completely new concept, which it is, no matter how it is defined but
people may not perceive it as such....
But thanks for your suggestion(s). I need to think a bit about this and hear
from more of the community before passing judgement ....
It is only a preliminary I-D on the topic...
-Al
Antony Bowesman
>
> Multipart sounds better than Application _BUT_ the fact that this machinery
> is in place causes me to desire a new definition. For one thing, I do not
> want MIME implementations to fall back to some other multipart definition if
> they do not understand fs.
Some fallback is inevitable when introducing a new type, top level or
otherwise, until it gathers support. multipart/fs will fallback to
multipart/mixed and file/fs will fallback to text/plain. What _do_ you
really expect or want to happen in this case?
> The control elements in FS will possibly become MIME defined control
> elements, it is my hope that the community will embrace FS as a way to
> easily attach directory structures to email messages.
It is pretty easy now to attach directory structure into a MIME message
e.g.
CT: multipart/mixed
CD: attachment; filename=/usr/bin;
creation-date="1 Sep 98 09:00:00 GMT";
modification-date="12 Sep 98 09:00:00 GMT";
read-date="15 Sep 98 05:39:32 GMT";
x-permission=0775; x-owner=0; x-group=3; boundary=x
--x
CT: multipart/mixed; boundary=y
--y
CT: application/octet-stream;
CD: attachment; filename=sh;
creation-date="1 Sep 98 09:00:00 GMT";
modification-date="12 Sep 98 09:00:00 GMT";
read-date="15 Sep 98 05:39:32 GMT";
x-permission=0775; x-owner=0; x-group=3
CTE: base64
DATA
--y
CT: application/octet-stream;
CD: attachment; filename=ksh; etc...
CTE: base64
DATA
--y--
--x--
So it just requires a few CD extensions to handle the above x-* values.
Introducing a specific multipart/fs would give MIME parsers an
instruction to indicate the data SHOULD be interpreted as a directory
structure rather than a simple set of files.
>
> Secondly, I do not want FS to become lost in Multipart.
Why? It seems to me you want to put FS into MIME intact rather than to
use the existing MIME infrastructure which if you did would actually
render FS unnecesary.
> To this end, in my opinion, it may be better to re-invent a wheel; to
From a commercial point of view, 'wheels' never get reinvented. The
development/validation effort forbids it.
> embrace the vehicle riding upon it. (Not all cars ride on the same tires :-)
But they are all round :-)
> I hate the idea of mailers that may not know what to do with an fs object
> trying to do some Multipart 'ing fall back to it.
Again, why? What do you want them to do? Return a non-delivery and
reject the message. Your fs object you have defined is currently a
compressed single object which contains an application directory
structure. This, if sent now as application/x-fs, would would be
interpreted as octet-stream by most parsers and be delivered, probably
sucessfully. If you included the filename parameter to a CD (or name to
CT) then many browsers/mailers would be able to launch a suitable
decoder.
> Thirdly, The definition of a new type give more of an 'appearance' that this
> is a completely new concept, which it is, no matter how it is defined but
> people may not perceive it as such....
I'm sorry, this is not new at all. The concept of attaching directory
structures has been around for some time and is already implemented in
at least one client I know of (TeamWARE Embla). It uses nexted
multipart/mixed to define the structure (as above). If sending between
these clients it allows you to extract the nested multipart structure to
the same format as sent. However, it does not preserve time, date or
permissions.
Other forms of attaching directory structures include tar/gzip, pkzip,
cpio, backup, pkgtrans etc. However, these are all treated as
application/*.
Summary:
(1) You have an existing FS object format which can be suitably handled
by application/fs
(2) There almost exists a suitable MIME method of transferring native
directory structures with a few extensions required.
The completely separate issue is the one you propose of MIME
compression. While there are examples that can be cited where some form
of compression would be a good idea I think that the impetus required to
get a new CTE type accepted, implemented and installed will be to great.
Rgds
--
Antony Bowesman
a...@to.sel.fujitsu.co.jp
Nathaniel Borenstein
it worth considering using multipart/related, with that containing
application/octet-stream for the "raw data" and something else
(application/fs-info) for the directory & other metadata about the file?
That way, we can get fallback to the current application/octet-stream
behavior pretty easily.
Just a thought. -- Nathaniel
--------
Nathaniel Borenstein <n...@nsb.si.umich.edu>
http://guppylake.com/~nsb/nsb-faq.html
Dan Grillo
antony Bowesman writes in comp.mail.mime:
> [...]
> So it just requires a few CD extensions to handle the above x-* values.
> Introducing a specific multipart/fs would give MIME parsers an
> instruction to indicate the data SHOULD be interpreted as a directory
> structure rather than a simple set of files.
> > Thirdly, The definition of a new type give more of an 'appearance' that this
> > is a completely new concept, which it is, no matter how it is defined but
> > people may not perceive it as such....
>
> I'm sorry, this is not new at all. The concept of attaching directory
> structures has been around for some time and is already implemented in
> at least one client I know of (TeamWARE Embla). It uses nexted
> multipart/mixed to define the structure (as above). If sending between
> these clients it allows you to extract the nested multipart structure to
> the same format as sent. However, it does not preserve time, date or
> permissions.
Other MIME programs do this also.
NeXT's (now Apple's) mail program has for years (since NEXTSTEP
3.3) uses multipart/x-folder to send directories, with a x-unix-mode
C-D extention to preserve file mode, and with a top-level C-D of
attachment so the receiving MUA an icon of a folder instead
of exploding the directory view.
My mail MIME software (dmail) has also used the method for years.
For other filesystem types "x-unix-mode" is not good enough,
for the basic concept (use a multipart type with C-D extentions) works.
--Dan
--
Dan Grillo d...@grillo.net 650 917-0685 fax 773 975-1557
Bart Schaefer
} Subject: Re: application/fs vs. fs/*
}
} Thinking about "fallback behavior" led me to the following question: Is
} it worth considering using multipart/related, with that containing
} application/octet-stream for the "raw data" and something else
} (application/fs-info) for the directory & other metadata about the file?
I actually wrote up a draft for something like this some months ago and
had Chris Newman take a look at it. My suggestion was for a generic
application/annotation type, with a content-type parameter to describe
the specific type of annotation (modeled after message/external-body
type parameters).
Chris's response was that it was a good idea, but that XML was already
proposing an annotation mechanism and it would be bad form for the IETF
to appear to be competing with it.
As a result of this discussion going by, I'm looking again at that draft.
For a single file object (not a hierarchy), the approach would be to use
an application/xml entity as the "start=" part of the multipart/related,
where the XML contains RDF notation with a Content-ID URL pointing to the
"raw data" part.
The complexity arises when you want to represent a filesystem hierarchy
(or any other hierarchical structure). It's wasteful, not to mention
difficult for agents that are not multipart/related -aware to handle, if
every object in the hierarchy has to have its own multipart/related. A
better scheme, it seems to me, would be to have one multipart/related at
the top level, put the entire raw hierarchy in one part of that, and put
all the annotations in another part.
The problem then becomes one of describing a sensible mapping from the
annotation part to the raw hierarchy. It can of course be done entirely
with Content-ID URLs; but it'd be nice to have some other mechanism, for
example a one-to-one correspondence between parallel hierarchies (though
that specific solution feels a bit clumsy).
Another question is, should the layout of the hierarchy be represented in
the annotation part or by constructing nested multiparts in the data part?
The former is more powerful -- annotations could describe concepts such as
hard and soft links, which a multipart hierarchy can't easily represent --
but the latter stands a better chance of recovering the layout even when
the recipient can't interpret the annotations.
--
Bart Schaefer Zanshin
http://www.well.com/user/barts http://www.zanshin.com
Dan Kohn
>Thinking about "fallback behavior" led me to the following question:
Is
>it worth considering using multipart/related, with that containing
>application/octet-stream for the "raw data" and something else
>(application/fs-info) for the directory & other metadata
>about the file?
From an optimizing "fallback behavior" standpoint in transferring
filesystems, it seems difficult to beat a multipart/filesystem approach
which uses the appropriate MIME type for individual files and
Content-Disposition parameters for the file parameters. The
multipart/filesystem would also need a Content-Disposition parameter to
allow the naming of directories.
Mailers which don't support multipart/filesystem will fall back to
interpreting a (possibly nested) multipart/mixed, and display
representations of all files. There will be basic interoperability for
all mailers that satisfy RFC 2049 (i.e., treat unrecognized multipart
subtypes as mixed and unrecognized content types as
application/octet-stream). And, recognized content subtypes can be
dealt with appropriately, such as on the Mac by adding the MSWD file
type for application/msword.
Mailers which support multipart/filesystem (which, of course, is
currently 0.0% of the ~40M mailers out there), would, for instance on
Windows, display the entire multipart as a folder or series of nested
folders.
In either case, there does seem some value to specifying additional
Content-Disposition parameters to allow file info to be transferred,
which the obvious security caveats about adding cron jobs with root
permissions, etc.
The multipart/related approach would, on mailers that don't understand
it, produce a random seeming file (the application/fs-info interpreted
as application/octet-stream) for every real file sent. Even for
advanced users, it would be annoying to edit all of these out. RFC 2049
says (as of course you know since you wrote it) that unknown content
type parameters must be ignored, which seems a far superior behavior
from a "fallback perspective."
Bart Schaefer [mailto:scha...@brasslantern.com] wrote:
>Another question is, should the layout of the hierarchy be represented
in
>the annotation part or by constructing nested multiparts in the data
part?
>The former is more powerful -- annotations could describe concepts such
as
>hard and soft links, which a multipart hierarchy can't easily represent
--
>but the latter stands a better chance of recovering the layout even
when
>the recipient can't interpret the annotations.
Including all directory structure in one XML application/fs-info file
would at least produce less annoying results for existing mailers (only
one random file rather than one for each real file), but I'm not sure
that it provides that much more flexibility. Couldn't you accomplish
the same thing with an application/hardlink which uses Content-IDs to
show a relationship (well, probably an "application/filesystem-link;
type=hard; cid=<123.abc>")?
Al Costanzo [mailto:a...@akc.com] wrote:
>"FILE" was to have as subtypes file systems, for example,
>fat32, fat, ntfs, hpfs, etc.
>and one special one called independent (or something which
>implies that the file system is unimportant.
>Inside of the subtypes would be MIME one-to-one mappings, file hook
>extension to MIME types of FILE. So "file.doc" may be defined
differently
>depending on the File system. In the windows world it would be a Word
Document.
>These defs would come from the platform. If there was a conflict i.e.
.doc
>may be defined two different ways, a fallback procedure would be
defined how
>to determine which definition would be correct.
This already happens today. It is the sending application's
responsibility to set the MIME content type and subtype based on its
best understanding of the body. The sending application has the best
chance of knowing whether .doc is application/msword, or some sort of
oddly labeled application/vnd.ms-excel.
By contrast, only the receiving application knows what file system, what
byte order, what extensions make sense, etc. So, is needed is a
file-system independent way of labeling media types. That is MIME.
>This FILE top level type is supplemental in nature, it was not intended
to
>circumvent the other MIME types _BUT_ what it does do it provide a way
for
>the sender let the receiver know what exactly the object being
>sent is and from what OS is from. Thereby letting the receiver have an
>understand of what to do with the object.
>I thought this information would be useful.
>My thoughts were that if this was done, you would not have to
>inspect the file to figure out what kind of object it actually was.
Not only would it be useful to know what media type a file is without
inspecting it, but hundreds of engineers have spent over 5 years
creating a system to allow you to do just that. It's called MIME.
You seem to be making an honest effort to add useful functionality, but
I really can't recommend highly enough that you go back and carefully
read RFC 2045 through 2049 all the way through. The 5 documents are
some of the most clearly written and compellingly engineered Internet
standards (given the extreme constraints of operating with RFC822).
MIME is, as RFC 2045 says, "somewhat strange or even baroque at first
reading." But it is also a carefully engineered system which works
amazing well much of the time. There are certainly numerous areas where
it might be extended (and certainly many mailers in which it could be
better implemented), but that is no reason to throw out all of the great
work that has been done.
>I think defining file system objects as MIME types within OS
>to be a worthy endeavor.
If, for some reason, you wanted to know what file system a MIME entity
originated on, than I believe a Content-Disposition parameter would be
the most appropriate place to transfer the information. But note that
Word 97 for Windows uses a file format that is binary compatible with
Word 98 for the Mac. And, of course, the format is the same whether the
Windows disk was formatted with FAT, FAT32, or NTFS. So, it's hard to
imagine why a receiver would care about this information. What the
receiving application wants to know is application/msword (well, and
maybe ;version=97).
>An FS object would be defined as FILE/independent
Conceptually, all MIME entities fit under this heading. image/jpeg
represents the same picture no matter what filesystem it is sent from or
to.
I agree with Antony Bowesman that if you have an existing FS object
format, then use application/fs. However, you'd need to make an
argument on why that format is any better than application/gzip or
application/zip, which at least already exist on many platforms.
If you want to send multiple files and keep their directory structure,
then use multipart/mixed or look at creating a multipart/filesystem.
If you want to add compression to arbitrary files, then look at a new
Content-Transfer-Encoding. Of course, this will never make sense until
there is a feature negotiation for email. Otherwise, all ~40 million
existing mailers out there will be stuck with unreadable content because
they won't understand the CTE.
Personally, I send Winzipped archives of files to some select recipients
all the time, and would love for this to be automatic in my mailer.
But, it would clearly need to be on a per recipient basis, as I don't
want to send an archive to people in a format they can't read. Before
you go any further with your fs idea, you might consider why
implementers wouldn't just add native ZIP or gzip support to their
application, simplifying the archiving process. And note, that the
mailer would then send the archive as an application/zip or
application/gzip content type, with no need whatsoever for new top-level
fs or file types.
- dan
--
Daniel Kohn <d...@teledesic.com>
Teledesic LLC
+1-425-602-6222 (voice) 602-0002 (fax)
http://www.teledesic.com