Any suggestions of which IMAP server is better?

14 views
Skip to first unread message

Minh Trieu

unread,
May 6, 2002, 6:12:28 AM5/6/02
to
Im currently using uw-imapd, but its slow when your mailbox gets really big,
and has other problems with outlook such as timing out issues.

Which IMAP server should i upgrade to?
Cyrus or Courier?

Can someone with user experience with the above 2 imapd software provide me
with direction as to which one i should use, as well as why? Also as with
uw-imapd i have to create drafts and sent folders under the inbox folders in
order for squirrel mail and outlook to use. Is there a imapd server that had
inbox/draft/sent mailboxes on an equivalent hierarchy?

Thanks
Minh


Frank Klauert

unread,
May 6, 2002, 11:55:59 AM5/6/02
to
> Im currently using uw-imapd, but its slow when your mailbox gets really big,

Are you using MBX-format oder Unix-mailspool?
MBX is much better!

> and has other problems with outlook such as timing out issues.

Maybe INETD-issues?

>
> Which IMAP server should i upgrade to?
> Cyrus or Courier?

Courier sucks!
It is not IMAP-compatible (by design), the author is an idiot and the format
it uses is insane.

Cyrus has the problem that it won't accept 8bit-Mails by default.
I was able to choose between "reject" and "destroy mail by replacing
everything >127 with X". There are rumours that this _incorrect_
behaviour can be patched away.
Using cyradm is a great pain! (at least when I had to use it)

Want a really good IMAP-Server? Really fast, good coding style (C++),
made for >1GB-mailboxes, very easy to install?

http://www.dbox.handshake.de/dkimap4/

It's just a great product, IMO!

> Can someone with user experience with the above 2 imapd software provide me
> with direction as to which one i should use, as well as why?

For "Why not use courier":

1. http://www.inter7.com/courierimap/BUGS.txt

As one clearly see: Courier is broken by design!

2. Look at things like this:

----------

----- Original Message -----
From: "Mark Crispin" <M...@CAC.Washington.EDU>
To: "Dag Nygren" <d...@newtech.fi>
Cc: "Michael stergaard Pedersen" <mic...@daimi.au.dk>; <c-cl...@u.washington.edu>
Sent: Thursday, April 18, 2002 12:45 AM
Subject: Re: IMAP messages "disappear"


> On Thu, 18 Apr 2002, Dag Nygren wrote:
> > > My setup is:
> > > Courier-IMAP 1.4.1
> > > C-Client: CVS-Snapshot from 10-01-2002 (have also tried others)
> > > PHP 4.1.1
> > > IMP 3.0
> > > My problem is that when I log on to IMP using an IMAP server, I can see the
> > > new messages in my Inbox. As soon as I click one of them I'm told that the
> > > message does not exist and all new messages "disappear".
> > I see the same problem with IMP 3.0 and IMAP and just traced it down
> > to IMP using the "SORT" imap command and this seems to behave
> > strangely.
>
> I strongly suggest that you try some IMAP server other than Courier.
>
> If the problem goes away (and I suspect it will) with some non-Courier
> IMAP server, then you know that the problem is in Courier.
>
> Courier does not implement IMAP. It implements what the author of Courier
> thinks IMAP should be, which is somewhat different from the IMAP
> specification. I gave up on him a long time ago, and I'm not particularly
> interested in wasting any more time going through the cycle of:
> 1) problem gets reported to me
> 2) I respond that it's due to a wrong thing that Courier does
> 3) the author of Courier insists that I am wrong
> 4) I quote the section of the IMAP specification that Courier violates
> 5) the author of Courier calls me an idiot, and says that the IMAP
> specification is wrong
>
> -- Mark --
>
> http://staff.washington.edu/mrc
> Science does not emerge from voting, party politics, or public debate.
>
>

---------

Enough said.

> Also as with
> uw-imapd i have to create drafts and sent folders under the inbox folders in
> order for squirrel mail and outlook to use. Is there a imapd server that had
> inbox/draft/sent mailboxes on an equivalent hierarchy?

Cyrus has a problem with this.
All your mailboxes must be subfolders of your INBOX.

Cyrus will force you to set the path to your mail to "INBOX". (It's somewhere
in the account-options of Outlook Express and Outlook).

But that is just a workaround. Better use DKIMAP. It has no problems like
this!


Nancy McGough

unread,
May 6, 2002, 12:17:21 PM5/6/02
to
On 6 May 2002 Frank Klauert (frank....@hotmail.de) wrote:
> Want a really good IMAP-Server? Really fast, good coding style (C++),
> made for >1GB-mailboxes, very easy to install?
>
> http://www.dbox.handshake.de/dkimap4/

I looked at the above URL and on it, it says:

dkimap4 is tested with the following clients:
* Netscape 4.x
* Outlook Express 5.x
* Eudora 5.0 (not fully functional yet)

It seems odd that they haven't tested it with Mulberry and Pine,
both of which are known as robust standards-compliant powerful
IMAP clients. The 3 clients listed above are known as *not* being
good IMAP clients...

Do you know if any of the IMAP Service Providers that I list here

<http://www.ii.com/internet/messaging/imap/isps/>

are using dkimap4? How does dkimap4 respond when one telnets to
port 143? Do you know of any (other) IMAP service provider that
uses it so I can test it and add it to my list?

Thanks,
Nancy
also interested in a comparison and ranking of IMAP servers


REFERENCE:
The message I'm replying to -- and this entire thread & group --
may be available at

<http://groups.google.com/groups?selm=ab68vh$fd5hn$1...@ID-140432.news.dfncis.de>

--
Nancy McGough <http://www.ii.com/> Infinite Ink
--= Sent via Pine 4.44: IMAP, NNTP & ESMTP for Unix/Win/MacOS X =--

Mark Crispin

unread,
May 6, 2002, 12:31:23 PM5/6/02
to
On Mon, 6 May 2002, Minh Trieu wrote:
> Im currently using uw-imapd, but its slow when your mailbox gets really big,
> and has other problems with outlook such as timing out issues.

If you are using the traditional UNIX mailbox format, have you considered
using another format such as mbx? You'll have to switch formats anyway if
you are moving to one of the other formats.

The problems with Outlook are due to bugs in Outlook.

If you do switch to another server, use Cyrus.

Frank Klauert

unread,
May 6, 2002, 1:05:50 PM5/6/02
to
"Mark Crispin" <m...@CAC.Washington.EDU> wrote:
> If you are using the traditional UNIX mailbox format, have you considered
> using another format such as mbx? You'll have to switch formats anyway if
> you are moving to one of the other formats.

I have a question to you:

You kind of promised (as far as one could say that when it's about free
software of cource...) in one posting a new version of the mbx-format
which would cache more meta-information. I think that this is a great
idea which would make UW-IMAP superior not only in actually being an
IMAP-server (compared to courier ;-) ) but in performance!

Is there any beta-version available?

I would be glad if I could test it, although I don't believe in that
being possible. (I could even provide a real email-adress ;-) )

And I think you would do a good thing to UW-IMAP if you would write everywhere
in the documentation and the website in really huge letters some warning
that unix-mailspool-format is slow.
Almost *everybody* has this problem with your (really good!) software! And
I just think that this is not fair, because everybody measures your software
compared to Unix-mailspool.


Sam

unread,
May 6, 2002, 1:31:57 PM5/6/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <ab68vh$fd5hn$1...@id-140432.news.dfncis.de>,
"Frank Klauert" <frank....@hotmail.de> writes:

>>
>> Which IMAP server should i upgrade to?
>> Cyrus or Courier?
>
> Courier sucks!

To each his own.

> It is not IMAP-compatible (by design),

Really? You better tell that to hundreds of thousands of users. They're
going to be shocked to find out that their IMAP mail clients are not really
supposed to work.

> the author is an idiot

We aim to please.

> and the format it uses is insane.

"Insanity -- a perfectly rational adjustment to an insane world."
- R.D. Lang

> For "Why not use courier":
>
> 1. http://www.inter7.com/courierimap/BUGS.txt
>
> As one clearly see: Courier is broken by design!

Here's a question: do you actually open a URL, in order to get some vague
grasp of a clue on its contents, or do you pronounce judgment based solely
on the URL's path?

> 2. Look at things like this:
>
> ----------
>
> ----- Original Message -----
> From: "Mark Crispin" <M...@CAC.Washington.EDU>

Oh, more of Crispin's FUD... How un-original. Can't you come up with
something on your own? Preferrably something that reads above a
grade-school intelligence level.

>> 1) problem gets reported to me
>> 2) I respond that it's due to a wrong thing that Courier does
>> 3) the author of Courier insists that I am wrong
>> 4) I quote the section of the IMAP specification that Courier violates
>> 5) the author of Courier calls me an idiot, and says that the IMAP
>> specification is wrong

Well, last time I checked, Crispin's own revision to RFC 2060 had at least
a couple dozen errors, omissions, clarifications, or corrections to that
document. You cannot design anything based on that document, word for
word, and expect it to interoperate with other IMAP software. It just
doesn't work. And it's not my fault that it doesn't work.

Well, it's now been eight years since RFC 2060.
http://www.imap.org/products/showall.php now lists only 78 IMAP
applications. That's across ALL hardware/software platforms; and of all
possible categories: client, server, whatever. Why so little? Well,
before you choose to run your mouth and blather something about a topic
that you don't apparently know much about, think about it first. You'd
look less of a fool, then.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE81r4L3ejdWUS0ltARAnaAAJ9mGKE1tdaArBlFQ1XoJUmN+fGfqgCgkpJJ
odj5U25HXW/zWIMyRPmjMZ0=
=azBw
-----END PGP SIGNATURE-----

Mark Crispin

unread,
May 6, 2002, 1:32:31 PM5/6/02
to
On Mon, 6 May 2002, Frank Klauert wrote:
> You kind of promised (as far as one could say that when it's about free
> software of cource...) in one posting a new version of the mbx-format
> which would cache more meta-information.

There are two projects in progress. One is by a fellow in the Czech
Republic named Martin Devera, the other is by our local computer center.
I don't think that either one is in a shape for others to try it yet.

I may do a third one of my own, but strictly limited to compatible
changes (upwards and downwards) to the existing mbx format.

> And I think you would do a good thing to UW-IMAP if you would write everywhere
> in the documentation and the website in really huge letters some warning
> that unix-mailspool-format is slow.
> Almost *everybody* has this problem with your (really good!) software! And
> I just think that this is not fair, because everybody measures your software
> compared to Unix-mailspool.

Well, the world is unfair. However, I can not simultaneously announce
that my server is the only one which is fully compatible with the legacy
UNIX mail world, and whine about how slow the legacy is. The two
positions are incompatible.

The drivers.txt documentation file does have a features/performance
comparison of the various formats that are bundled with UW imapd.

However, part of the problem with doing comparisons is that there are so
many variables which are not always obvious. A one-message/one-file
format such as mh or maildir can be "very poor" or "good" depending a
great deal upon how fast stat() and open() are and if per-message metadata
can be acquired other than with stat(). On Linux, these calls are fast;
on other UNIX variants, they are quite slow. Not surprisingly, much of
the maildir fan community uses Linux.

Mark Crispin

unread,
May 6, 2002, 3:33:04 PM5/6/02
to
Most Internet software developers cooperate with other software
developers.

When their software is shown not to comply with the published
specification, they fix their software.

If some people can't log into their server because they did not precisely
implement the syntax in the published specification, they change their
server to comply with the published specification.

They do not say that they won't change their server because the published
specification is wrong, and that complying with the published
specification would make their code into "spaghetti."

They assume, whenever an interoperability problem with older established
software occurs, that the flaw is in their software unless proven
otherwise in the specification.

They accept correction when shown that they misinterpreted a portion of
the specification, particularly when some other portion of the
specification unambiguously clarifies the matter.

If a portion of the specification seems ambiguous, they ask for a
clarification, either to the author of the specification or to the working
which underlies the author's work.

When assessing whether or not their product is compliant with a published
specification, they eschew converse accidents such as "my product
interoperates with such-and-such other product."

They do not publish a "bug list" of other software when, in fact, that
other software adheres to the published specification or when there is
nothing in any published specification to support the claim.

Example 1: "Pine chokes on whitespace between BODY and [" is such a case;
the syntax in the IMAP specification does not permit whitespace between
BODY and [, and thus it is quite reasonable for Pine to "choke".

Example 2: "Pine's implementation of the Drafts folder is buggy." The
claim is that it is a bug for Pine to expect to be able to create and
delete its postponed-messages mailbox at will, and uses the existance of a
postponed-messages mailbox to determine if there are any postponed
messages. Some other programs expect that there be a static Drafts
mailbox, and key on whether or not that mailbox is empty. However, no
published specification has ruled on the matter, or even has gone so far
as to identify a type of mailbox called "Drafts" which has certain
interoperable semantics. IMAP defines the semantics of only one mailbox
name: INBOX. All other names are private use.

Example 3: "Pine fails to quote certain special characters in the login
userid and password." The characters in question are not atom-specials as
defined in the IMAP specification, and therefore do not need to be quoted.
Any server which fails to accept those characters unquoted is
non-compliant with the IMAP specification.

Example 4: A complete non sequitor: "RFC 2060 clearly indicates that
a non-empty reference is non-standard behavior." I don't know where to
begin on this.

Example 5: "Netscape Communicator insists that the response in
HEADER.FIELDS is terminated by a blank line, supposedly the end of message
headers." Goodness knows that I am no fan of Netscape's client, but in
this case it is correct in expecting this [RFC 2060, page 42].

Example 6: "The IMAP4REV1 RFC is not clear as to what LIST should return
for a mailbox that can contain both messages and other mailboxes. The RFC
strongly implies that the response should be one LIST reply without
\Noinferiors and \Noselect tags." This is supposedly a bug in RFC 2060,
yet the supporting evidence is nothing more than complaints about the
respective user interfaces in Pine and Netscape. There's some substance
to the complaints, but they are mostly bogus.

To be fair, there is some substance in the other assertions in that "bug
list," but it is buried under all the nonsense.

Frank Klauert

unread,
May 6, 2002, 4:38:25 PM5/6/02
to
> > It is not IMAP-compatible (by design),
>
> Really? You better tell that to hundreds of thousands of users. They're
> going to be shocked to find out that their IMAP mail clients are not really
> supposed to work.

According to the bug-liste it's not supposed to work with some of them...

In real live, many IMAP-servers are actually used with Netscape, Outlook Express,
Pine, ...

If you create your software the way it is not or hardly usable with clients
that are used very often, then your software is pretty useless...

BTW: Which client do you think does it right? If not Pine? Mutt????


> > the author is an idiot
>
> We aim to please.

Sorry, I didn't really think you would read this newsgroup

>
> > For "Why not use courier":
> >
> > 1. http://www.inter7.com/courierimap/BUGS.txt
> >
> > As one clearly see: Courier is broken by design!
>
> Here's a question: do you actually open a URL, in order to get some vague
> grasp of a clue on its contents, or do you pronounce judgment based solely
> on the URL's path?

If someone tells me that Pine (which simply is a reference implementation of
IMAP) would be broken when it speaks IMAP, then this whole document simply
MUST be a joke!

And just how can you say something like that the RFC is broken? By definition,
the software is broken, not the RFC. If you ignore the RFC, then please don't
call it IMAP.


Maybe you should remove your client-bug-list or at least rewrite it to only
complain about real problems in those clients.


> > From: "Mark Crispin" <M...@CAC.Washington.EDU>
>
> Oh, more of Crispin's FUD... How un-original.

Your clientbuglist is exactly what I would call FUD.
Telling everybody that the faults are all those clients
around helps you justify not fixing problems, but not the users!

Don't use sentences like "Ask Microsoft to actually read the RFCs
before implementing them." if you ignore RFCs yourself!

> Can't you come up with
> something on your own? Preferrably something that reads above a
> grade-school intelligence level.

Of course I can. (But I don't have to)

Why this stupid maildirformat? (Which Courier is about after all)

It simply can't work! If there is no UID-file (which is there), then
IMAP can't work because you need to assign new messages individual
ascending UIDs. No client which caches metadata or messages would
be able to find the message in a maildir. This would not be IMAP,
it would not allow disconnected use.

But if you have this UID-mapping-file, then it's no maildir anymore:
You'll need locking, you'll have to rebuild that file if it's broken...

And I really don't like the idea to have one inode plus the rest of
the cluster wasted for every mail, no matter how small it is.

The best format I've seen so far (and that does NOT include Exchange
which I was not able to install because it forces one to have w2k
server and a working Active Directoy) is the one from DKIMAP, IMO.

>
> >> 1) problem gets reported to me
> >> 2) I respond that it's due to a wrong thing that Courier does
> >> 3) the author of Courier insists that I am wrong
> >> 4) I quote the section of the IMAP specification that Courier violates
> >> 5) the author of Courier calls me an idiot, and says that the IMAP
> >> specification is wrong
>
> Well, last time I checked, Crispin's own revision to RFC 2060 had at least
> a couple dozen errors, omissions, clarifications, or corrections to that
> document. You cannot design anything based on that document, word for
> word, and expect it to interoperate with other IMAP software. It just
> doesn't work. And it's not my fault that it doesn't work.
>
> Well, it's now been eight years since RFC 2060.
> http://www.imap.org/products/showall.php now lists only 78 IMAP
> applications. That's across ALL hardware/software platforms; and of all
> possible categories: client, server, whatever. Why so little? Well,
> before you choose to run your mouth and blather something about a topic
> that you don't apparently know much about, think about it first. You'd
> look less of a fool, then.

Why don't you create your own remote-mail-protocol? Go ahead, do it,
publish it as RFC, implement a reference-server for it, a reference library
to use it, persuade others to implement it...

I think that IMAP is a great thing (especially disconnected use), a protocol
urgently needed by everyone who seriously uses Email, and it's great that
Mark developed it, published it, and that it is supported by the most
widespread clients (in browsers) and many others.
He has written a sever which is very good and really easy to install and
a library which is very flexible and useable (I really enjoyed copying
mails from pop3 to imap with mbxcopy, it's a very good concept to have
a uniform view to mails). So please don't mess around with that and try
to pull it down.

Maybe you should remove IMAP-support from Courier, if you hate it so
much. I'm sure it would be the best for all of us. Maybe a NFS-mounted
maildir is the right solution for your prefered audience. I don't
see much difference for them.

And I don't think it's possible to develop such a complex (complex
because it is so very powerful compared to SMTP or POP3. Not because
it would be possible to make it much simpler) protocol as RFC 2600
(and to foresee every aspect of it's use in the future) without the
need for some clarification afterwards. But that is really no reason why it
is not a great achievement that it exists and that it actually DOES
work (e.g. Mulberry with UW-IMAP&Cyrus, Outlook Express with Cyrus&
UWIMAP, Netscape with Cyrus&UW-IMAP, ...) and not very bad!


Frank Klauert

unread,
May 6, 2002, 4:50:30 PM5/6/02
to
Hi!

> It seems odd that they haven't tested it with Mulberry and Pine,
> both of which are known as robust standards-compliant powerful
> IMAP clients.

I agree with that.

> The 3 clients listed above are known as *not* being
> good IMAP clients...

For me it worked with Mulberry.

What really seems odd to me is that I don't know of anybody
else using DKIMAP. There's not very much about it in google.
I would really like to know why, because it not only makes
a very good impression to me (especially for my individual needs),
but also works really good (at least for me!)

>
> Do you know if any of the IMAP Service Providers that I list here
>
> <http://www.ii.com/internet/messaging/imap/isps/>
>
> are using dkimap4?

No, I don't. The only IMAP-servers I've used so far are
in my company and at my home. (And I have really to much
mail to consider keeping it at an external IMAP Service Provider)

> How does dkimap4 respond when one telnets to
> port 143?

It says "* OK ROUTER DKIMAP4 IMAP Server"

> Do you know of any (other) IMAP service provider that
> uses it so I can test it and add it to my list?

Unfortunately I don't know one.
But it was very easy to me to get DKIMAP running under
Linux (Suse and Debian) and it was even possible to
use it with Cygwin.

If anybody tells me what is wrong with DKIMAP I would
be glad to hear about it. So far it works quite great
here!


Sam

unread,
May 6, 2002, 5:17:17 PM5/6/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <Pine.LNX.4.50.020506...@shiva0.cac.washington.edu>,
Mark Crispin <m...@CAC.Washington.EDU> writes:

> Most Internet software developers cooperate with other software
> developers.
>
> When their software is shown not to comply with the published
> specification, they fix their software.

Your specifications are ambiguous, unclear, mis-designed, and in many cases
are clearly wrong and don't even match up with UW-IMAP's, or Pine's,
output, in some cases.

> Example 1: "Pine chokes on whitespace between BODY and [" is such a case;
> the syntax in the IMAP specification does not permit whitespace between
> BODY and [, and thus it is quite reasonable for Pine to "choke".

IMAP does not have any kind of a formal grammar, or a lexical syntax. The
protocol is defined as an unstructured glop of regular expressions.

> Example 2: "Pine's implementation of the Drafts folder is buggy." The
> claim is that it is a bug for Pine to expect to be able to create and
> delete its postponed-messages mailbox at will, and uses the existance of a
> postponed-messages mailbox to determine if there are any postponed
> messages.

Right. If you want to know if there's anything in the folder, wow, what a
brilliant idea: just open the folder and check. Don't assume that if the
folder exists, it must be non-empty.

> Example 3: "Pine fails to quote certain special characters in the login
> userid and password." The characters in question are not atom-specials as
> defined in the IMAP specification, and therefore do not need to be quoted.

This goes back to point 1: the protocol's syntax, and grammar, is a mess.
Good design, in practice, indicates that any protocol above some level of
complexity should have a well-defined lexical syntax and grammar. I defy
anyone to present a complete definition of IMAP's syntax or grammar.
There is none. It's just a hairball of regular expressions.

I defy anyone to cite any text-based wire protocol that demands explicit
presense, or absence, of whitespace delimiters.

> Any server which fails to accept those characters unquoted is
> non-compliant with the IMAP specification.
>
> Example 4: A complete non sequitor: "RFC 2060 clearly indicates that
> a non-empty reference is non-standard behavior." I don't know where to
> begin on this.

It's really very simple: you define, yourself, that a certain form of the
LIST command is implementation-defined: namely a non-empty reference
parameter for LIST. Then you go ahead and have Pine spit out non-empty
LIST references.

You don't know where to begin? How about explaining why, originally,
Pine's LIST requests were designed for UW-IMAP-specific namespace and LIST
semantics, instead of any IMAP server's implementation. Not that there's
anything particularly wrong with that, but let's get off this
holier-than-though high horse, 'kay?

> Example 5: "Netscape Communicator insists that the response in
> HEADER.FIELDS is terminated by a blank line, supposedly the end of message
> headers." Goodness knows that I am no fan of Netscape's client, but in
> this case it is correct in expecting this [RFC 2060, page 42].

I have an idea: let's just add random empty lines to random commands, for
no good reason. And because I said so, it must make sense, then.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE81vLc3ejdWUS0ltARAq7tAKCEuPojG5ZvgbnJGhDXOCjhcwQz3gCfY0Ed
aXRFAXVrOtRBLkDAgnkMsIU=
=SaZK
-----END PGP SIGNATURE-----

Mark Crispin

unread,
May 6, 2002, 5:50:40 PM5/6/02
to
On Mon, 6 May 2002, Frank Klauert wrote:
> What really seems odd to me is that I don't know of anybody
> else using DKIMAP. There's not very much about it in google.
> I would really like to know why, because it not only makes
> a very good impression to me (especially for my individual needs),
> but also works really good (at least for me!)

I am also curious about it. I have not heard much about it at all.

I have had *zero* reports of interoperability problems between c-client
based software (e.g. Pine) and DKIMAP. I don't know if that means that
DKIMAP is perfectly compliant, or if it hasn't been exercised enough.

There are, however, a number of client traps which await even a perfectly
compliant server.

For example, if DKIMAP offers SSL/TLS support, it must be careful not to
send full-sized (16K) SSL payloads due to a bug in Microsoft's SSL
implementation that effectively limits the maximum to a smaller value.
Many IMAP implementations limit their SSL payloads to 8K to be safe. This
problem affects both IMAP clients and IMAP servers when the other end is
Windows-based.

Another pitfall awaiting even a perfectly compliant server has to do with
clients which expect that an optional facility of IMAP is mandatory, and
which abuse optional facilities to the point that a server is obliged to
take defensive action.

There is a torture test message on:
ftp://ftp.cac.washington.edu/mail/mime/torture-test.mbox that is
helpful in testing IMAP servers. It is truly horrible; it has multiparts
inside nested messages inside multiparts; it violates the MIME
specification all over the place (many of those violations comply with old
preliminary drafts of MIME); etc.

If a server renders the torture test into the correct IMAP BODYSTRUCTURE
(UW, Cyrus, and Exchange all do), then its authors have done their
homework. No server (not even UW) got it right the first time; in fact I
created the torture test specifically to be something to break the UW code
so I could then go and fix it. There may be other servers besides UW,
Cyrus, and Exchange that now pass it; I just know about those three
because I witnessed their torture (and salvation) first-hand.

Since some of the torture test does involve invalid MIME (although
possible in pre-MIME use of Content-Type and in preliminary MIME), some
variation is permitted. A server does not have to adhere strictly to the
concensus result from UW/Cyrus/Exchange in order to have passed. But at
the very least it should recognize all the levels of testing and all the
individual MIME parts.

Frank Klauert

unread,
May 6, 2002, 6:11:38 PM5/6/02
to
>
> > Example 1: "Pine chokes on whitespace between BODY and [" is such a case;
> > the syntax in the IMAP specification does not permit whitespace between
> > BODY and [, and thus it is quite reasonable for Pine to "choke".
>
> IMAP does not have any kind of a formal grammar, or a lexical syntax. The
> protocol is defined as an unstructured glop of regular expressions.

And here we have again to quote you:


"Ask [Microsoft] to actually read the RFCs
before implementing them."

Well, let's look at
http://www.isi.edu/in-notes/rfc2060.txt

What can we find there?

-----------

9. Formal Syntax

The following syntax specification uses the augmented Backus-Naur
Form (BNF) notation as specified in [RFC-822] with one exception; the
delimiter used with the "#" construct is a single space (SPACE) and
not one or more commas.

[...]

flag_list ::= "(" #flag ")"

greeting ::= "*" SPACE (resp_cond_auth / resp_cond_bye) CRLF

header_fld_name ::= astring

header_list ::= "(" 1#header_fld_name ")"

LF ::= <ASCII LF, line feed, 0x0A>

list ::= "LIST" SPACE mailbox SPACE list_mailbox

list_mailbox ::= 1*(ATOM_CHAR / list_wildcards) / string

list_wildcards ::= "%" / "*"

literal ::= "{" number "}" CRLF *CHAR8
;; Number represents the number of CHAR8 octets

[...]

----------

What do you think is this?
[x] BNF
[x] something formal
[x] something you can work with
[?] something you don't understand


But: Even if there are problems understanding the
IMAP-protocol, or the given examples in RFC2600:

One can always use the reference-implementations
for IMAP to check one's software wether it works
or is broken.
And this would be UW-IMAP and Pine.
(And maybe/surely Cyrus and Mulberry).

Can anybody hear Cyrussoft complain how bad RFC 2600
is???

> > Example 3: "Pine fails to quote certain special characters in the login
> > userid and password." The characters in question are not atom-specials as
> > defined in the IMAP specification, and therefore do not need to be quoted.
>
> This goes back to point 1: the protocol's syntax, and grammar, is a mess.
> Good design, in practice, indicates that any protocol above some level of
> complexity should have a well-defined lexical syntax and grammar. I defy
> anyone to present a complete definition of IMAP's syntax or grammar.

Great!
I think I'm the one.
( http://www.isi.edu/in-notes/rfc2060.txt )

> There is none. It's just a hairball of regular expressions.

It was really easy to find BNF. But I just haven't seen this
hairball of regular expressions. Where is it?

>
> I defy anyone to cite any text-based wire protocol that demands explicit
> presense, or absence, of whitespace delimiters.

Well, let's try a exotic protocol which is hardly used today:

HTTP?

Go to
http://www-old.ics.uci.edu/pub/ietf/http/rfc1945.html

...
SP = <US-ASCII SP, space (32)>
...

Request = Simple-Request | Full-Request

Simple-Request = "GET" SP Request-URI CRLF

Seems to me there's exactly one space after GET.
I once had this problem, when several (rather bad) webservers
did not accept URLs with multiple spaces after GET.

>
> You don't know where to begin? How about explaining why, originally,
> Pine's LIST requests were designed for UW-IMAP-specific namespace and LIST

It is a good thing that UW allowed other people outside UW to use
IMAP, which is a great protocol! What would be if they hadn't made it
public for everyone?

> > Example 5: "Netscape Communicator insists that the response in
> > HEADER.FIELDS is terminated by a blank line, supposedly the end of message
> > headers." Goodness knows that I am no fan of Netscape's client, but in
> > this case it is correct in expecting this [RFC 2060, page 42].
>
> I have an idea: let's just add random empty lines to random commands, for
> no good reason. And because I said so, it must make sense, then.

If he gets this in the RFC and if most existing software accepts it, then
you'll have to obey.

Nice, isn't it?


Sam

unread,
May 6, 2002, 6:15:22 PM5/6/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <ab6ph5$foebp$1...@id-140432.news.dfncis.de>,
"Frank Klauert" <frank....@hotmail.de> writes:

>> > It is not IMAP-compatible (by design),
>>
>> Really? You better tell that to hundreds of thousands of users. They're
>> going to be shocked to find out that their IMAP mail clients are not really
>> supposed to work.
>
> According to the bug-liste it's not supposed to work with some of them...

Try reading the bug lists again, before passing judgment on them.

> In real live, many IMAP-servers are actually used with Netscape, Outlook Express,
> Pine, ...
>
> If you create your software the way it is not or hardly usable with clients
> that are used very often, then your software is pretty useless...

That's really going to be news to a whole bunch of people, who've been
minding their own business all along, and now someone comes in and tells
them their mail client is not supposed to work.

> BTW: Which client do you think does it right? If not Pine? Mutt????

Although I don't use mutt extensively, I haven't had any problems with it.
Looks fine to me. Recent versions of Mozilla have the bug with unsolicited
replies finally fixed, which was the only showstopper for Netscape/Mozilla.
MSOE is a lost cause. It's too much of a randomly moving target.
Depending on the actual combination of the Windows OS, the presence or
absence of any service packs, the installed version of MSIE, plus its
service packs, as well as the claimed version of MSOE, you may end up
everything between rock stable, to completely useless.

>> > For "Why not use courier":
>> >
>> > 1. http://www.inter7.com/courierimap/BUGS.txt
>> >
>> > As one clearly see: Courier is broken by design!
>>
>> Here's a question: do you actually open a URL, in order to get some vague
>> grasp of a clue on its contents, or do you pronounce judgment based solely
>> on the URL's path?
>
> If someone tells me that Pine (which simply is a reference implementation of
> IMAP) would be broken when it speaks IMAP, then this whole document simply
> MUST be a joke!

The question here is a "reference implementation" of what?

> And just how can you say something like that the RFC is broken?

It's a free country, and all that.

> By definition,
> the software is broken, not the RFC.

Crispin's own revision to RFC 2060 makes almost a hundred corrections to
the document.

Here you go: http://www.imc.org/draft-crispin-imapv; the list of 91
corrections and revisions to RFC 2060 begins at the bottom of page 85.

> If you ignore the RFC, then please don't
> call it IMAP.

You cannot implement the RFC to the letter, and end up with a functional
result.

> Maybe you should remove your client-bug-list or at least rewrite it to only
> complain about real problems in those clients.

Maybe you should avoid blathering off on subjects that you do not have
direct first-hand experience with.

>> > From: "Mark Crispin" <M...@CAC.Washington.EDU>
>>
>> Oh, more of Crispin's FUD... How un-original.
>
> Your clientbuglist is exactly what I would call FUD.

You can call it whatever you want. Until you actually know what you're
talking about, your comments aren't that very important.

> Don't use sentences like "Ask Microsoft to actually read the RFCs
> before implementing them." if you ignore RFCs yourself!

When you see an error, the logical thing is to ignore it. The other
alternative is becoming a mindless sheep that's only following the herd.

>> Can't you come up with
>> something on your own? Preferrably something that reads above a
>> grade-school intelligence level.
>
> Of course I can. (But I don't have to)

Don't be so modest.

> Why this stupid maildirformat? (Which Courier is about after all)

Don't ask me. Ask the thousands of engineers and technicians that have
tossed out legacy mail spools, and replaced them with maildir plants that
ended up handling more mail traffic, with less of a load.

> It simply can't work!

Really? Why do you carry on, like that, blindly believing that thousands
of people across the world simply can't be using what they already use?
It really makes you look very naive, and unsophisticated. Of course, I
can't really stop you from denying reality, if you're hell-bent on doing
so, I suppose. But I just thought that you might want to know that
something that you believe can't possibly work is now pulling in enough
traffic to rank in the top 5-10% of sourceforge.net-hosted projects.

Perhaps, just perhaps, you overlooked some key factors that drastically
changes the overall picture?

> If there is no UID-file (which is there), then
> IMAP can't work because you need to assign new messages individual
> ascending UIDs. No client which caches metadata or messages would
> be able to find the message in a maildir. This would not be IMAP,
> it would not allow disconnected use.
>
> But if you have this UID-mapping-file, then it's no maildir anymore:
> You'll need locking, you'll have to rebuild that file if it's broken...

Perhaps you should educate yourself regarding how the whole thing's
supposed to work, before making ignorant comments, like that. Somehow,
your grandioze theories notwithstanding, everything appears to work without
any need for locking.

Here's a free clue, Einstein: maildirs are designed to provide concurrent
and parallel access to individual mail messages without requiring any kind
of locking and synchronization. I'm hoping (perhaps vainly) that you know
enough about maildirs to at least accept that fact. Soooooo: how come it's
possible to access one set of data in concurrent, parallel, manner without
requiring locking or synchronization; but you can't (supposedly) do the
same with a second sent of data? Put that into your RFC, set the blender
on "high", and see what comes out.

> And I really don't like the idea to have one inode plus the rest of
> the cluster wasted for every mail, no matter how small it is.

It's going to come as a shock, for some, but the size of the average E-mail
message is between 5K-7K.

The first 49MB mailbox that I could get my hands on:

[root@ny Maildir]# du -s .
49016 .
[root@ny Maildir]# find . -name cur -exec find {} -type f -print \; | wc -l
6026

About 49MB, with slightly over 6000 messages, or about 8Kb per message.
Which happens to be a typical cluster size on most filesystems. I've got a
couple of messages in there with multi-megabyte attachments, that skews the
average a little bit. Take that out and the average message size falls
comfortably under 8K.

If you were to actually investigate the matter, you'd discover that, by
default, most modern filesystems allocate either 4K and 8K bytes per inode,
and are optimized for that average file size.

The inode argument is a red herring. Take a look at your filesystem, one
of these days. You'll be shocked to discover that you have many small
files all over the place. Most of them in /etc, but it's surprising where
they turn up. This is not an a.out world any more. Most ELF binaries are
relatively small, with most of the bloat being shoved into ELF-linked
dynamic objects.

>> Well, it's now been eight years since RFC 2060.
>> http://www.imap.org/products/showall.php now lists only 78 IMAP
>> applications. That's across ALL hardware/software platforms; and of all
>> possible categories: client, server, whatever. Why so little? Well,
>> before you choose to run your mouth and blather something about a topic
>> that you don't apparently know much about, think about it first. You'd
>> look less of a fool, then.
>
> Why don't you create your own remote-mail-protocol? Go ahead, do it,

I already have my own future plans set out, but thanks for the suggestions.
I'll definitely give them all the due consideration they deserve.

> publish it as RFC, implement a reference-server for it, a reference library
> to use it, persuade others to implement it...
>
> I think that IMAP is a great thing (especially disconnected use), a protocol
> urgently needed by everyone who seriously uses Email, and it's great that
> Mark developed it, published it, and that it is supported by the most
> widespread clients (in browsers) and many others.

I notice that you've neglected to mention servers, which is a key omission,
and is something that's worth pondering over.

> He has written a sever which is very good and really easy to install and

The server already comes pre-installed in most operating systems. It goes
without saying that it's real easy if someone else already did it for you.
And it's also real easy if you want to offload the messy details of user
authentication, and account management, to someone else as well.

> a library which is very flexible and useable (I really enjoyed copying
> mails from pop3 to imap with mbxcopy, it's a very good concept to have
> a uniform view to mails). So please don't mess around with that and try
> to pull it down.

Sorry, but I do not share your hero-worshipping afflictions.

> Maybe you should remove IMAP-support from Courier, if you hate it so
> much.

I think I'll keep it. Sometimes it's fun to annoy people that have a
perpetual holier-than-though attitude.

> I'm sure it would be the best for all of us. Maybe a NFS-mounted
> maildir is the right solution for your prefered audience. I don't
> see much difference for them.

Well, I do declare that the 'preferred audience' has already made its
decision.

> And I don't think it's possible to develop such a complex (complex
> because it is so very powerful compared to SMTP or POP3. Not because
> it would be possible to make it much simpler) protocol as RFC 2600
> (and to foresee every aspect of it's use in the future) without the
> need for some clarification afterwards.

Yes, it is.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE81wB23ejdWUS0ltARAottAKCELL0zmv5FETGLoSnfXIWfsaKKVACffDg2
sMQLKyvqcRg1+kZA1i3jQ0s=
=A10k
-----END PGP SIGNATURE-----

Sam

unread,
May 6, 2002, 6:37:33 PM5/6/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <ab6qes$fp018$1...@id-140432.news.dfncis.de>,
"Frank Klauert" <frank....@hotmail.de> writes:

> If anybody tells me what is wrong with DKIMAP I would
> be glad to hear about it. So far it works quite great
> here!

- From a casual glance, it's MIME parsing is incomplete. Try forwarding, as
a MIME attachment, a message with other attachment, and also add some
additional attachment to the forwarded message. This is what I got:

* OK ny.email-scan.com DKIMAP4 IMAP Server
a login mrsam ***********
a OK LOGIN completed
a select inbox
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft)]
* OK [UIDVALIDITY 1]
* 1 EXISTS
* 0 RECENT
a OK [READ-WRITE] SELECT completed
a fetch 1:* (envelope)
* 1 FETCH (ENVELOPE (NIL "test" NIL NIL NIL NIL NIL NIL NIL NIL))

Barf.

There was also the little matter of the server still running as root after
authentication; so I was one buffer overflow from a rooted box, but that's
easily fixed...


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE81wWr3ejdWUS0ltARAiHYAJ9trpPAVmOXGGkCj/JBs2pDf+1OpACeMkpS
MXfOPzJL1jEmYRLi0mpmxXk=
=NSQ6
-----END PGP SIGNATURE-----

Sam

unread,
May 6, 2002, 6:57:34 PM5/6/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <ab6uvu$fr229$1...@id-140432.news.dfncis.de>,
"Frank Klauert" <frank....@hotmail.de> writes:

>>
>> > Example 1: "Pine chokes on whitespace between BODY and [" is such a case;
>> > the syntax in the IMAP specification does not permit whitespace between
>> > BODY and [, and thus it is quite reasonable for Pine to "choke".
>>
>> IMAP does not have any kind of a formal grammar, or a lexical syntax. The
>> protocol is defined as an unstructured glop of regular expressions.
>
> And here we have again to quote you:
> "Ask [Microsoft] to actually read the RFCs
> before implementing them."
>
> Well, let's look at
> http://www.isi.edu/in-notes/rfc2060.txt
>
> What can we find there?
>
> -----------
>
> 9. Formal Syntax

Please avoid putting your foot in your mouth again, at least until you
learn terms such as "deterministic finite automaton", "non-deterministic
finite automaton", "context-free grammar", "LALR(1)", and others.

You don't know what you're talking about.

> What do you think is this?
> [x] BNF
> [x] something formal
> [x] something you can work with
> [?] something you don't understand

[x] A definition for large wad of spaghetti, designed to amuse
simple-minded fools into believing that it's D'Avincian art.


>> There is none. It's just a hairball of regular expressions.
>
> It was really easy to find BNF. But I just haven't seen this
> hairball of regular expressions. Where is it?

It's right there in front of your eyes, except that you don't know it. How
said.

>> I defy anyone to cite any text-based wire protocol that demands explicit
>> presense, or absence, of whitespace delimiters.
>
> Well, let's try a exotic protocol which is hardly used today:
>
> HTTP?

HTTP consists of less than a dozen simple commands: CONNECT, GET, PUT,
POST, and one or two others. It is a completely stateless, line-oriented
protocol, compared to IMAP being a stateful protocol, with multiple,
recursive structures.

You can continue to embarass yourself, as long as you would like. I have
all evening.

I only with IMAP was as simple as HTTP.

>> You don't know where to begin? How about explaining why, originally,
>> Pine's LIST requests were designed for UW-IMAP-specific namespace and LIST
>
> It is a good thing that UW allowed other people outside UW to use
> IMAP, which is a great protocol! What would be if they hadn't made it
> public for everyone?

Someone else would've designed something similar; hopefully it would be
more sane and we would have many more applications that use it, eight years
later.

>> > Example 5: "Netscape Communicator insists that the response in
>> > HEADER.FIELDS is terminated by a blank line, supposedly the end of message
>> > headers." Goodness knows that I am no fan of Netscape's client, but in
>> > this case it is correct in expecting this [RFC 2060, page 42].
>>
>> I have an idea: let's just add random empty lines to random commands, for
>> no good reason. And because I said so, it must make sense, then.
>
> If he gets this in the RFC and if most existing software accepts it, then
> you'll have to obey.
>
> Nice, isn't it?

Yeah. So was the Barnum-and-Bailey circus, when it visited NYC last month.
What a show.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE81wpU3ejdWUS0ltARAv6UAJ9VZ6XdshZcrpOa59sMv0uKUtIarwCgmBUq
hXLQcQ4ODqREDQZzWbelsKc=
=yK8+
-----END PGP SIGNATURE-----

Sam

unread,
May 6, 2002, 6:59:07 PM5/6/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <Pine.LNX.4.50.02050...@shiva0.cac.washington.edu>,
Mark Crispin <m...@CAC.Washington.EDU> writes:

> There is a torture test message on:
> ftp://ftp.cac.washington.edu/mail/mime/torture-test.mbox that is
> helpful in testing IMAP servers. It is truly horrible; it has multiparts
> inside nested messages inside multiparts; it violates the MIME
> specification all over the place (many of those violations comply with old
> preliminary drafts of MIME); etc.

I've never seen this file before. Rather than reproduce the monstrosity
that BODYSTRUCTURE spits out, I'll just reproduce what Pine shows after
being fed Courier-IMAP's output:

Subject: Multi-media mail demonstration
Parts/Attachments:
1 Shown 3 lines Text, "Explanation"
2 Shown 4.9 KB Message, "Rich Text demo"
2.1 OK 16 lines Text
2.2.1 Shown ~13 lines Text
2.3 OK 917 bytes Application
3 Shown 562 KB Message, "Voice Mail demo"
3.1 420 KB Audio, "Hi Mark"
4 27 KB Audio, "Flint phone"
5 OK 1.3 KB Image, "MTR's photo"
6 Shown 182 KB Message, "Star Trek Party"
6.1.1 Shown 16 lines Text
6.1.2 23 KB Audio, "He's dead, Jim"
6.2.1 OK 19 KB Image, "Kirk/Spock/McCoy"
6.2.2 OK 13 KB Image, "Star Trek Next Generation"
6.2.3 46 KB Application
6.2.4 9.2 KB Application
6.3 35 KB Audio, "Distress calls"
7 Shown 86 KB Message, "Digitizer test"
7.1 Shown 0 lines Text
7.2 OK 63 KB Image, "Bellcore mug"
7.3 Shown 8 lines Text
8 Shown 74 KB Message, "More Imagery"
8.1 Shown 26 lines Text
8.2 OK 53 KB Image, "Mail architecture slide"
9 Shown 398 KB Message, "PostScript demo"
9.1 OK 397 KB Application, "Captain Picard"
10 OK ~78 KB Image, "Quoted-Printable test"
11 Shown 103 KB Message, "q-p vs. base64 test"
11.1 ~62 KB Audio, "I'm sorry, Dave (q-p)"
11.2 30 KB Audio, "I'm sorry, Dave (BASE64)"
12 Shown 314 KB Message, "Multiple encapsulation"
12.1 OK 40 KB Application, "The Simpsons!!"
12.2 13 KB Binary, "Alice's PDP-10 w/ TECO & DDT"
12.3 Shown 241 KB Message, "Going deeper"
12.3.1 Shown 7 lines Text
12.3.2.1 OK 2.4 KB Image, "Bunny"
12.3.2.2 117 KB Audio, "TV Theme songs"
12.3.3 4.9 KB Application
12.3.4 Shown 75 KB Message, "Yet another level deeper..."
12.3.4.1 56 KB Audio, "I'm Twying..."

When I fed this to dkimap4, it segfaulted in strcpy().

Maybe I didn't install it right, but my install of dkimap ran as root even
after authentication. Unless I made an error installing the server, this
means that anyone running dkimap4 on the Internet has a rootable box.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE81wq53ejdWUS0ltARAqCpAJ0cu55RDMdZXdtR1eSCbXQuAQHnBwCeJogG
XjtN3Hz1KXiEfx/5KWSjz1k=
=YOUz
-----END PGP SIGNATURE-----

Mark Crispin

unread,
May 6, 2002, 7:06:21 PM5/6/02
to
On Mon, 6 May 2002, Sam wrote:
> Your specifications are ambiguous, unclear, mis-designed, and in many cases
> are clearly wrong and don't even match up with UW-IMAP's, or Pine's,
> output, in some cases.

It it curious that you feel this way, yet felt obliged to come out with an
IMAP server product.

In any case, the IMAP WG and the IESG did not feel this way, since both
bodies approved the document.

There is now a revision to RFC 2060 under consideration by the IESG. If
you have concerns with the document, and have revisions to propose, you
should bring them up there. That would be a constructive action.
Flinging insults at me is not.

> > Example 1: "Pine chokes on whitespace between BODY and [" is such a case;
> > the syntax in the IMAP specification does not permit whitespace between
> > BODY and [, and thus it is quite reasonable for Pine to "choke".
> IMAP does not have any kind of a formal grammar, or a lexical syntax. The
> protocol is defined as an unstructured glop of regular expressions.

This is an irrelevant conclusion. Attacking the form in which the IMAP
protocol is defined does not disprove an assertation that IMAP does not
permit whitespace between BODY and [.

> Right. If you want to know if there's anything in the folder, wow, what a
> brilliant idea: just open the folder and check. Don't assume that if the
> folder exists, it must be non-empty.

You do not approve of Pine's behavior. That's fine.

However, disapproval is not equivalent to "bug".

> > Example 3: "Pine fails to quote certain special characters in the login
> > userid and password." The characters in question are not atom-specials as
> > defined in the IMAP specification, and therefore do not need to be quoted.
> This goes back to point 1: the protocol's syntax, and grammar, is a mess.
> Good design, in practice, indicates that any protocol above some level of
> complexity should have a well-defined lexical syntax and grammar. I defy
> anyone to present a complete definition of IMAP's syntax or grammar.
> There is none. It's just a hairball of regular expressions.

Another irrelevant conclusion. Attacks on the form in which the IMAP
protocol is define do not disprove an assertation that the characters do


not need to be quoted.

The question of whether the IMAP protocol's syntax and grammar is properly
defined is a red herring. I do not agree with your opinions on that
question, but I don't have to; that is not the matter at hand.

> I defy anyone to cite any text-based wire protocol that demands explicit
> presense, or absence, of whitespace delimiters.

What other protocols specify is also a red herring.

It is irrelevant whether any other text-based wire protocols demand
explicit presence, or absence, of whitespace delimiters. IMAP's
requirements are defined by its protocol, not by any other protocol.

I don't know what would be gained by citing protocols which have explicit
requirements for whitespace. I doubt that an apology (or even begrudging
acceptance) would be forthcoming; and it distracts from the main issue,
which is that IMAP has explicit and strict requirements for whitespace.

> > Example 4: A complete non sequitor: "RFC 2060 clearly indicates that
> > a non-empty reference is non-standard behavior." I don't know where to
> > begin on this.
> It's really very simple: you define, yourself, that a certain form of the
> LIST command is implementation-defined: namely a non-empty reference
> parameter for LIST. Then you go ahead and have Pine spit out non-empty
> LIST references.

The non-sequitor is in the claim that a standard-defined mechanism is
non-standard by virtue of having implementation-defined results.

Something can not simultaneously be defined in a standard and be
non-standard.

> You don't know where to begin? How about explaining why, originally,
> Pine's LIST requests were designed for UW-IMAP-specific namespace and LIST
> semantics, instead of any IMAP server's implementation.

This presupposes a definite answer to another question which has not been
asked.

Before asking "why, originally, Pine's LIST requests were designed for


UW-IMAP-specific namespace and LIST semantics, instead of any IMAP

server's implementation?" it is first necessary to ask "were Pine's LIST
requests designed for UW-IMAP-specific namespace and LIST semantics,
instead of any IMAP server's implementation?"

The answer to that question is "no".

The reference argument to the LIST command was specifically designed and
added to the specification (necessitating the replacement of the former
FIND command with LIST) four years before Pine used that functionality.

IMAP servers implemented it long before Pine did. As it turned out,
Cyrus' initial implementation was not particularly useful, and it was
changed. RFC 2683 discusses the matter in greater detail.

> > Example 5: "Netscape Communicator insists that the response in
> > HEADER.FIELDS is terminated by a blank line, supposedly the end of message
> > headers." Goodness knows that I am no fan of Netscape's client, but in
> > this case it is correct in expecting this [RFC 2060, page 42].
> I have an idea: let's just add random empty lines to random commands, for
> no good reason. And because I said so, it must make sense, then.

It is not "for no good reason". It is because RFC 2060 page 42 requires
it.

Mark Crispin

unread,
May 6, 2002, 7:25:47 PM5/6/02
to
On Mon, 6 May 2002, Sam wrote:
> I've never seen this file before. Rather than reproduce the monstrosity
> that BODYSTRUCTURE spits out, I'll just reproduce what Pine shows after
> being fed Courier-IMAP's output:

So far, so good. That is the correct output, and thus a tenative pass of
the torture test. You have the correct number of body parts, correct body
part identifiers, correct sizes, and correct content descriptions.

Make sure that fetches of each part return the correct data, and if
possible try to play the sounds and view the pictures. There have been
cases of servers which returned a good body structure, which weren't
actually able to handle BODY[12.3.4.1].

If each segment of the torture test is correctly fetched, the next thing
to do is to try bogus section specifiers, e.g. BODY[11.4], and make sure
those return NIL.

Finally, if all that comes through without problems, the final task is to
compare the horrendous BODYSTRUCTURE output between different servers.
For each mismatch, determine whether it's a fault or not. There will be
some differences, especially in how invalid MIME is handled (e.g. treating
"TEXT" as "TEXT/PLAIN"), in which a mismatch is not a fault.

If, after doing all this, no fault has been found, then the torture test
has been passed.

If Courier accomplishes a pass on the torture test the first time, that
will be an impressive accomplishment, regardless of any other issues. I
give credit where credit is due.

Sam

unread,
May 6, 2002, 8:16:13 PM5/6/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <Pine.LNX.4.50.02050...@shiva1.cac.washington.edu>,
Mark Crispin <m...@CAC.Washington.EDU> writes:

> It it curious that you feel this way, yet felt obliged to come out with an
> IMAP server product.

> In any case, the IMAP WG and the IESG did not feel this way, since both
> bodies approved the document.

Is that supposed to validate it, somehow? Since when did anything the IESG
does automatically proclaimed to be logical, or consistent?

Nobody's perfect, you know. Everyone makes mistake.

>> IMAP does not have any kind of a formal grammar, or a lexical syntax. The
>> protocol is defined as an unstructured glop of regular expressions.
>
> This is an irrelevant conclusion.

This is not a conclusion, but an observation. There's a key conceptual
difference between the two processes.

> Attacking the form in which the IMAP
> protocol is defined

Not the form, but the actual substance. You can dress it up with many
layers of make-up, but despite anyone's efforts, it's still the same wad of
spaghetti, underneath.

> does not disprove an assertation that IMAP does not
> permit whitespace between BODY and [.

And misrepresenting my arguments does not disprove the protocol's design
faults.

>> Right. If you want to know if there's anything in the folder, wow, what a
>> brilliant idea: just open the folder and check. Don't assume that if the
>> folder exists, it must be non-empty.
>
> You do not approve of Pine's behavior. That's fine.
>
> However, disapproval is not equivalent to "bug".

I guess I'm just a perfectionist: I consider an implementation flaw to be
the equivalent of a bug.

>> > Example 3: "Pine fails to quote certain special characters in the login
>> > userid and password." The characters in question are not atom-specials as
>> > defined in the IMAP specification, and therefore do not need to be quoted.
>> This goes back to point 1: the protocol's syntax, and grammar, is a mess.
>> Good design, in practice, indicates that any protocol above some level of
>> complexity should have a well-defined lexical syntax and grammar. I defy
>> anyone to present a complete definition of IMAP's syntax or grammar.
>> There is none. It's just a hairball of regular expressions.
>
> Another irrelevant conclusion.

What exactly is irrelevant about not having any kind of a separation,
whatsoever, between the lexical and the grammatical structure of the
protocol? It is a very relevant point to make concerning the overall
soundness, and well-formness, of the design.

You know, even back in BASIC days, circa the '80s, they figured out that
you have to stick a $ (or a %) to a variable name, so that the program
could be parsed in a sane manner without leading to ambiguity between
keyword names, and other parts of the language's grammar. Even back then
they had the good sense to avoid treating any unrecognized keyword as a
variable, requiring a $ (or a %) only when the variable name matches any
reserved keyword.

> Attacks on the form in which the IMAP
> protocol is define do not disprove an assertation that the characters do
> not need to be quoted.

Once again you are building a strawman. The form used for defining IMAP is
irrelevant, the actual IMAP definition is flawed. The form is indeed
directly dictated by the underlying definition; however the real problem
goes much further than that.

> The question of whether the IMAP protocol's syntax and grammar is properly
> defined is a red herring. I do not agree with your opinions on that
> question, but I don't have to; that is not the matter at hand.

However it is the matter at hand, which I keep bringing up. Everytime I
do, you try to steer it off on a tangent, since you don't want to discuss
it. Well, I do and if you don't, that's fine; perhaps someone else will
care to address them instead of you. Go and appoint a spokesman, or
something.

>> I defy anyone to cite any text-based wire protocol that demands explicit
>> presense, or absence, of whitespace delimiters.
>
> What other protocols specify is also a red herring.

Well, a wise man learns from history. Reading and understanding existing
protocols -- their faults and accomplishments -- is probably the best thing
that anyone can do if they plan to build something of their own. Learn
from others' mistakes, and all that. It is just unfortunate that too many
people ignore the wealth of historical precendent and accumulated
knowledge, before they go charging off, half-cocked, into darkness.

> It is irrelevant whether any other text-based wire protocols demand
> explicit presence, or absence, of whitespace delimiters.

Well, I hope that I'll never grow to have such as close mindset. In the
event I ever choose to design something new, as opposed to reimplementing
something already exists, I hope that I'll have enough smarts to learn,
study, and understand how similar things worked in the past; and that I'll
be able to learn from their successes. Or their failures.

>
> I don't know what would be gained by citing protocols which have explicit
> requirements for whitespace.

Perhaps you would then understand why so many IMAP clients implement IMAP
so poorly, as you, yourself, have often complained? Or why there are so
comparatively few IMAP servers in existence? Come on, eight years
down the road there should be more than just four POSIX IMAP servers out
there (and only 24 knows IMAP servers in the entire world).

How come there isn't anything like the "ESMTP Product Database" (not that I
know of), that tries to help people to find an ESMTP application that they
can use?

Maybe there'd be less bellyaching on the subject, once the underlying root
causes of both phenomenon (and they are certainly related) are understood.

> I doubt that an apology (or even begrudging
> acceptance) would be forthcoming; and it distracts from the main issue,
> which is that IMAP has explicit and strict requirements for whitespace.

And which is absolutely, positively, and unquestionably, silly. That's it.
There's no other way of putting it. Mandating explicit whitespacing is
just silly. I'd prefer to use something other than "silly" to describe it,
but there are kids present.

>> > Example 4: A complete non sequitor: "RFC 2060 clearly indicates that
>> > a non-empty reference is non-standard behavior." I don't know where to
>> > begin on this.
>> It's really very simple: you define, yourself, that a certain form of the
>> LIST command is implementation-defined: namely a non-empty reference
>> parameter for LIST. Then you go ahead and have Pine spit out non-empty
>> LIST references.
>
> The non-sequitor is in the claim that a standard-defined mechanism is
> non-standard by virtue of having implementation-defined results.
>
> Something can not simultaneously be defined in a standard and be
> non-standard.

Well, you wrote it yourself:

A non-empty
reference name argument is the name of a mailbox or a level of
mailbox hierarchy, and indicates a context in which the mailbox
name is interpreted in an implementation-defined manner.
=============================


Well, there you go. By using a non-empty reference name for LIST, you are
relying on an implementation-defined server behavior. Therefore, it can no
longer be claimed that Pine is designed to be interoperable with any IMAP
server, since it clearly depends on the implementation-defined behavior of
UW-IMAP's LIST command.

Henceforth, let it be known that Pine failing to work with a particular
IMAP server does not necessarily mean that the server breaks some
particular aspect of the IMAP protocol. It might mean that the server
simply does not reimplement a UW-IMAP specific extension to -- or an
implementation-defined aspect of -- the IMAP protocol.

In that case, I do not then understand how Pine should then be considered a
reference implementation, when a key portion of its functionality depends
on the UW-IMAP server's implementation-defined behavior. A reference
implementation, by definition, should not rely on any particular
implementation's behavior.

At least that's what I always thought a reference implementation should
be...

> Before asking "why, originally, Pine's LIST requests were designed for
> UW-IMAP-specific namespace and LIST semantics, instead of any IMAP
> server's implementation?" it is first necessary to ask "were Pine's LIST
> requests designed for UW-IMAP-specific namespace and LIST semantics,
> instead of any IMAP server's implementation?"
>
> The answer to that question is "no".
>
> The reference argument to the LIST command was specifically designed and
> added to the specification (necessitating the replacement of the former
> FIND command with LIST) four years before Pine used that functionality.

And that does not answer, in any way, why a supposed reference
implementation relies on implementation-defined behavior. There's no need
to generate long-worded paragraphs, in an attempt to avoid the subject. I
think a simple answer should be possible here.

>> > Example 5: "Netscape Communicator insists that the response in
>> > HEADER.FIELDS is terminated by a blank line, supposedly the end of message
>> > headers." Goodness knows that I am no fan of Netscape's client, but in
>> > this case it is correct in expecting this [RFC 2060, page 42].
>> I have an idea: let's just add random empty lines to random commands, for
>> no good reason. And because I said so, it must make sense, then.
>
> It is not "for no good reason". It is because RFC 2060 page 42 requires
> it.

Something that's required simply for the sake of requiring it, can't be a
very good reason. Even a "good reason".

I have another great idea: let's go out and design various protocols, parts
of which have absolutely no logical basis to them whatsoever. Let's try to
come up with the most ridiculous, laughable protocols, and when asked to
justify why they're there we'll simply say that it's because the defining
document states so.

Gee, it's so easy to define a protocol, these days... It appears that
logical thought, and reason, are no longer required...

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE81xzH3ejdWUS0ltARAsl3AJ9AeWBTXPdVGAu+NZ7p4JTfMUoSxwCaAxkN
BDAyZnSeUc3zXmvRV1xqy7w=
=Zj1s
-----END PGP SIGNATURE-----

Nancy McGough

unread,
May 7, 2002, 4:16:38 AM5/7/02
to
On 6 May 2002 Sam (s...@email-scan.com) wrote:

>
> "Frank Klauert" <frank....@hotmail.de> writes:
>
> > BTW: Which client do you think does it right? If not Pine? Mutt????
>
> Although I don't use mutt extensively, I haven't had any problems with it.
> Looks fine to me. Recent versions of Mozilla have the bug with unsolicited
> replies finally fixed, which was the only showstopper for Netscape/Mozilla.
> MSOE is a lost cause. It's too much of a randomly moving target.
> Depending on the actual combination of the Windows OS, the presence or
> absence of any service packs, the installed version of MSIE, plus its
> service packs, as well as the claimed version of MSOE, you may end up
> everything between rock stable, to completely useless.

I'm also curious about what IMAP clients you, and other IMAP
developers, think are good. On my IMAP Service Providers page, I
have a section about IMAP clients, which is here

<http://www.ii.com/internet/messaging/imap/isps/#clients>

and I don't know what to recommend as IMAP clients other than
Mulberry and Pine. Mutt doesn't seem that great as an IMAP client
because of the reasons I list in the last bullet of the Mutt vs
Pine section of my main Pine page

<http://www.ii.com/internet/messaging/pine/#muttVpine>

What do you, Sam, use as your primary mail client? I'm pretty
sure that the UW IMAP people use Pine and that the Cyrus people
use Mulberry, but I'd love to know what other IMAP developers
use.

Thanks,
Nancy


REFERENCE:
The message I'm replying to -- and this entire thread & group --
may be available at

<http://groups.google.com/groups?selm=courier.3CD7...@ny.email-scan.com>

Ian G Batten

unread,
May 7, 2002, 4:29:21 AM5/7/02
to
In article <Pine.LNX.4.50.020506...@shiva0.cac.washington.edu>,

Mark Crispin <m...@CAC.Washington.EDU> wrote:
> many variables which are not always obvious. A one-message/one-file
> format such as mh or maildir can be "very poor" or "good" depending a
> great deal upon how fast stat() and open() are and if per-message metadata
> can be acquired other than with stat(). On Linux, these calls are fast;

I was very interested to see how Cyrus would perform for many users,
when I had historically been using MH for myself on a similar platform
(Solaris). ``scan +inbox'' was not exactly fast, and it worried me that
Cyrus would be doing this on a regular basis for many more users.

It's certainly noticeable that the occasions when Cyrus has to rebuild
its metadata cache on a mailbox, the performance is less than sparkling,
but once the cache is in step with reality everything is fine. I
deduce, unscientifically, from this that in Cyrus's case the metadata
cache is absolutely critical to performance, and without it the
requirement to not merely stat() but open() and read() every message to
generate a scan listing of a mailbox would be an intolerable load.
Which in turn dictates Cyrus's need to absolutely own the mailboxes, to
ensure it knows when caches are valid.

Of course, for systems with small numbers of users, the mere fact that
most of the messages will end up in RAM finesses the issue.

ian

Bertrand

unread,
May 7, 2002, 4:32:42 AM5/7/02
to
[Below are not a troll, just the opinions of an almost happy courier-imap user]

> Courier sucks!
> It is not IMAP-compatible (by design), the author is an idiot and the format
> it uses is insane.

You're talking about Maildir? Maildir is a good storage format

> Cyrus has the problem that it won't accept 8bit-Mails by default.
> I was able to choose between "reject" and "destroy mail by replacing
> everything >127 with X". There are rumours that this _incorrect_
> behaviour can be patched away.

Courier can handle them.

> Want a really good IMAP-Server? Really fast, good coding style (C++),
> made for >1GB-mailboxes, very easy to install?
>
> http://www.dbox.handshake.de/dkimap4/
>
> It's just a great product, IMO!
>
>> Can someone with user experience with the above 2 imapd software provide me
>> with direction as to which one i should use, as well as why?
>
> For "Why not use courier":
>
> 1. http://www.inter7.com/courierimap/BUGS.txt

Guys of dkimp4 don't made a page referencing their bugs? There are none?

>> Also as with
>> uw-imapd i have to create drafts and sent folders under the inbox folders in
>> order for squirrel mail and outlook to use. Is there a imapd server that had
>> inbox/draft/sent mailboxes on an equivalent hierarchy?
>
> Cyrus has a problem with this.
> All your mailboxes must be subfolders of your INBOX.

It's a namespace issue.

> Cyrus will force you to set the path to your mail to "INBOX". (It's somewhere
> in the account-options of Outlook Express and Outlook).
>
> But that is just a workaround. Better use DKIMAP. It has no problems like
> this!

So it doesn't implement RFC 2342?

--
Bertrand

This is not a bug. It is a missing feature.

Minh Trieu

unread,
May 7, 2002, 5:59:10 AM5/7/02
to
Wow more info then i needed :-)

I'm not going to try Dkmap4 its too early in its development, not enough
support documents etc.
It has to be one of the popular IMAP servers for linux, UW, Cyrus or
Courier. UW is slow, getting locking problems, its mailbox consisting of a
large flat file is horrible. Can UW work with other mailbox format?
I think i might go with Cyrus, since people are saying Courier IMAP come
with all these design flaws. Are there any particular features of Courier
that Cyrus cant match up too? how does maildir compete with the way cyrus
stores its mail? Cyrus stores accounts in a sasldb file but can made made to
use PAM/LDAP/MYSQL, can courier do the same thing?

Thanks for everyones input :-)
Minh


"Minh Trieu" <minh...@optushome.com.au> wrote in message
news:3cd6538f$0$15143$afc3...@news.optusnet.com.au...

DINH Viet Hoa

unread,
May 7, 2002, 7:56:44 AM5/7/02
to
Maybe you should look at the lastest IMAP draft, many errors
have been fixed.

http://search.ietf.org/internet-drafts/draft-crispin-imapv-16.txt

Still the problem that there is an historical dependance on RFC 822.

The IMAP protocol is complex to implement, that's why there are not
so many client. Because it offers a lot of services, it is complex.
But I still think that it is worth.

Maybe some changes could be done in order to get easier performant
(as few copies as possible on FETCH requests). But it would rather
make it easier to the implementation to do that in a performant
way rather than really make it more performant.

Currently I believe that many IMAP client store all
FETCH response or all the response in memory, although the entire
response may not fit in memory. although it could simply store the
response on disk.

--
DINH V. Hoa,
libEtPan! - a mail library - http://libetpan.sourceforge.net

Sam

unread,
May 7, 2002, 9:00:12 AM5/7/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <Pine.WNT.4.44.0205070905300.-4003781-100000@no>,
Nancy McGough <nm-this-addr...@no.sp.am> writes:

> What do you, Sam, use as your primary mail client? I'm pretty
> sure that the UW IMAP people use Pine and that the Cyrus people
> use Mulberry, but I'd love to know what other IMAP developers
> use.

I don't really use a specific client most of the time. Depending on the
situation I use sqwebmail, pine, mutt, and Mozilla.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE818/Y3ejdWUS0ltARAtNKAKCap894JshLtqhe+hOUMgK3ng1E/wCgk1jU
gEcfBj1MwnLTV4wNVv6QtZA=
=QMri
-----END PGP SIGNATURE-----

Nancy McGough

unread,
May 7, 2002, 9:46:51 AM5/7/02
to
On 7 May 2002 Sam (s...@email-scan.com) wrote:
>
> In article <Pine.WNT.4.44.0205070905300.-4003781-100000@no>,
> Nancy McGough <nm-this-addr...@no.sp.am> writes:
>
> > What do you, Sam, use as your primary mail client? I'm pretty
> > sure that the UW IMAP people use Pine and that the Cyrus people
> > use Mulberry, but I'd love to know what other IMAP developers
> > use.
>
> I don't really use a specific client most of the time. Depending on the
> situation I use sqwebmail, pine, mutt, and Mozilla.

Thanks Sam. What clients do other power IMAP people use and
recommend? What do people think of Mozilla as an IMAP client?
What about Evolution? From what I've read, it seems that many
(probably most) mail clients are pretty wimpy when it comes to
taking advantage of the full IMAP protocol.

Thanks,
Nancy


REFERENCE:
The message I'm replying to -- and this entire thread & group --
may be available at

<http://groups.google.com/groups?selm=courier.3CD7...@ny.email-scan.com>

--
IMAP Providers List: <http://www.ii.com/internet/messaging/imap/isps/>

Sam

unread,
May 7, 2002, 12:06:30 PM5/7/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <Pine.WNT.4.44.0205071441400.-3997921-100000@no>,
Nancy McGough <nm-this-addr...@no.sp.am> writes:

> On 7 May 2002 Sam (s...@email-scan.com) wrote:
>>
>> In article <Pine.WNT.4.44.0205070905300.-4003781-100000@no>,
>> Nancy McGough <nm-this-addr...@no.sp.am> writes:
>>
>> > What do you, Sam, use as your primary mail client? I'm pretty
>> > sure that the UW IMAP people use Pine and that the Cyrus people
>> > use Mulberry, but I'd love to know what other IMAP developers
>> > use.
>>
>> I don't really use a specific client most of the time. Depending on the
>> situation I use sqwebmail, pine, mutt, and Mozilla.
>
> Thanks Sam. What clients do other power IMAP people use and
> recommend?

Different mail clients are better at different tasks. That's why I don't
have a particular preference. Some things are work better in Pine, others
in mutt; still others in Mozilla; and so on. I do not believe that any one
particular client has evolved to a one-size-fits all stage.

> What do people think of Mozilla as an IMAP client?
> What about Evolution? From what I've read, it seems that many
> (probably most) mail clients are pretty wimpy when it comes to
> taking advantage of the full IMAP protocol.

Recently most of the annoying bugs in Mozilla were fixed. Since then,
Mozilla has become a fairly usable product.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE81/uE3ejdWUS0ltARAvOrAJ9nU/lEsQZv+YUl5ZhASdbG0uxX9wCggSRI
aKgr5YVJO7oZNH+0Cwb0MMI=
=qMXF
-----END PGP SIGNATURE-----

Mark Crispin

unread,
May 7, 2002, 11:52:34 AM5/7/02
to
On 7 May 2002, Ian G Batten wrote:
> I
> deduce, unscientifically, from this that in Cyrus's case the metadata
> cache is absolutely critical to performance, and without it the
> requirement to not merely stat() but open() and read() every message to
> generate a scan listing of a mailbox would be an intolerable load.

I agree completely. Cyrus' fundamental architecture is based on the
presumption that most access is metadata access.

> Which in turn dictates Cyrus's need to absolutely own the mailboxes, to
> ensure it knows when caches are valid.

Also correct.

Clifton Royston

unread,
May 7, 2002, 7:27:33 PM5/7/02
to
Minh Trieu <minh...@optushome.com.au> wrote:
> Wow more info then i needed :-)

> I'm not going to try Dkmap4 its too early in its development, not enough
> support documents etc.
> It has to be one of the popular IMAP servers for linux, UW, Cyrus or
> Courier. UW is slow, getting locking problems, its mailbox consisting of a
> large flat file is horrible. Can UW work with other mailbox format?

Yes, it can. About 8, if I recall correctly, including some
directory-type formats. mbox format ("large flat file") is the default
because UW IMAP has the goal of working out-of-the-box with other UNIX
mail software, and that's the default UNIX mailbox format.


> I think i might go with Cyrus, since people are saying Courier IMAP come
> with all these design flaws. Are there any particular features of Courier
> that Cyrus cant match up too? how does maildir compete with the way cyrus
> stores its mail?

Cyrus is kind of its own universe in terms of mail storage. Read their
docs. It does seem to be becoming popular with people setting up very
large mail sites.

-- Clifton

--
Clifton Royston -- LavaNet Systems Architect -- clif...@lava.net
"What do we need to make our world come alive?
What does it take to make us sing?
While we're waiting for the next one to arrive..." - Sisters of Mercy

Frank Klauert

unread,
May 8, 2002, 12:01:53 PM5/8/02
to
"Mark Crispin" <m...@CAC.Washington.EDU> wrote:
> There are, however, a number of client traps which await even a perfectly
> compliant server.

I neither know nor believe in DKIMAP being the _perfect_ compilant server.

My personal reasons to use it are that it has a very fast storage
formatm, that it allows subfolders with messages also at toplevel
hierachy and that _I_ had no interoperability problems with
- Mulberry
- Mozilla
- Outlook Express
- the DMAIL-utility which uses c-client

I just want to make clcear that I don't want to promote DKIMAP for
being the best in standards-compilance, although I don't know any
other problems beside this mime-test.

Are there further methods of testing it beside using the testmail?

> There is a torture test message on:
> ftp://ftp.cac.washington.edu/mail/mime/torture-test.mbox that is
> helpful in testing IMAP servers. It is truly horrible; it has multiparts
> inside nested messages inside multiparts; it violates the MIME
> specification all over the place (many of those violations comply with old
> preliminary drafts of MIME); etc.

Thanks for pointing me and others to this message!

I tried it with DKIMAP and it crashed. I've sent the URL and
the error-message to the author and I'm quite sure and hope that this
problem will be gone soon.

BUT I also have to say that for ME this problem is not very annoying,
because until now I don't regularly use clients which use the
MIME-features of IMAP (Outlook Express e.g. always fetches the complete
message to do its own parsing.)


Frank Klauert

unread,
May 8, 2002, 12:01:55 PM5/8/02
to
"Sam" <s...@email-scan.com> wrote:
> >> Right. If you want to know if there's anything in the folder, wow, what a
> >> brilliant idea: just open the folder and check. Don't assume that if the
> >> folder exists, it must be non-empty.
> >
> > You do not approve of Pine's behavior. That's fine.
> >
> > However, disapproval is not equivalent to "bug".
>
> I guess I'm just a perfectionist: I consider an implementation flaw to be
> the equivalent of a bug.

You should make clear how you define the word "bug" and maybe
replace it with more commonly used words like "design flaw".

Maybe it's time to update your client-"bug"-list NOW!

> >> I defy anyone to cite any text-based wire protocol that demands explicit
> >> presense, or absence, of whitespace delimiters.
> >
> > What other protocols specify is also a red herring.
>
> Well, a wise man learns from history. Reading and understanding existing
> protocols -- their faults and accomplishments -- is probably the best thing
> that anyone can do if they plan to build something of their own. Learn
> from others' mistakes, and all that.

It believe Mark did exactly this.
Having a exact definition of a protocol is a good thing, not allowing
more than one whitespace makes much sense! There's no reason to say
"You can randomly pick any amount of whitespace". Having exactly
one whitespace is much better, the parser is simpler and there's
no doubt (except with you) how the command should look like.

Of course it's also good to accept extra spaces if you get more compilance
with existing software:
Be liberal in what you accept but not loose in what you generate!

This seems to be one of the most important rules in protocol-implementation,
maybe you can give it a try!

> > It is not "for no good reason". It is because RFC 2060 page 42 requires
> > it.
>
> Something that's required simply for the sake of requiring it, can't be a
> very good reason. Even a "good reason".

Requiring from anybody to use more than one whitespace where exactly one
whitespace is just fine is also "for no good reason".

Maybe you can come up with really good reasons why it's so damn important for
you to have more than one?

>
> Gee, it's so easy to define a protocol, these days... It appears that
> logical thought, and reason, are no longer required...
>

Sounds to me like an ideal situation for YOU to design and implement
your own replacement for IMAP.

Don't just complain about Mark, show us that you can do better.
NOW!


Frank Klauert

unread,
May 8, 2002, 12:01:57 PM5/8/02
to
"Mark Crispin" <m...@CAC.Washington.EDU> wrote:
> On Mon, 6 May 2002, Frank Klauert wrote:
> > You kind of promised (as far as one could say that when it's about free
> > software of cource...) in one posting a new version of the mbx-format
> > which would cache more meta-information.
>
> There are two projects in progress. One is by a fellow in the Czech
> Republic named Martin Devera, the other is by our local computer center.
> I don't think that either one is in a shape for others to try it yet.

I guess that there is no homepage or similar yet?

Can you say anything about when they *might* be ready?

> I may do a third one of my own, but strictly limited to compatible
> changes (upwards and downwards) to the existing mbx format.

If you have the time to (and if there is no other information
for it in the web) it would be very interessting to know how the
extensions to mbx could look like and how they would work.


Frank Klauert

unread,
May 8, 2002, 12:01:59 PM5/8/02
to
> > Well, let's look at
> > http://www.isi.edu/in-notes/rfc2060.txt
> >
> > What can we find there?
> >
> > -----------
> >
> > 9. Formal Syntax
>
> Please avoid putting your foot in your mouth again, at least until you
> learn terms such as "deterministic finite automaton", "non-deterministic
> finite automaton", "context-free grammar", "LALR(1)", and others.

I know about these, believe it or not!

> >> I defy anyone to cite any text-based wire protocol that demands explicit
> >> presense, or absence, of whitespace delimiters.
> >
> > Well, let's try a exotic protocol which is hardly used today:
> >
> > HTTP?
>
> HTTP consists of less than a dozen simple commands: CONNECT, GET, PUT,
> POST, and one or two others. It is a completely stateless, line-oriented
> protocol, compared to IMAP being a stateful protocol, with multiple,
> recursive structures.

Yes, you are right. HTTP is one of the simplest and most basic protocols
existing. And they in their wisdom have choosen exactly ONE whitespace,
which is a good thing.

And IMAP is a very powerful protocol which achieves very much. So if it
is too complex for you to understand then start writing a webserver (which
will make us all happy because it keeps you away from IMAP) and tell them
who defined HTTP to allow more than one whitespace and change their
standard and all existing software violating your ideas!

>
> You can continue to embarass yourself, as long as you would like.

It's funny to see someone who even has written his own IMAP server to
be so agressive about it and embarassing himself quarreling with someone
like me.

> I only with IMAP was as simple as HTTP.

As everybody already knows, you really might want to define your own
standard!


Frank Klauert

unread,
May 8, 2002, 12:02:01 PM5/8/02
to
"Sam" <s...@email-scan.com> wrote:
> I've never seen this file before. Rather than reproduce the monstrosity
> that BODYSTRUCTURE spits out, I'll just reproduce what Pine shows after
> being fed Courier-IMAP's output:

Congratulations!

>
> When I fed this to dkimap4, it segfaulted in strcpy().

It hardly uses strcpy ;-)

But for me it also crashed!
It's good to know about this problem with DKIMAP.

Thanks for testing this with linux!

>
> Maybe I didn't install it right, but my install of dkimap ran as root even
> after authentication.

Mine not.
And it even did not run as "root" _before_ administration.
And it didn't run as "Administrator", neither.

And as far as I know, it does not have to run as root on linux.

I always create separate users with very restricted privileges
to the system and only the necessary access-rights to the
harddrive for every service I install.


But the only thing important is that bugs like this have to be
fixed soon!

> Unless I made an error installing the server, this
> means that anyone running dkimap4 on the Internet has a rootable box.

I think you didn't really install it way it's supposed.


Sam

unread,
May 8, 2002, 12:40:59 PM5/8/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <abbi33$gggre$2...@id-140432.news.dfncis.de>,
"Frank Klauert" <frank....@hotmail.de> writes:

> "Sam" <s...@email-scan.com> wrote:
>> >> Right. If you want to know if there's anything in the folder, wow, what a
>> >> brilliant idea: just open the folder and check. Don't assume that if the
>> >> folder exists, it must be non-empty.
>> >
>> > You do not approve of Pine's behavior. That's fine.
>> >
>> > However, disapproval is not equivalent to "bug".
>>
>> I guess I'm just a perfectionist: I consider an implementation flaw to be
>> the equivalent of a bug.
>
> You should make clear how you define the word "bug" and maybe
> replace it with more commonly used words like "design flaw".

A design flaw *is* a bug. The most serious kind, actually.

> Maybe it's time to update your client-"bug"-list NOW!

Some day I need to figure out exactly what about my bug list gets up so
much into some people's craw. I just don't understand. I, myself, have
read some completely off-the-wall published blatherings, on this topic, by
other parties. It doesn't seem to bother me much, it actually amuses me,
to a small degree. I even link to some of the best blatherings, from my
web sites.

I guess that some people just can't handle criticism.

>
>> >> I defy anyone to cite any text-based wire protocol that demands explicit
>> >> presense, or absence, of whitespace delimiters.
>> >
>> > What other protocols specify is also a red herring.
>>
>> Well, a wise man learns from history. Reading and understanding existing
>> protocols -- their faults and accomplishments -- is probably the best thing
>> that anyone can do if they plan to build something of their own. Learn
>> from others' mistakes, and all that.
>
> It believe Mark did exactly this.

You're still scheduled to produce a protocol, of comparable complexity,
that has a similar anal fixation on whitespace. You tried to offer HTTP as
an example, but quickly backtracked with the next post.

I'm still waiting.

> more than one whitespace makes much sense! There's no reason to say
> "You can randomly pick any amount of whitespace". Having exactly

Yes there is. Once you actually try to build something more complex than
Solitaire, you might actually figure out why.

>> > It is not "for no good reason". It is because RFC 2060 page 42 requires
>> > it.
>>
>> Something that's required simply for the sake of requiring it, can't be a
>> very good reason. Even a "good reason".
>
> Requiring from anybody to use more than one whitespace where exactly one
> whitespace is just fine is also "for no good reason".

Who said anything about _requiring_ multiple whitespacing? Look up what
'*' or '+' mean in the context of regular expressions, and get back to me.

> Maybe you can come up with really good reasons why it's so damn important for
> you to have more than one?

I already did: any communications protocol above a certain level of
complexity should have distinct lexical and grammatical properties. If
this is something that may be too advanced of a concept for you to grasp,
that's ok - nobody's perfect.

>> Gee, it's so easy to define a protocol, these days... It appears that
>> logical thought, and reason, are no longer required...
>>
>
> Sounds to me like an ideal situation for YOU to design and implement
> your own replacement for IMAP.
>
> Don't just complain about Mark, show us that you can do better.
> NOW!

I can, and I already did. You're just posturing because you do not have a
better argument to make.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE82VUM3ejdWUS0ltARAr9WAJ4r0rbgABldKkBQfg0tiPOVXWqjsACfbHpY
mHCAVWIbD4o12LDAmRsTRw4=
=RZ/n
-----END PGP SIGNATURE-----

Sam

unread,
May 8, 2002, 12:40:59 PM5/8/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <abbi39$gggre$4...@id-140432.news.dfncis.de>,
"Frank Klauert" <frank....@hotmail.de> writes:

>> > 9. Formal Syntax
>>
>> Please avoid putting your foot in your mouth again, at least until you
>> learn terms such as "deterministic finite automaton", "non-deterministic
>> finite automaton", "context-free grammar", "LALR(1)", and others.
>
> I know about these, believe it or not!

Ok, then, in that case WHAT exactly is your excuse for being so obtuse?

Go ahead, and translate that definition to context-free grammar. You might
succeed, but the results won't be pretty.


>> HTTP consists of less than a dozen simple commands: CONNECT, GET, PUT,
>> POST, and one or two others. It is a completely stateless, line-oriented
>> protocol, compared to IMAP being a stateful protocol, with multiple,
>> recursive structures.
>
> Yes, you are right. HTTP is one of the simplest and most basic protocols
> existing. And they in their wisdom have choosen exactly ONE whitespace,
> which is a good thing.

Simple protocols do not need anything more than that. However once you
exceed the point where formal lexical and grammatical structure is
necessary, bean-counting all your whitespace becomes stupid, pointless, and
silly.

> And IMAP is a very powerful protocol which achieves very much. So if it
> is too complex for you to understand then start writing a webserver (which

Would a web service do the trick?

>> You can continue to embarass yourself, as long as you would like.
>
> It's funny to see someone who even has written his own IMAP server to
> be so agressive about it and embarassing himself quarreling with someone
> like me.

I hear they're making a remake of Monty Python. You should audition for
the role of the Black Knight. You'd be a natural.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE82VUT3ejdWUS0ltARAkurAJ9RUO6j7h/HdXRB4WFEcEYfrRlp4QCcC2An
QVPZ+ngB4r4rS0ah5sYsYmk=
=aVI4
-----END PGP SIGNATURE-----

Mark Crispin

unread,
May 8, 2002, 1:28:07 PM5/8/02
to Frank Klauert
Frank -

Although I appreciate your support, I don't think that there is anything
productive in continuing to argue with the author of Courier.

The merits (or lack thereof) of his criticisms of the protocol and of how
the specification is written are not relevant. The only thing that is
relevant is whether or not Courier has been fixed to be compliant with the
IMAP specification.

Courier claims to be a product which implements the IMAP protocol. As
such, it is obliged to comply with the IMAP specification. It doesn't
matter if the IMAP specification is written in Linear A; or if it's the
worst designed protocol in the history of humanity.

The only thing that matters is compliance with IMAP as specified in the
published specifications. Courier was shown to be non-compliant, with
specific citations from the specification.

I do not know if Courier has been fixed to comply or not. There is no
definitive word saying that these problems have been fixed. There is
ongoing bad-mouthing of the protocol and its specification. Courier's
"bug list" continues to blame these problems on the client.

Lacking any better information, I must conclude that Courier is probably
still non-compliant. My advice is based upon that conclusion. It is
possible that a future version of Courier will comply; I hope so.

It is even possible that the current version of Courier complies. If that
is so, a simple statement to the effect that the problems are fixed would
have saved everyone a lot of time and trouble.

I've tuned out the bad-mouthing of the specification. It is a waste of
time to discuss it in newsgroups. The opinions which count are those of
the IESG, and of the people who comprised the IMAP and IMAP Extensions
Working Groups within the IETF.

IESG is the appropriate forum to present a case that BNF should not be
used to specify the syntax of a protocol, or that protocols should not
have rigid requirements in the use of delimiters. This is especially the
case now that IESG is considering whether to approve the revision to RFC
2060.

The IESG is also the proper forum to debunk that nonsense, should it
become necessary.

Sam

unread,
May 8, 2002, 3:49:23 PM5/8/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <Pine.LNX.4.50.020508...@shiva1.cac.washington.edu>,
Mark Crispin <m...@CAC.Washington.EDU> writes:

> The only thing that matters is compliance with IMAP as specified in the
> published specifications.

The only glitch with that rosy scenario is that the published
specifications are riddled with errors, ambiguities, and design flaws.
Which you forced yourself to admit in the revised document. You still
avoid addressing the fact that your own list of errors and revisions to RFC
2060 currently numbers almost a hundred, or that an alleged reference
implementation is forced to be tied out to an implementation-defined
behavior. Why's that?

Certainly it's more politically-correct to sweep that abortion under the
rug; but I still enjoy pointing fingers at it, every time someone starts
flapping their gums, on this topic. That mess is still there, and it's not
going to go away in a puff of smoke.

And that's pretty much all there is to it.

> I do not know if Courier has been fixed to comply or not. There is no
> definitive word saying that these problems have been fixed. There is

The definitive word is that nothing needs to be fixed.

> IESG is the appropriate forum to present a case that BNF should not be
> used to specify the syntax of a protocol, or that protocols should not

IESG is also a waste of time. Waiting for the heat death of the Universe
would be a more productive way to spend quality time.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE82YEz3ejdWUS0ltARAheuAJ9t1AOC0eGKATSQQ7PXDBGis2JIPwCfeeJo
3YKp8CLOHkHi4LFdae5mxmc=
=2shs
-----END PGP SIGNATURE-----

Frank Klauert

unread,
May 11, 2002, 3:38:00 PM5/11/02
to
> I've never seen this file before. Rather than reproduce the monstrosity
> that BODYSTRUCTURE spits out, I'll just reproduce what Pine shows after
> being fed Courier-IMAP's output:
>
> Subject: Multi-media mail demonstration
> Parts/Attachments:
> 1 Shown 3 lines Text, "Explanation"
[...]

> 12.3.4.1 56 KB Audio, "I'm Twying..."
>
> When I fed this to dkimap4, it segfaulted in strcpy().
>
> Maybe I didn't install it right, but my install of dkimap ran as root even
> after authentication. Unless I made an error installing the server, this
> means that anyone running dkimap4 on the Internet has a rootable box.
>

For the records: (Because there might be people who search newsgroup
for DKIMAP and would be frightened by this crash and I wouldn't like that)

The bug is fixed in the newest version 2.39.

It can be downloaded here:
http://www.dbox.handshake.de/download/

And DKIMAP can be run as not-root and it needs no access
to the homedirs of the users!


Sam

unread,
May 11, 2002, 8:22:49 PM5/11/02
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

In article <abjrra$j0mdb$1...@id-140432.news.dfncis.de>,
"Frank Klauert" <frank....@hotmail.de> writes:

> The bug is fixed in the newest version 2.39.

Did you check if it generates a correct BODYSTRUCTURE response?

> And DKIMAP can be run as not-root and it needs no access
> to the homedirs of the users!

Well, that's not what the installation instructions say:

[ ... ]

3. Add an appropriate line to your /etc/inetd.conf, such as

imap2 stream tcp nowait root /usr/bin/dkimap4 dkimap4

[ ... ]

> And DKIMAP can be run as not-root and it needs no access
> to the homedirs of the users!

Well, it seems to me that it does need to pull the mail out
/var/spool/mail, which it does - as you know - by running movemail. The
way things are set up on most system these days is that the individual mail
files are not globally readable/writable, even by the mail group. The mail
group only has write permissions on the /var/spool/mail directory itself,
to create mailboxes for new mail accounts; but the mail files themselves
will not be readable by group or world.

I suppose that you could get this to work by modifying the local mail
delivery agent to create mail files with read/write access to the mail
group, and run dkimap under the mail group.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt

iD8DBQE83bXU3ejdWUS0ltARAn6aAJ96D4gQSVOxkJbkzMufRCyXqSbRuwCglV/y
EZJ1N+Vml9j4qTxnvPi7Fpw=
=vMiB
-----END PGP SIGNATURE-----

Chris Wiegand

unread,
May 12, 2002, 1:14:07 PM5/12/02
to
It's also nice for us sites that don't want to give our users accounts on
the linux box....

"Clifton Royston" <clif...@lava.net> wrote in message
news:ab9nt5$cok$1...@mochi.lava.net...


---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.351 / Virus Database: 197 - Release Date: 4/19/2002


Reply all
Reply to author
Forward
0 new messages