Mulberry gone, now what?
Yiorgos Adamopoulos
bankrupt. So now what is left to be a decent Win32 IMAP GUI client?
--
#include <std/disclaimer.h> /* Yiorgos Adamopoulos */
David Magda
> As you may have seen from http://www.mulberrymail.com/ ISAMET has
> gone bankrupt. So now what is left to be a decent Win32 IMAP GUI
> client?
Thunderbird?
--
David Magda <dmagda at ee.ryerson.ca>
Because the innovator has for enemies all those who have done well under
the old conditions, and lukewarm defenders in those who may do well
under the new. -- Niccolo Machiavelli, _The Prince_, Chapter VI
NM Public
> As you may have seen from http://www.mulberrymail.com/ ISAMET has gone
> bankrupt. So now what is left to be a decent Win32 IMAP GUI client?
For myself, personally, I'm going to continue to use these IMAP
clients:
Mulberry
Pine
Horde/IMP
There's no reason to stop using Mulberry at this point. By the
time Mulberry is truly dead, I bet there will be a good *fast*
powerful browser-based IMAP client. It seems like everyone and
his dog is working on such a thing right now.
Desktop messaging clients are dead!
Nancy
--
Nancy McGough ~ <http://www.ii.com> ~ <http://deflexion.com>
IMAP, pine, procmail, data deflexion, infinity, and more
> > > Please keep the discussion in the group < < <
NM Public
> For myself, personally, I'm going to continue to use these IMAP clients:
>
> Mulberry
> Pine
> Horde/IMP
>
> There's no reason to stop using Mulberry at this point. By the time Mulberry
> is truly dead, I bet there will be a good *fast* powerful browser-based IMAP
> client. It seems like everyone and his dog is working on such a thing right
> now.
>
> Desktop messaging clients are dead!
Here are a couple more of my thoughts about IMAP clients that I
forgot to mention.
* Since you explicitly mentioned Win32, you might want to try
PC-Pine. It's not very GUI, but it supports some GUI things like
being able to right click on a message to change its status. It
supports a lot of things that no other IMAP client supports, for
example you can use 30+ IMAP keywords to help you organize your
messages. (The number of keywords depends on the IMAP server,
e.g., I think Cyrus IMAP can support 128 (or 256?) keywords.)
* The current Opera (8) has pretty flaky IMAP support, but they
are completely rewriting the IMAP part of their
mail/nntp/rss/atom/irc/messaging client. I (and many others) are
hoping that the upcoming Opera 9 will be a decent IMAP client. As
of Sept 20, Opera is ad-free and registration-free.
I'm curious what other people think about current and future IMAP
clients.
Thanks,
Stephen Chadfield
> On 3 Oct 2005, NM Public wrote:
>> For myself, personally, I'm going to continue to use these IMAP clients:
>>
>> Mulberry
>> Pine
>> Horde/IMP
>>
>> There's no reason to stop using Mulberry at this point. By the time
>> Mulberry is truly dead, I bet there will be a good *fast* powerful
>> browser-based IMAP client. It seems like everyone and his dog is working on
>> such a thing right now.
I use Solaris at work and was rather dissapointed with the way Mulberry
would lock up when left running for a while. I always hoped these problems
would get fixed and was keeping my eyes open for a non-beta release of
version 4 for Unix/Linux but I guess now that will never happen. There is
no GUI IMAP client out there which has the same geek appeal that Mulberry
had.
>> Desktop messaging clients are dead!
I do tend to use the FastMail web interface quite a bit for my home email
accounts.
If the last, stable Unix/Linux release had been a really solid GTK2 based
app that blended in with the GNOME desktop I may have continued using
Mulberry for a while. As it stands I think it is time to move on.
> * The current Opera (8) has pretty flaky IMAP support, but they are
> completely rewriting the IMAP part of their mail/nntp/rss/atom/irc/messaging
> client. I (and many others) are hoping that the upcoming Opera 9 will be a
> decent IMAP client. As of Sept 20, Opera is ad-free and registration-free.
My experience with Mulberry shows the perils of adopting closed-source
programs. Given that Opera has ceased charging for their product it seems
more likely than ever that they might disappear at any time. I have tried
their current client and thought it was truly awful.
At the moment I using mostly Pine supplemented by webmail and Evolution.
I would like a GTK2 based program that implements the best elements of
Mulberry in a rock-solid fashion. As I am too lazy to code this I
shouldn't be too upset that no-one else wants to either ;-)
--
Stephen Chadfield
http://www.chadfield.com/
Mark Crispin
> My experience with Mulberry shows the perils of adopting closed-source
> programs.
I have a mixed-view of that issue. I have a decades-long preference for
open source, from long before the time that Richard Stallman started the
GNU church.
On the other hand, if a closed-source program is truly compelling, I
wouldn't refuse to use it just because it is closed-source.
The big problem of the 1980s was of non-portable software, and hardware
vendors who locked you into their hardware through their software. For
all our criticism of Microsoft, the evil empire did us all a invaluable
service by breaking that lock forever. We still see the last dying
remants of hardware-locking -- most notoriously Apple -- but those vendors
are fighting a doomed battle and we won't mourn their ultimate defeat.
The problem today is that there is no "truly compelling" GUI IMAP client
today, either closed or open source. Pine is truly compelling, but it is
not GUI.
I think that this establishes that it is difficult to write a "truly
compelling" GUI IMAP client.
I'd like to think that the open source community will eventually produce
one; but the scale of development costs suggest to me that it isn't going
to happen. I also don't see much interest or motivation. Sadly, the open
source community does very little that's new; instead, it reimplements
existing software that is either not open source or insufficiently
compliant with the church's liturgy for open source.
Closed source developers have a different set of problems. Small vendors,
such as the late ISAMET, don't have much of a chance of recovering
development costs of email clients, much less making a profit. Large
vendors, such as Microsoft, can do email clients as loss-leaders but such
programs have to support other goals of the company (Outlook is is not an
IMAP client; it is an Exchange application that does IMAP too).
I don't think that academia will do it either. Academia operates on a
pendulum, and the current swing is heading in the direction of the late
1970s when academia abandoned software development like villagers fleeing
from a tsunami. There are a few holdouts (Pine is one) but in general I
fear that it will be several years before we once again will see academia
producing much software for external consumption.
I would be very happy to have this gloomy view proven wrong, but I don't
see much sign of it in the near term.
-- Mark --
http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
NM Public
>
>>> Desktop messaging clients are dead!
>
> I do tend to use the FastMail web interface quite a bit for my home email
> accounts.
I read a lot of discussion groups/blogs about messaging and the
majority of people who think about these things are moving to
browser-based messaging tools. Unfortunately, a lot of them are
moving to Gmail, which at this point is not an especially open
system.
>> * The current Opera (8) has pretty flaky IMAP support, but they are
>> completely rewriting the IMAP part of their
>> mail/nntp/rss/atom/irc/messaging client. I (and many others) are hoping
>> that the upcoming Opera 9 will be a decent IMAP client. As of Sept 20,
>> Opera is ad-free and registration-free.
>
> My experience with Mulberry shows the perils of adopting closed-source
> programs. Given that Opera has ceased charging for their product it seems
> more likely than ever that they might disappear at any time. I have tried
> their current client and thought it was truly awful.
I agree that the current Opera IMAP client is awful. The Opera
developers seem to agree with that too. It, like almost all
messaging clients other than Pine and Mulberry, was not designed
*from the beginning* with IMAP in mind.
But, there is nothing inherently perilous about a closed-source
program. As long as it uses open-standard protocols and
open-standard data formats, users aren't locked in and can easily
move to another program that uses those standards. For example,
it should not be too hard for Mulberry users to switch to another
IMAP client. I use multiple IMAP clients every day and do not
feel locked in at all to any operating system or messaging
system.
Also now, thanks to del.icio.us, I use multiple web browsers on
multiple platforms every day and my bookmarks are available
everywhere.
My goal when choosing any software -- messaging, browser, or
whatever -- is that I am not locked in. If Opera goes under, it
doesn't matter because my del.icio.us bookmarks are available
everywhere. If Mulberry or Pine or Horde/IMP go under, it doesn't
matter because my IMAP messages are available everywhere. The key
is to support the standards and the people and companies who
produce standard-based software. That is what I have been trying
to do for years at Infinit Ink, e.g., my IMAP service providers
page.
Mark Crispin
> I read a lot of discussion groups/blogs about messaging and the majority of
> people who think about these things are moving to browser-based messaging
> tools.
There is no doubt that browser-based messaging tools have a place; but I
see these as complementary to IMAP-based tools rather than as a
substitution. There's a lot good to be said about browser-based tools,
not the least being the rapid prototyping of the UI and not needing to
install software on each client.
However, I believe that there is a long term future for IMAP-based
messaging tools, especially for heavier-duty tasks. I've observed that
I'm not alone in using both browser-based and IMAP-based messaging tools;
this indicates that the two types of tools must interact and interoperate
seamlessly.
NM Public
> On Tue, 4 Oct 2005, NM Public wrote:
>> I read a lot of discussion groups/blogs about messaging and
>> the majority of people who think about these things are moving
>> to browser-based messaging tools.
>
> There is no doubt that browser-based messaging tools have a
> place; but I see these as complementary to IMAP-based tools
> rather than as a substitution. There's a lot good to be said
> about browser-based tools, not the least being the rapid
> prototyping of the UI and not needing to install software on
> each client.
>
> However, I believe that there is a long term future for
> IMAP-based messaging tools, especially for heavier-duty tasks.
> I've observed that I'm not alone in using both browser-based
> and IMAP-based messaging tools; this indicates that the two
> types of tools must interact and interoperate seamlessly.
Mark, from what you wrote above, it seems that your view of
messaging clients looks like this:
+---------------+ +---------------+
| | | |
| IMAP | | browser-based |
| clients | | clients |
| | | |
+---------------+ +---------------+
My view is this:
+---------------+
| |
| IMAP +--+---------------+
| clients | | |
| | | browser-based |
+------------+--+ clients |
| |
+------------------+
And all I'm saying is that many people seem to be moving to using
the browser as their platform. This *includes* heavy-duty IMAP
tools that run in a browser. I wasn't implying that IMAP has a
short-term future, although that might be the case given this
movement to the browser-as-platform. Possible replacements for
IMAP include DAV, SOAP, Atom, or something else. I don't have a
strong opinion about what are the best messaging protocols for
this browser-based future. The main thing I care about is that
there are standards and that data, including metadata, never gets
locked into any system.
Whatever happens, it is certainly interesting to be watching
right now!
Jeffrey Goldberg
> Mark, from what you wrote above, it seems that your view of
> messaging clients looks like this:
>
> +---------------+ +---------------+
> | | | |
> | IMAP | | browser-based |
> | clients | | clients |
> | | | |
> +---------------+ +---------------+
>
>
> My view is this:
>
>
> +---------------+
> | |
> | IMAP +--+---------------+
> | clients | | |
> | | | browser-based |
> +------------+--+ clients |
> | |
> +------------------+
>
>
> And all I'm saying is that many people seem to be moving to using
> the browser as their platform.
I think, but could be wrong, that you NM and MRC are using the word
"browser" in two different ways. I think that MRC is using the word
"browser" as "doing the job through HTTP(S)", while NM may be talking
about a client which really talks IMAP as well as HTTP(S).
I'm sure that everyone here knows that the main advantages of using HTTP
for mail over using POP are things that IMAP does better. As far as I
can tell the *only* thing that gives the HTTP systems an advantage (and
it appears to be a whopping advantage) is the familiar user interface.
In the early days of IMAP, Pine and Simeon where the choices. Pine
wasn't GUI, and Simeon had stability problems. Then, probably the worst
thing to happen to IMAP, was the introduction of clients, Netscape,
Outhouse, etc which did what I can only call "POP over IMAP". The
extent to which the community "didn't get it" wrt to IMAP surprised me
at the time. Even now, I've had a friend complain to me that her
university is switching from something that does IMAP just fine (and
where she happily uses Pine) to a webmail system, that really doesn't do
IMAP.
On the whole, I think that MRC is right. Universities are no longer the
driving force for innovation in software. Commercial attempts to create
really good IMAP clients have not gone well. And most of the Open
Source stuff seems to stop at "good enough" when it comes to mail
clients. (Though unlike MRC, I don't see that as a systemic problem
with open source, but it does seem to be true in this case.)
-j
Mark Crispin
> I think, but could be wrong, that you NM and MRC are using the word "browser"
> in two different ways. I think that MRC is using the word "browser" as
> "doing the job through HTTP(S)", while NM may be talking about a client which
> really talks IMAP as well as HTTP(S).
Interesting analysis, and that would certainly explain why Nancy and I are
talking past each other. I agree with you as to what I'm talking about;
but I won't presume to speak for Nancy.
> As far as I can tell
> the *only* thing that gives the HTTP systems an advantage (and it appears to
> be a whopping advantage) is the familiar user interface.
I think that this understates the advantages of HTTP.
HTTP greatly simplifies the software distribution problem. You can
literally walk up to an Internet station anywhere in the world and access
your mail with your preferred interface.
HTTP also revives the dream of a "thin client" station which doesn't have
any smarts and instead is controlled totally by the server. This has been
a holy grail for portions of the industry for many years.
The other side of the coin is pushback against the "thin client" concept;
thin clients have never been particularly successful.
There is also the question of control; the use of HTTP-based MUAs requires
users to surrender almost complete control over their data to the server
administrator. This was an issue in the POP vs. IMAP debate, but HTTP
goes far further in this direction.
> On the whole, I think that MRC is right. Universities are no longer the
> driving force for innovation in software.
Fortunately, this seems to be cyclic. I recall a several-year period,
from the late 1970s until the mid 1980s, that was quite dry. I think that
academia is midway through the current dry spell.
I only wish that I could accurately predict what will be The Next Really
Big Thing; if I could, I'd be setting myself up to retire quite wealthy.
I have some guesses, but not good enough to bet the farm.
> Commercial attempts to create
> really good IMAP clients have not gone well.
I wish that I could understand why this was so. I know, first-hand, that
there are numerous individuals in the commerical world who have an
excellent understanding of IMAP.
Then again, a similar problem seems to exist within the field of
calendaring. Perhaps the explanation is as simple as "these are very
difficult problems."
> And most of the Open Source
> stuff seems to stop at "good enough" when it comes to mail clients. (Though
> unlike MRC, I don't see that as a systemic problem with open source, but it
> does seem to be true in this case.)
If I implied that this was a systemic problem with open source, I want to
retract that. Or at least rephase it.
I believe that there *is*, at the present time, a systemic problem with a
lack of innovation within the current open source community. That was not
the case in the past. I am confident that it will not be the case in the
future. But, right now, a lot of time is spent doing (and redoing) stuff
that has already been repeatedly done in the past 35 years.
It sticks in my craw that there is some truth to Microsoft's claim that
they are innovating; and, by implication, that the rest of us are not.
I'd like to see that change.
Rather than hypothesizing as to the cause of the problem (which would
undoubtably unleash a major flamewar), I'm more interested in being part
of the solution.
-- Mark --
http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
kael
> As you may have seen from http://www.mulberrymail.com/ ISAMET has gone
> bankrupt. So now what is left to be a decent Win32 IMAP GUI client?
Polymer, currently in development, is a cross-plateform ACAP/IMAP GUI
client - <http://dave.cridland.net/acap/polymer.html> ; it's probably
the third real IMAP client, after Pine and Mulberry (okay, I'm missing
Cone and Mutt).
The features I like :
- very fast, specially with prefetching;
- IMAP keywords displayed in a non-exclusive manner under header fields.
Emails can be tagged thanks to the SIEVE-IMAP-flags extension or
manually, directly with Polymer;
- IMAP URLs ; e.g. : <imap://;AUTH=*@example.org/INBOX?KEYWORD%20foo>;
- MANAGESIEVE;
- filtered views, with the ability to customise colors and save all
those settings on the ACAP server;
- ACAP bookmarks.
It also supports other IMAP extensions which are not yet implemented on
servers like the CATENATE extension.
Polymer is very promising, IMHO.
--
kael
use...@isbd.co.uk
> Yiorgos Adamopoulos wrote:
> > As you may have seen from http://www.mulberrymail.com/ ISAMET has gone
> > bankrupt. So now what is left to be a decent Win32 IMAP GUI client?
>
> Polymer, currently in development, is a cross-plateform ACAP/IMAP GUI
> client - <http://dave.cridland.net/acap/polymer.html> ; it's probably
> the third real IMAP client, after Pine and Mulberry (okay, I'm missing
> Cone and Mutt).
>
client out there - it works best with a limited range of IMAP servers.
Personally I think this is the fundamental flaw in IMAP, it's not
tightly enough specified so IMAP servers vary quite a lot with the
result that it's well nigh impossible to write an IMAP client that
works well with them all.
I use mutt and, while it is a reasonably functional IMAP client, I
find that it is so much pleasanter to use with local mail files I just
about never use mutt with IMAP.
I get similar capability to IMAP by running mutt remotely (via ssh) so
I can see and manage the same mail folders etc. wherever I am working
from.
--
Chris Green
Mark Crispin
> ... and suffers from the same problem as virtually every other IMAP
> client out there - it works best with a limited range of IMAP servers.
"Works best with a limited range of IMAP servers", or "works best with
standards-compliant IMAP servers"?
> Personally I think this is the fundamental flaw in IMAP, it's not
> tightly enough specified so IMAP servers vary quite a lot with the
> result that it's well nigh impossible to write an IMAP client that
> works well with them all.
Can you point to anything in the IMAP specification that illustrates this
point?
I ask this, because the interoperability problems that I have seen have
always been due to an implementation that failed to comply with an
explicitly stated requirement in the IMAP specification. One of the most
common blunders is to implement one's own idea of syntax rather than the
syntax described in the Formal Syntax.
use...@isbd.co.uk
> On Fri, 7 Oct 2005, use...@isbd.co.uk wrote:
> > ... and suffers from the same problem as virtually every other IMAP
> > client out there - it works best with a limited range of IMAP servers.
>
> "Works best with a limited range of IMAP servers", or "works best with
> standards-compliant IMAP servers"?
>
> > Personally I think this is the fundamental flaw in IMAP, it's not
> > tightly enough specified so IMAP servers vary quite a lot with the
> > result that it's well nigh impossible to write an IMAP client that
> > works well with them all.
>
> Can you point to anything in the IMAP specification that illustrates this
> point?
>
says:-
IMAP allows a great deal of latitude in much of the specification. As
a result, some behaviour is rare, and relatively untested. As always
with IMAP, you should select a server and client which interoperate
well, supporting the features you require.
It then goes on to list IMAP servers that the client will work with,
with varying levels of performance.
Many/most people won't have a choice of IMAP servers.
My exeperience using mutt with different IMAP servers is that it never
works quite the same with different servers.
--
Chris Green
Mark Crispin
> See http://dave.cridland.net/acap/polymer.html for example where it
> says:-
> IMAP allows a great deal of latitude in much of the specification. As
> a result, some behaviour is rare, and relatively untested. As always
> with IMAP, you should select a server and client which interoperate
> well, supporting the features you require.
That isn't the same as what you said ("not tightly enough specified").
I am not certain; but I suspect that Dave is referring to the fact that
IMAP permits the server to export whatever sort of namespace and naming
conventions for the mail store that it wants.
That is, rather than take a position between the various forms of naming
and hierarchy conventions that were used for contemporary mail stores,
IMAP declares that they're all good, and provides a means for the client
to determine what sort of naming and hierarchy conventions that the server
uses for its mail store.
Although this is considered to be a mistake today (and that IMAP should
have used URI style naming and hierarchy as the Only Way), the fact is
that URI style wasn't even on the table back then. The contemporary
contenders were netnews and UNIX flat-files. There was a nod to the
Windows filesystem (a half-hearted attempt to support its hierarchy), and
lip service to legacy filesystems such as VMS and TOPS-20 (no attempt was
made to support their hierarchy).
HTTP definitely got hierarchy right; and FTP and IMAP did not.
On the other hand, HTTP did not have to deal with the export of a legacy
store; nor did it have to deal with the expectations of users who have
accessed these stores from the shell and thus know that system's naming
conventions.
It's easy to look at such issues in hindsight. The problem with doing so
is that it assumes that one single thing in the past can be done
differently without impact to anything else.
In almost every other respect, IMAP requires one, and only one, way of
doing things. There is minimal latitude in server behavior, and almost
none in syntax.
> It then goes on to list IMAP servers that the client will work with,
> with varying levels of performance.
I suspect, but am not certain, that these are issues with the hierarchy
browser in his client. It is very difficult to write a general hierarchy
browser in an IMAP client. Many client authors implement against the
hierarchy rules offered by the mail store of their server, ignore those
funny flags discussed in the protocol, and then are shocked to discover
that other servers have completely different hierarchy.
Paul Rubin
> In almost every other respect, IMAP requires one, and only one, way of
> doing things. There is minimal latitude in server behavior, and
> almost none in syntax.
The syntax in the spec is so complicated it's very hard to implement
it correctly with a hand coded parser. I gave up about a tenth of the
way through and switched to an ad hoc mechanically generated one which
had drawbacks of its own. Many other implementations get it wrong in
a few places, like putting an extra space character here or there.
I think you mentioned once that some future version might use XML
instead of the existing syntax. That will make things a lot easier.
Mark Crispin
> I think you mentioned once that some future version might use XML
> instead of the existing syntax. That will make things a lot easier.
This isn't outside of the realm of possibility. However, ABNF is pretty
much the standard for Internet protocol syntax rules.
There is an effort underway to restate the current ABNF rules in ways that
are easier to understand/use. The current ABNF, while accurately
describing the syntax, was ad-hoc and it shows. This effort will almost
certainly happen before the specification advanced to Draft Standard.
Jeffrey Goldberg
> Rather than hypothesizing as to the cause of the problem (which would
> undoubtably unleash a major flamewar), I'm more interested in being part
> of the solution.
I would like to be part of the solution too, but since I neither have
the programming talent to make a solid contribution IMAP clients, nor
the self-discipline to seriously run a business doing so, I am stuck in
the role of participating in flamewars. After all it's what I do best.
Now only if I can figure out a way to get paid for it.
Cheers,
-j
Mark Crispin
> The syntax in the spec is so complicated it's very hard to implement
> it correctly with a hand coded parser.
Unless you are both an "old fart" who does things by hand (and wishes we
were all still using assembly language), and a truly expert programmer, I
strongly advise against implementing IMAP (or for that matter any modern
protocol) with a hand coded parser.
It is extremely difficult to build a hand coded parser for non-trivial
protocols; and these days it is completely unnecessary due to the
excellent ABNF parser generators that are out there.
This isn't to criticize old farts who still write hand coded parsers; I'm
one myself. We've been there, and done that, and gotten the t-shirt. It
just doesn't need to be done any more.
use...@isbd.co.uk
> On Fri, 7 Oct 2005, use...@isbd.co.uk wrote:
> > See http://dave.cridland.net/acap/polymer.html for example where it
> > says:-
> > IMAP allows a great deal of latitude in much of the specification. As
> > a result, some behaviour is rare, and relatively untested. As always
> > with IMAP, you should select a server and client which interoperate
> > well, supporting the features you require.
>
> That isn't the same as what you said ("not tightly enough specified").
>
latitude" sound pretty similar to me. OK they are looking from
different directions but I still think they are saying much the same
thing.
> I am not certain; but I suspect that Dave is referring to the fact that
> IMAP permits the server to export whatever sort of namespace and naming
> conventions for the mail store that it wants.
>
> That is, rather than take a position between the various forms of naming
> and hierarchy conventions that were used for contemporary mail stores,
> IMAP declares that they're all good, and provides a means for the client
> to determine what sort of naming and hierarchy conventions that the server
> uses for its mail store.
>
But the problem is that the client can't "determine what sort of
naming and hierarchy conventions ....", the server dictates it in
general.
> > with varying levels of performance.
>
> I suspect, but am not certain, that these are issues with the hierarchy
> browser in his client. It is very difficult to write a general hierarchy
> browser in an IMAP client. Many client authors implement against the
> hierarchy rules offered by the mail store of their server, ignore those
> funny flags discussed in the protocol, and then are shocked to discover
> that other servers have completely different hierarchy.
>
Quite, and that's exactly one of my major problems with IMAP.
--
Chris Green
Mark Crispin
> But the problem is that the client can't "determine what sort of
> naming and hierarchy conventions ....", the server dictates it in
> general.
I'm not sure which meaning of "determine" you intend here.
If by "determine" you mean "decide", then that is correct; in IMAP the
server (and by extension the server management) decides the naming and
hierarchy conventions, not the client.
Over a decade ago, this was considered to be a feature, not a bug. The
idea here was that all clients of the server would be made to behave in
the same way; and thus the server provider's helpdesk doesn't need to know
how a particular client works.
This conflicts with the desire of client authors to have all servers
accessed by the client behave in the same way; and thus the client
vendor's helpdesk doesn't need to know how a particular server works.
The sad result was that both sides refused to compromise. At least one
client author told me that he did not care what the server exported; his
policy was that his client defined the one true way of hierarchy and
naming, regardless of server policy.
The lesson is that it is a mistake to allow either server or client to
decide naming and hierarchy policy; the protocol must decide. But it
wasn't until the widespread deployment of URIs that this was generally
recognized (not to mention any sort of concensus of what the one true
naming and hierarchy should look like!).
If by "determine" you mean "figure out the server's naming and hierarchy
conventions", IMAP provides the means to do this. The problem is that
many clients don't use these mechanisms and/or ignore what these
mechanisms tell them.
None of this, by the way, is an attribute of IMAP; naming and hierarchy
was glommed onto IMAP after the fact. IMAP2 had none of this nonsense;
and I opposed the addition of the mailbox management commands to IMAP. I
always felt that a separate protocol should manage mailboxes, leaving IMAP
to focus on its original design of managing messages within a mailbox.
Dave Cridland
> Mark Crispin <m...@cac.washington.edu> wrote:
> > On Fri, 7 Oct 2005, use...@isbd.co.uk wrote:
> > > ... and suffers from the same problem as virtually every other IMAP
> > > client out there - it works best with a limited range of IMAP servers.
> >
> > "Works best with a limited range of IMAP servers", or "works best with
> > standards-compliant IMAP servers"?
> >
Actually, it works best with a server that hasn't yet been written.
Polymer supports a lot of extensions, and no server exists with the
full combination of extensions, yet.
Polymer should work with any IMAP4rev1 server. It might work with a
plain IMAP4 server, but I'd treat any errors there as low priority.
> > > Personally I think this is the fundamental flaw in IMAP, it's not
> > > tightly enough specified so IMAP servers vary quite a lot with the
> > > result that it's well nigh impossible to write an IMAP client that
> > > works well with them all.
> >
> > Can you point to anything in the IMAP specification that illustrates this
> > point?
> >
> See http://dave.cridland.net/acap/polymer.html for example where it
> says:-
>
Cool. Mark, see that? My webpage is a more important reference than the
specification. ;-)
> IMAP allows a great deal of latitude in much of the specification. As
> a result, some behaviour is rare, and relatively untested. As always
> with IMAP, you should select a server and client which interoperate
> well, supporting the features you require.
>
Latitude is definitely not intended to mean the specification is in any
way problematic. IMAP is very highly specified. It does, however,
provide various protocol features which allow server to behave
reasonably differently within some areas, in order to support very
different server-side mail spools. That's not a fault in the protocol,
that's a benefit. (One that has, in many respects, outlived itself). In
addition, there's exciting choices like multiple commands in progress,
unsolicited reponses at any time, etc.
These are minor little variations in the gran scheme of things, though.
The vast bulk of IMAP is very tightly specified - in some cases so
tightly specified that virtually nobody implements it correctly
(witness the \Recent flag).
> It then goes on to list IMAP servers that the client will work with,
> with varying levels of performance.
>
I think you're reading too much into that listing. :-)
> Many/most people won't have a choice of IMAP servers.
>
True, and Polymer will work with any standards compliant IMAP4rev1
server, or at least should do. If it doesn't, it's a bug in Polymer, no
question about it.
>
> My exeperience using mutt with different IMAP servers is that it never
> works quite the same with different servers.
Mutt is pretty basic in its IMAP support, so yes, server behaviour -
such as hierarchy - is directly visible to the user.
Polymer is not simple, nor basic, and attempts to make all servers
behave identically. It uses extensions such as NAMESPACE to do this,
emulates others where possible and prudent, and so on. The difference
visible comes down to performance, usually, either speed or bandwidth
consumption.
Polymer supports 21 IMAP extensions. (That's counting STARTTLS as an
extension, but not other capability strings mentioned in RFC3501, so
AUTH, LOGIN-DISABLED, etc).
Different servers support different subsets of these (and Polymer
doesn't yet support every IMAP extension, of course), so if you are in
a position to select either your server or your client - and many
people are in a position to do that - then my advice would always be to
consider which extensions and features you require, and make sure both
support them.
Moreover, I can't test every permutation and combination of extensions.
I can't test every possible combination of optional behaviour in
RFC3501. Like, for instance, servers sending FETCH or EXISTS responses
when no command is in progress, even if they don't support IDLE. It's
legal, but nobody does it. Hardly any clients can cope, either, but
Polymer can, just as it can cope with commands executing out of order.
And different support on the server means that potentially different
features become available, (unlimited keywords? ANNOTATE?). Polymer
can't easily emulate message metadata (I could map it onto ACAP, but
it's rather messy), so some features are dependent on server behaviour.
I happen to have opted to list the servers I happen to have tested
relatively recently and reasonably extensively, and can continue to
test on a frequent basis, in an order which makes sense for my
technical requirements.
Note that the last entry is "Other Servers" - I do know that Polymer
works perfectly well with UW-IMAP, for instance, because UW-IMAP
conforms to RFC3501 etc. I've not put UW-IMAP on that list, because it
happens that I don't have an account on such a server.
I hope that clears up any misunderstanding.
Dave.
use...@isbd.co.uk
>
> If by "determine" you mean "figure out the server's naming and hierarchy
> conventions", IMAP provides the means to do this. The problem is that
> many clients don't use these mechanisms and/or ignore what these
> mechanisms tell them.
>
decides how the hierarchy should look. Either that or, as you
suggest, it should always be the same and not be left up to the IMAP
implementation.
--
Chris Green
use...@isbd.co.uk
> > > > Personally I think this is the fundamental flaw in IMAP, it's not
> > > > tightly enough specified so IMAP servers vary quite a lot with the
> > > > result that it's well nigh impossible to write an IMAP client that
> > > > works well with them all.
> > >
> > > Can you point to anything in the IMAP specification that illustrates this
> > > point?
> > >
> > See http://dave.cridland.net/acap/polymer.html for example where it
> > says:-
> >
>
> Cool. Mark, see that? My webpage is a more important reference than the
> specification. ;-)
>
However I think you will find quite a few other IMAP clients have a
similar attitude and therein lies much of IMAP's problem.
> > IMAP allows a great deal of latitude in much of the specification. As
> > a result, some behaviour is rare, and relatively untested. As always
> > with IMAP, you should select a server and client which interoperate
> > well, supporting the features you require.
> >
>
> Latitude is definitely not intended to mean the specification is in any
> way problematic. IMAP is very highly specified. It does, however,
> provide various protocol features which allow server to behave
> reasonably differently within some areas, in order to support very
> different server-side mail spools. That's not a fault in the protocol,
> that's a benefit. (One that has, in many respects, outlived itself). In
> addition, there's exciting choices like multiple commands in progress,
> unsolicited reponses at any time, etc.
>
... but it *is* a fault in the protocol from the user's point of
view. I want my IMAP client to behave the same whatever server I'm
using, that's what makes things easy for the user.
I see you say that you try and make your client seem the same to the
user regardless of the IMAP server. That's an admirable aim but it
feels you are trying to do it in spite of IMAP rather than as a result
of using IMAP.
--
Chris Green
Mark Crispin
> decides how the hierarchy should look. Either that or, as you
> suggest, it should always be the same and not be left up to the IMAP
> implementation.
That opinion is not (at least was not 12 years ago) universally held. A
very different opinion ("all clients must behave the same when using our
site") held sway when the decision was made. Whether or not we, with the
benefit of hindsight, agree with that decision today is beside the point.
I felt then, and still do now, that IMAP should not be in the mailbox
management business at all. IMAP is a "message access" protocol.
Mailbox management is orthogonal to that purpose. Had the CREATE, DELETE,
RENAME, and LIST/LSUB commands been in a separate protocol as I wished, we
would have had the option of changing that protocol (as needs and requires
change over time) without impacting IMAP.
Nevertheless, it was decided not to do that, for reasons that seemed
compelling to an overwhelming majority.
As it turned out, CREATE and DELETE turned out to be relatively innocuous.
RENAME was a disaster; *no* server implements RENAME correctly.
Fortunately, few clients depend upon RENAME (much less RENAME behaving
correctly).
LIST/LSUB was a worse disaster. This is, after all, what this thread is
discussing. Nothing else in IMAP has anything to do with hierarchy or
naming semantics. Now we have LISTEXT. I wonder what new tarpit LIST
will venture into in another 10 years.
If there ever is a movement to deprecate the CREATE, DELETE, RENAME, LIST,
and LSUB commands (maybe STATUS as well), I'll join in a flash. I won't
hold my breath.
Dave Cridland
use...@isbd.co.uk wrote:
> Dave Cridland <da...@cridland.net> wrote:
> > > > > Personally I think this is the fundamental flaw in IMAP, it's not
> > > > > tightly enough specified so IMAP servers vary quite a lot with the
> > > > > result that it's well nigh impossible to write an IMAP client that
> > > > > works well with them all.
> > > >
> > > > Can you point to anything in the IMAP specification that illustrates this
> > > > point?
> > > >
> > > See http://dave.cridland.net/acap/polymer.html for example where it
> > > says:-
> > >
> >
> > Cool. Mark, see that? My webpage is a more important reference than the
> > specification. ;-)
> >
> That wasn't *really* what I was suggesting! :-)
>
I didn't think so, really.
> However I think you will find quite a few other IMAP clients have a
> similar attitude and therein lies much of IMAP's problem.
>
A similar attitude to what? All I say in the paragraph quoted below is
that I haven't tested every combination of every extension or optional
behaviour because there are lots of them, and that you should always
pick a server and client which, combined, do what you want.
I don't really see how that could be construed as thinking that IMAP is
badly specified.
> > > IMAP allows a great deal of latitude in much of the specification. As
> > > a result, some behaviour is rare, and relatively untested. As always
> > > with IMAP, you should select a server and client which interoperate
> > > well, supporting the features you require.
> > >
> >
> > Latitude is definitely not intended to mean the specification is in any
> > way problematic. IMAP is very highly specified. It does, however,
> > provide various protocol features which allow server to behave
> > reasonably differently within some areas, in order to support very
> > different server-side mail spools. That's not a fault in the protocol,
> > that's a benefit. (One that has, in many respects, outlived itself). In
> > addition, there's exciting choices like multiple commands in progress,
> > unsolicited reponses at any time, etc.
> >
> ... but it *is* a fault in the protocol from the user's point of
> view. I want my IMAP client to behave the same whatever server I'm
> using, that's what makes things easy for the user.
>
Ah, but as Mark pointed out, the original aim was to make the mail
system's layout appear the same no matter what client you were using...
That, too, makes things easy for people.
NAMESPACE changes this to effectively publish the server's hierarchy
arrangement, allowing a client author to present a consistent hierarchy
style between servers, should they choose.
> I see you say that you try and make your client seem the same to the
> user regardless of the IMAP server. That's an admirable aim but it
> feels you are trying to do it in spite of IMAP rather than as a result
> of using IMAP.
No, I'm just using the base specification plus one extension
(NAMESPACE) for that.
It was very easy to do. There are much more complex bits of Polymer,
such as sending a message efficiently, for instance. The hierarchy
normalization only took about two days, perhaps three. Message assembly
and transmission took several weeks.
The ACAP server took several months, and that really did feel like I
was doing it in spite of the state of ACAP, and certainly not because
of it.
Dave.
Mark Crispin
> Polymer should work with any IMAP4rev1 server. It might work with a
> plain IMAP4 server, but I'd treat any errors there as low priority.
I don't think that it makes much sense to work with IMAP4 (RFC 1730).
IMAP4 was a false start. If you must fall back from IMAP4rev1 (RFC
2060/3501), it would be better to fall back to IMAP2 (RFC 1064/1176).
>> See http://dave.cridland.net/acap/polymer.html for example where it
>> says:-
> Cool. Mark, see that? My webpage is a more important reference than the
> specification. ;-)
:-)
> Latitude is definitely not intended to mean the specification is in any
> way problematic. IMAP is very highly specified. It does, however,
> provide various protocol features which allow server to behave
> reasonably differently within some areas, in order to support very
> different server-side mail spools. That's not a fault in the protocol,
> that's a benefit. (One that has, in many respects, outlived itself).
Agreed.
> In
> addition, there's exciting choices like multiple commands in progress,
> unsolicited reponses at any time, etc.
I never considered these to be particularly important, but there was a
small but very loud contigent (now long gone) that screamed for multiple
commands in progress.
The original IMAP had commands (with no tags) and four responses: OK, NO,
BAD, and *; * carried data, and OK/NO/BAD indicated command completion.
RTT avoidance would be done via command pipelining, as in several other
protocols.
Supposedly, the need for multiple commands in progress (and thus, for
command tagging) was that a SEARCH might take a long while because the
operator might have to find the right backup 9-track and mount it; so it
was desirable to allow other commands to be executed out of order.
Needless to say, that was never a scenario that existed in real life. I
have never seen a server execute commands out of order, much less a client
expect command execution out of order.
As for unsolicited responses at any time, besides the BYE response (which
is important), there was a hope to avoid the need both for polling and an
IDLE command. As it turned out, NAT boxes effectively killed IDLE in its
infancy. Long live polling.
> These are minor little variations in the gran scheme of things, though.
> The vast bulk of IMAP is very tightly specified - in some cases so
> tightly specified that virtually nobody implements it correctly
> (witness the \Recent flag).
UW and Cyrus do... :-)
>> My exeperience using mutt with different IMAP servers is that it never
>> works quite the same with different servers.
> Mutt is pretty basic in its IMAP support, so yes, server behaviour -
> such as hierarchy - is directly visible to the user.
Yes, and (as I said earlier) that was once considered to be a feature, not
a bug.
> Like, for instance, servers sending FETCH or EXISTS responses
> when no command is in progress, even if they don't support IDLE. It's
> legal, but nobody does it. Hardly any clients can cope, either, but
> Polymer can, just as it can cope with commands executing out of order.
This is a good example of Postel's robustness principle. The c-client
library (thus programs based upon it such as Pine) can cope, but I
wouldn't think of actually sending such. The most that UW imapd does in
this regard is send unsolicited EXISTS, RECENT, and a FETCH carrying
flags, but only while a command is in progress.
> Note that the last entry is "Other Servers" - I do know that Polymer
> works perfectly well with UW-IMAP, for instance, because UW-IMAP
> conforms to RFC3501 etc. I've not put UW-IMAP on that list, because it
> happens that I don't have an account on such a server.
Can't you install it on your own system? It supports almost every UNIX
variant (certainly all the modern ones). With some effort, you can also
run it on Windows.
ftp://ftp.cac.washington.edu/mail/imap.tar.Z
for the current release version, and
ftp://ftp.cac.washington.edu/mail/imap-2005.DEV.tar.Z
for the alpha version of the next release (which adds UIDPLUS support).
Mark Crispin
> using, that's what makes things easy for the user.
Thus the conflict: does your client work the same, no matter what server;
or does the server work the same, no matter what client?
Most of us who've been in this war for the past 15 or so years have
concluded that the only solution would be to define one way and make it
mandatory for both clients and servers. The answer to the people who say
that "you can't do that" is "the web proves that you can do that."
Paul Rubin
> It is extremely difficult to build a hand coded parser for non-trivial
> protocols; and these days it is completely unnecessary due to the
> excellent ABNF parser generators that are out there.
Oh cool, I didn't realize that was the normal approach now. I thought
I had done something clever in kludging one together specifically for
IMAP. In this case it will be good to see the IMAP formal syntax
reorganized somewhat, to make it easier to put callbacks in the right
places in the automatically generated parser.
> This isn't to criticize old farts who still write hand coded parsers;
> I'm one myself. We've been there, and done that, and gotten the
> t-shirt. It just doesn't need to be done any more.
I do have the impression that (some of) the implementations running
out there now are hand coded, and have errors. There is also the
generating side, which I doubt anyone does automatically from the BNF.
So that's another opportunity for errors. E.g. I remember something
about Outlook putting some extra spaces someplace where there was
supposed to only be one space.
Mark Crispin
> out there now are hand coded, and have errors.
Indeed. The most common blunder (in any protocol) is to implement from
examples rather than the ABNF.
> There is also the
> generating side, which I doubt anyone does automatically from the BNF.
It's actually easier, even when hand-coding, to create generator code from
the ABNF than it is for parsing code. In generator code, you don't have
multiple paths, with the possible exception of deciding which form to use
for an astring.
I guess that there will always be people who want to hand-code a parser
and/or a generator.
Paul Rubin
> It's actually easier, even when hand-coding, to create generator code
> from the ABNF than it is for parsing code. In generator code, you
> don't have multiple paths, with the possible exception of deciding
> which form to use for an astring.
Hmm, interesting, yeah I guess you could. Do you know of any
particular programs that are good for this?
> I guess that there will always be people who want to hand-code a
> parser and/or a generator.
Clearly it's to be avoided if you've got the tools to do it
automatically.
Dave Cridland
Mark Crispin wrote:
> On Fri, 7 Oct 2005, Dave Cridland wrote:
> > Polymer should work with any IMAP4rev1 server. It might work with a
> > plain IMAP4 server, but I'd treat any errors there as low priority.
>
> I don't think that it makes much sense to work with IMAP4 (RFC 1730).
> IMAP4 was a false start. If you must fall back from IMAP4rev1 (RFC
> 2060/3501), it would be better to fall back to IMAP2 (RFC 1064/1176).
>
No, I don't think there's much point either, which is why it's not
really supported as such, it just "might work" - Polymer won't send
STATUS, for instance, but that's about it.
> > In
> > addition, there's exciting choices like multiple commands in progress,
> > unsolicited reponses at any time, etc.
>
> I never considered these to be particularly important, but there was a
> small but very loud contigent (now long gone) that screamed for multiple
> commands in progress.
>
Not long gone, actually. :-)
> Supposedly, the need for multiple commands in progress (and thus, for
> command tagging) was that a SEARCH might take a long while because the
> operator might have to find the right backup 9-track and mount it; so it
:-)
> was desirable to allow other commands to be executed out of order.
>
Yes, and I think it is.
> Needless to say, that was never a scenario that existed in real life. I
> have never seen a server execute commands out of order, much less a client
> expect command execution out of order.
>
Polymer doesn't expect commands to execute out of order, but it
certainly copes. My ACAP server very deliberately does this, in fact,
which is partly why I'm confident Polymer will cope with an IMAP server
doing the same.
> As for unsolicited responses at any time, besides the BYE response (which
> is important), there was a hope to avoid the need both for polling and an
> IDLE command. As it turned out, NAT boxes effectively killed IDLE in its
> infancy. Long live polling.
>
IDLE is a right royal pain in the neck. I'd infinitely prefer
unsolicited responses at any time. Polling is also pretty awful, and
equally, both waste bandwidth.
> > These are minor little variations in the gran scheme of things, though.
> > The vast bulk of IMAP is very tightly specified - in some cases so
> > tightly specified that virtually nobody implements it correctly
> > (witness the \Recent flag).
>
> UW and Cyrus do... :-)
>
And I think that's the sum total of correct implementations in this
regard.
> > Like, for instance, servers sending FETCH or EXISTS responses
> > when no command is in progress, even if they don't support IDLE. It's
> > legal, but nobody does it. Hardly any clients can cope, either, but
> > Polymer can, just as it can cope with commands executing out of order.
>
> This is a good example of Postel's robustness principle. The c-client
> library (thus programs based upon it such as Pine) can cope, but I
> wouldn't think of actually sending such. The most that UW imapd does in
> this regard is send unsolicited EXISTS, RECENT, and a FETCH carrying
> flags, but only while a command is in progress.
>
I think there are servers which actually do, these days.
> > Note that the last entry is "Other Servers" - I do know that Polymer
> > works perfectly well with UW-IMAP, for instance, because UW-IMAP
> > conforms to RFC3501 etc. I've not put UW-IMAP on that list, because it
> > happens that I don't have an account on such a server.
>
> Can't you install it on your own system?
Only when I find a spare system. :-) All my servers have IMAP
implementations on them at the moment.
Dave.
Mark Crispin
>> As for unsolicited responses at any time, besides the BYE response (which
>> is important), there was a hope to avoid the need both for polling and an
>> IDLE command. As it turned out, NAT boxes effectively killed IDLE in its
>> infancy. Long live polling.
> IDLE is a right royal pain in the neck. I'd infinitely prefer
> unsolicited responses at any time. Polling is also pretty awful, and
> equally, both waste bandwidth.
Unsolicited responses at any time require one end to be asynchronous and
the other end to observe when it's running out of TCP window; otherwise a
deadlock can result. This can be done, but it would kick a number of
individuals out of the cadre of IMAP software developers (some would argue
whether that is a bug or a feature).
Unfortunately, NAT decided this issue for us. Clients MUST poll at
frequent intervals to keep the NAT mapping alive. IDLE and unsolicited
responses at any time are unworkable when NAT is in the picture; and these
days you have to assume that NAT is in the picture. Server-initiated
traffic on a session whose NAT mapping has timed out will reward the
server with a TCP RST.
I don't see any change to the situation on the horizon. IPv6 isn't likely
to deliver us from NAT, since NAT is now being used for many other
purposes other than dealing with address space exhaustion.
Jorey Bump
news:Pine.WNT.4.65.05...@Tomobiki-Cho.CAC.Washington.EDU:
> On Fri, 7 Oct 2005, use...@isbd.co.uk wrote:
>> I want my IMAP client to behave the same whatever server I'm
>> using, that's what makes things easy for the user.
>
> Thus the conflict: does your client work the same, no matter what
> server; or does the server work the same, no matter what client?
>
> Most of us who've been in this war for the past 15 or so years have
> concluded that the only solution would be to define one way and make
> it mandatory for both clients and servers. The answer to the people
> who say that "you can't do that" is "the web proves that you can do
> that."
But there's an interesting difference in the way HTTP was specified
compared to most mail protocols. IMAP, SMTP, and POP servers announce a
variety of information about capabilities and even name/version of the
server, but take no interest in the identity of the client software.
At first, this seems very practical, because it demands strict adherence
to the protocol, regardless of the source. Theoretically, it puts all the
pressure for conformance on the client, which must learn to behave in
order to participate.
Compare this to HTTP's inclusion of the (optional) User-Agent header
field. The bane of server and web developers, alike, it at least offered
something to check in case modifications were needed to get things
working *today*. At the beginning of this thread, I was convinced it was
a perfect example of why a protocol should be user-agent agnostic. But
then I asked myself, "Why is the user experience for HTTP so much better
today than it is for email?" Perhaps it's because developers got sick and
tired of misbehaving clients, and they could name names to focus pressure
where it belonged. But it might also be because the user experience was
allowed to evolve and mature to the point where everyone could agree on
what the goal was, what kind of behaviour everyone could reasonably
expect.
It's quite a stretch to say that HTTP is now perfect, but email is
*nowhere* near the same point concerning user experience. Can you believe
that we still have to explain to users that they need to expunge the mail
they just deleted? That local folders must be compressed to reclaim disk
space? That some servers allow subdirectories beneath folders, while
others only allow subdirectories within directories, but first let me
describe the difference between a directory and a folder? That you must
specify where your mail is kept on the server, and that "mail" is not the
same as "Mail"? That your webmail interface might rewrite .mailboxlist in
a way that's incompatible with your desktop client, hiding folders and
causing you to go over quota? That your login might not be the same as
the LHS of your email address? That "Secure Password Authentication" is
not the same as "server requires authentication"?
And that's just for the user. Now the admin gets to discover that in
spite of a common interface like IMAP, the effect of choosing a mailbox
format on the server reverberates all the way to the client. Or that
there seem to be more authentication methods than there are choices of
servers, and your preferred MTA uses a different one than your favorite
IMAP server. Or that webmail is unbelievably slow, and makes a thousand
times more connections to your IMAP server than a desktop client, even
though it's running on the same box.
But now I seem to be ranting. I simply meant to say that email sucks, for
everyone involved. I think these issues offer a pretty clear roadmap for
innovation. I want to simply enter a URI to read my mail, end of
configuration. The Next Big Thing is and always will be: Make my life
easier. Not yours, mine.
use...@isbd.co.uk
> On Fri, 7 Oct 2005, use...@isbd.co.uk wrote:
> > I want my IMAP client to behave the same whatever server I'm
> > using, that's what makes things easy for the user.
>
> Thus the conflict: does your client work the same, no matter what server;
> or does the server work the same, no matter what client?
>
> Most of us who've been in this war for the past 15 or so years have
> concluded that the only solution would be to define one way and make it
> mandatory for both clients and servers. The answer to the people who say
> that "you can't do that" is "the web proves that you can do that."
>
it isn't.
--
Chris Green
Dave Cridland
Jorey Bump wrote:
> Mark Crispin <M...@CAC.Washington.EDU> wrote in
> news:Pine.WNT.4.65.05...@Tomobiki-Cho.CAC.Washington.EDU:
>
> > On Fri, 7 Oct 2005, use...@isbd.co.uk wrote:
> >> I want my IMAP client to behave the same whatever server I'm
> >> using, that's what makes things easy for the user.
> >
> > Thus the conflict: does your client work the same, no matter what
> > server; or does the server work the same, no matter what client?
> >
> > Most of us who've been in this war for the past 15 or so years have
> > concluded that the only solution would be to define one way and make
> > it mandatory for both clients and servers. The answer to the people
> > who say that "you can't do that" is "the web proves that you can do
> > that."
>
> But there's an interesting difference in the way HTTP was specified
> compared to most mail protocols. IMAP, SMTP, and POP servers announce a
> variety of information about capabilities and even name/version of the
> server, but take no interest in the identity of the client software.
>
ACAP tells you the server, none of the others do by default. IMAP has
an extension to do both client and server identification, but it's
rarely implemented.
> Compare this to HTTP's inclusion of the (optional) User-Agent header
> field. The bane of server and web developers, alike, it at least offered
> something to check in case modifications were needed to get things
Woah! Wait there a minute.
The problem is that almost nothing works as advertised in HTTP, HTML,
etc. Having a browser which is actually standards compliant is a total
novelty, and most of the time, that field lies in ever more innovative
ways.
Designing extensions and applications of HTTP is a nightmare, because
with every step you discover that some mandatory feature of the base
specification is actually not implemented by anyone.
Thankfully, this simply doesn't happen in email. Yes, it might not be
perfect, but at least it does what it says on the tin.
> space? That some servers allow subdirectories beneath folders, while
> others only allow subdirectories within directories, but first let me
> describe the difference between a directory and a folder? That you must
This one I'll grant you - this is a bit of a pain.
> And that's just for the user. Now the admin gets to discover that in
> spite of a common interface like IMAP, the effect of choosing a mailbox
> format on the server reverberates all the way to the client. Or that
To a limited extent - that is, the factor above.
> there seem to be more authentication methods than there are choices of
> servers, and your preferred MTA uses a different one than your favorite
> IMAP server. Or that webmail is unbelievably slow, and makes a thousand
That I've never seen. Maybe I'm just lucky.
> times more connections to your IMAP server than a desktop client, even
> though it's running on the same box.
>
Yes, well, I haven't written a webmail just yet. :-)
> I want to simply enter a URI to read my mail, end of
> configuration.
Oh, yeah, you can do that with Polymer, if your admin has set it up.
Dave.
John Beardmore
Cridland <da...@cridland.net> writes
>A similar attitude to what? All I say in the paragraph quoted below is
>that I haven't tested every combination of every extension or optional
>behaviour because there are lots of them, and that you should always
>pick a server and client which, combined, do what you want.
Trouble is that to the casual naive user who just wants email
flexibility, all this is a bit off putting.
The naive user doesn't even know which extensions or optional behaviours
are out there.
Is this information collected together in a 'things to know before you
start' document ?
Cheers, J/.
--
John Beardmore
Mark Crispin
> The naive user doesn't even know which extensions or optional behaviours are
> out there.
Supposedly, a novice (I dislike the term "naive" in this context) user
shouldn't have to know any of this.
All IMAP implementations (client or server) are supposed to implement the
IMAP base specification, and work with other implementations that also
implement the base specification.
Extensions exist as a matter of convenience; typically, to make it
possible for a client to perform a particular task faster/more efficiently
than if the extension was not present. For example, there is an extension
for server-based message threading. Without that extension, a client must
fetch all the message data needed for threading, and then it must
calculate the threads. The client can certainly can do all that work
itself; it's just slower.
Problems occur when a client decides that a particular extension is
"mandatory", and declines to work with a server that does not offer it; or
if the server has a false/broken implementation of an extension. Either
way, the client or server implementation that does this is broken. It
should not be the user's fault.
Sadly, I know of circumstances in which a client used a "mandatory
extension" in order to promote that vendor's proprietary server product.
No, it wasn't Microsoft.
Optional behaviors are something else, and are tied to site policy.
Are you allowed to have mailboxes on the IMAP server other than your
INBOX? That is an optional behavior.
Are you allowed to put mailboxes inside other mailboxes along with
messages, or do you have mailboxes which contain messages and directories
which contain mailboxes and/or other directories? That is an optional
behavior; and you will find vigorous partisans of either way.
The IMAP client is required by the base specification to recognize what is
going on, using the tools provided in the protocol, and present the
server's behavior to the user in an intelligible manner.
Sadly, many clients fall short of this ideal, and claim to the user that
the server is defective when the real problem is that the client isn't
following the rules.
Jeffrey Goldberg
> ... but it *is* a fault in the protocol from the user's point of
> view. I want my IMAP client to behave the same whatever server I'm
> using, that's what makes things easy for the user.
Let's look at SMTP as an analogy to clarify this point. Servers, when
they reject mail, have the option of doing it during the SMTP session or
accepting the message and generating their own bounce. Servers (and
their administrators) will have their local reasons for deciding what to
do when. Is this a fault of the SMTP protocol? Should clients insist
that it be one way or the other? Should all servers conform to a single
policy?
I am not trying to argue that just because that server latitude seems
reasonable in SMTP that the hierarchy latitude is reasonable in IMAP.
The point that I am trying to make is that you can't argue in principle
that server side determination of behavior is always wrong.
IMAP has a mechanism for communicating to the client how it deals with
hierarchy. If clients ignore that information, the clients are broken.
If you feel that clients should present a uniform hierarchy scheme
across servers, than write a client that *uses* the appropriate scheme
for each server but *presents* all of them in a consistent way to the user.
-j
Dave Cridland
Mark Crispin wrote:
> On Sun, 9 Oct 2005, John Beardmore wrote:
> > The naive user doesn't even know which extensions or optional behaviours are
> > out there.
>
> Supposedly, a novice (I dislike the term "naive" in this context) user
> shouldn't have to know any of this.
>
Essentially, no. But even novice users are likely, in the future, to
want to know if their server (or provider, perhaps more accurately)
uses LEMONADE or not, for instance, if they care about low-bandwidth
(ie, mobile) email. LEMONADE happens to be a set of extensions, rather
than one.
There's likely to be a similar issue if/when ANNOTATE starts being
implemented by servers, or indeed if ANNOTATEMORE starts being useful
to clients. (And it's going in Polymer soon).
Really, I ought to make Polymer allow a user to probe servers and give
an opinion on how well-matched they are for its purposes. I could stick
the same on a web page, too, it might even be useful. (You can
right-click on a mailserver in Polymer and get an annotated capability
listing, but that's not quite the same.)
> All IMAP implementations (client or server) are supposed to implement the
> IMAP base specification, and work with other implementations that also
> implement the base specification.
>
Indeed - if Polymer doesn't do this, it's a bug. (And currently there
is such a bug - the current "build" fails to operate with servers that
don't support NAMESPACE).
In my opinion, if any client fails to work with any server that
implements the base IMAP4rev1 specification, but claims to be an IMAP
client, it's buggy.
> Extensions exist as a matter of convenience; typically, to make it
> possible for a client to perform a particular task faster/more efficiently
> than if the extension was not present. For example, there is an extension
> for server-based message threading. Without that extension, a client must
> fetch all the message data needed for threading, and then it must
> calculate the threads. The client can certainly can do all that work
> itself; it's just slower.
>
Although, of course, if a new message arrives, to re-thread that the
client must either ask the server to rethread the entire mailbox
(modulo current filter), or else fetch much of the data required
anyway. Server-side sorting is much the same, just slightly cheaper if
more round-trips are tolerable. Oh, for contexts. But I digress.
Other extensions exist which don't just provide optimizations, however,
such as the ANNOTATE/ANNOTATEMORE family, which actually extend
metadata. In other words, they'll let you stick a note against a
message, or mark a message as belonging to a particular project, or due
by a particular date. (Or, for me, mark a bug report as applicable to a
particular revision, and whether it's fixed or not, etc.)
> Sadly, I know of circumstances in which a client used a "mandatory
> extension" in order to promote that vendor's proprietary server product.
> No, it wasn't Microsoft.
I know of a case like this too. In fact, there are several cases of
"Outlook plugins" which only work with a particular server.
> Are you allowed to put mailboxes inside other mailboxes along with
> messages, or do you have mailboxes which contain messages and directories
> which contain mailboxes and/or other directories? That is an optional
> behavior; and you will find vigorous partisans of either way.
>
> The IMAP client is required by the base specification to recognize what is
> going on, using the tools provided in the protocol, and present the
> server's behavior to the user in an intelligible manner.
>
Out of curiosity, how does a client determine that a CREATE foo will
result in a mailbox such that CREATE foo/bar (assuming "/" seperates
hierarchy) will fail?
Dave.
Mark Crispin
> There's likely to be a similar issue if/when ANNOTATE starts being
> implemented by servers, or indeed if ANNOTATEMORE starts being useful
> to clients. (And it's going in Polymer soon).
I think that ANNOTATEMORE is more likely to happen than ANNOTATE, although
ANNOTATEMORE also suffers from somewhat strange syntax.
> Indeed - if Polymer doesn't do this, it's a bug. (And currently there
> is such a bug - the current "build" fails to operate with servers that
> don't support NAMESPACE).
Wouldn't it be relatively straighforward to use the return data from
tag LIST "" ""
as the first (and only) namespace? Is there something that I am missing
that makes it more difficult?
> Although, of course, if a new message arrives, to re-thread that the
> client must either ask the server to rethread the entire mailbox
> (modulo current filter), or else fetch much of the data required
> anyway.
The CPU time involved in threading is rarely the issue, even for large (6+
digit message counts) mailboxes. It's the data fetch.
There has been talk about being able to do a partial update to a previous
thread. The problem is that a new message can completely change the
thread tree. It isn't like server side sorting, where it's just a
question of inserting in the right place; that would be easy to do, but
the problem is that most users prefer to thread instead of sort.
> Other extensions exist which don't just provide optimizations, however,
> such as the ANNOTATE/ANNOTATEMORE family, which actually extend
> metadata.
You can accomplish the same user experience with a separate mailbox in
which you stash the data that you would have placed in an annotation.
The same thing holds true for address books and configuration; it may seem
to be a "poor man's" way of doing things, but it actually works remarkably
well and doesn't require any additional infrastructure.
There's a lesson to be learned there; before doing things "the right way",
make sure that the right way is compellingly better than the poor man's
way. Otherwise, the poor man's way will bury you.
>> Sadly, I know of circumstances in which a client used a "mandatory
>> extension" in order to promote that vendor's proprietary server product.
>> No, it wasn't Microsoft.
> I know of a case like this too. In fact, there are several cases of
> "Outlook plugins" which only work with a particular server.
I wouldn't be surprised if the evil empire did it too (although they have
two very different IMAP server products). It's just that the cases I'm
most familiar with weren't them.
> Out of curiosity, how does a client determine that a CREATE foo will
> result in a mailbox such that CREATE foo/bar (assuming "/" seperates
> hierarchy) will fail?
I'm not sure that I understand the question.
There's no general way to determine that CREATE ...name... will result in
a name such that such-and-such will fail. There is, however, a way to
determine that that CREATE ...name... will result in a name such that
such-and-such will *succeed*. After the fact, a LIST will answer all
questions about success or failure.
The hierarchy delimiter for the root namespace can be determined by the
command:
tag LIST "" ""
Assuming that the hierarchy delimiter is "/":
tag CREATE foo
will create a name foo such that foo can be accessed as a mailbox (a name
that contains messages). The name foo MAY also be accessed as a directory
(a name that contains other names); this can be determined by the command
tag LIST "" foo
and checking for the \NoInferiors state. If it has \NoInferiors state
then the name can not be accessed as a directory.
tag CREATE foo/
will create a name foo such that foo can be accessed as a directory (a
name that contains other names). The name foo MAY also be accessed as a
mailbox (a name that contains messages); this can be determined by the
command
tag LIST "" foo
and checking for the \NoSelect state. If it has \NoSelect state then the
name can not be accessed as a mailbox.
IMAP is success-oriented. There is no way in IMAP for the client to
create a name such that such-and-such will fail; you can only ask to
create a name such that such-and-such will succeed. There may be no such
thing on that server as a name in which such-and-such will fail.
Dave Cridland
> On Sun, 9 Oct 2005, Dave Cridland wrote:
> > There's likely to be a similar issue if/when ANNOTATE starts being
> > implemented by servers, or indeed if ANNOTATEMORE starts being useful
> > to clients. (And it's going in Polymer soon).
>
> I think that ANNOTATEMORE is more likely to happen than ANNOTATE, although
> ANNOTATEMORE also suffers from somewhat strange syntax.
>
Although ANNOTATE does appear to be finally gaining some momentum,
ANNOTATEMORE is way ahead, and in deployed usage. (And I agree its
syntax is a little strange).
> > Indeed - if Polymer doesn't do this, it's a bug. (And currently there
> > is such a bug - the current "build" fails to operate with servers that
> > don't support NAMESPACE).
>
> Wouldn't it be relatively straighforward to use the return data from
> tag LIST "" ""
> as the first (and only) namespace? Is there something that I am missing
> that makes it more difficult?
>
I think what made it difficult was probably a combination of lack of
sleep and my daughter crawling in at the wrong moment, but we'll never
know, now... Seriously, it's just a bug. I was just optimizing the
initial opening of a server and forgot to allow for the no-NAMESPACE
case.
> > Although, of course, if a new message arrives, to re-thread that the
> > client must either ask the server to rethread the entire mailbox
> > (modulo current filter), or else fetch much of the data required
> > anyway.
>
> The CPU time involved in threading is rarely the issue, even for large (6+
> digit message counts) mailboxes. It's the data fetch.
>
Indeed - of course, since Polymer aims to be (and, in my opinion, is)
highly usable over GPRS and other low-bandwidth connections, that's
definitely the problem.
But for a 6+ digit message count, the number of octets adds up very
quickly indeed even for a THREAD response (It's 683k or so for 100,000
message mailboxes), and there's no quick-fix like ESEARCH to reduce it,
nor any method for chopping it into manageable chunks.
> There has been talk about being able to do a partial update to a previous
> thread. The problem is that a new message can completely change the
> thread tree. It isn't like server side sorting, where it's just a
> question of inserting in the right place; that would be easy to do, but
> the problem is that most users prefer to thread instead of sort.
>
True. But it can only change the thread tree for one "thread". (It may
merge two threads, of course).
As it stands, I think THREAD is too expensive for mobile clients,
useless for disconnected mode clients, and only useful for client which
aim to provide a snapshot view of a mailbox - basically meaning webmail
clients.
This may sound a little harsh, but the fact is that those users likely
to want a threaded view are also likely to have large mailboxes.
> > Other extensions exist which don't just provide optimizations, however,
> > such as the ANNOTATE/ANNOTATEMORE family, which actually extend
> > metadata.
>
> You can accomplish the same user experience with a separate mailbox in
> which you stash the data that you would have placed in an annotation.
> The same thing holds true for address books and configuration; it may seem
> to be a "poor man's" way of doing things, but it actually works remarkably
> well and doesn't require any additional infrastructure.
>
And it's awful over low bandwidth.
I use ACAP for that, in all the cases you mention, aside from
(currently) "extra" message and mailbox metadata. I'll fallback from
ANNOTATE/ANNOTATEMORE to ACAP, once the ANNOTATE code gets a UI, and
the ANNOTATEMORE code gets written.
> There's a lesson to be learned there; before doing things "the right way",
> make sure that the right way is compellingly better than the poor man's
> way. Otherwise, the poor man's way will bury you.
>
Yes, and I find ACAP compellingly better:
1) It's markedly superior when changes happen when multiple clients are
online - the changes occur simultaneously in all running instances.
XCAP can do this, but only if the clients are also subscribing to SIP
event packages. That's a lot of code to write, and if you've an IMAP
client, you have 99% of an ACAP client written. PINE's method would
require an open connection purely observing the client mailbox.
2) It's easily extensible for client authors - you can add in
arbitrary, namespaced, data. Whether or not XCAP can in fact handle
this is a subject of much debate at the moment - it's likely to be
pulled back from the RFC Editor's queue to rectify it. PINE's system
would not, directly, although some horror involving special comments
might work.
3) Much of the hard work of standardizing configuration schemas able to
be common to all MUAs has been done.
4) For large addressbooks (and similar style data), both server-side
enumerations and powerful searching make ACAP very compelling over
lower bandwidth. In addition, changes to configuration are easily
synchronized.
Obviously this leaves out of the really interesting things that ACAP
can also do, like inheritance, which isn't to say that's not important
too.
I did originally think I'd have to ditch ACAP for running on mobile
phones, but when I examined round-trip and latency in ACAP in more
detail, I came to the conclusion that mobile clients can easily afford
the handful of octets which allow the mobile email client to be
reconfigured comfortably via, say, a webmail or desktop client.
> >> Sadly, I know of circumstances in which a client used a "mandatory
> >> extension" in order to promote that vendor's proprietary server product.
> >> No, it wasn't Microsoft.
> > I know of a case like this too. In fact, there are several cases of
> > "Outlook plugins" which only work with a particular server.
>
> I wouldn't be surprised if the evil empire did it too (although they have
> two very different IMAP server products). It's just that the cases I'm
> most familiar with weren't them.
>
I don't know of a case where Microsoft does this - all the cases I know
of are third-party Outlook plugins. Outlook itself is, as far as I'm
aware, intended to be capable of running against any IMAP server
compliant to the base spec.
> > Out of curiosity, how does a client determine that a CREATE foo will
> > result in a mailbox such that CREATE foo/bar (assuming "/" seperates
> > hierarchy) will fail?
>
> I'm not sure that I understand the question.
>
You've answered it, but I'll rephrase for the record.
If a client wishes to create a mailbox foo, with the potential
intention of creating inferior mailboxes *and* having foo contain
messages, is there any way of knowing a priori if this is possible?
If I understand your answer correctly, the answer's no, which is what I
thought - it was largely a rhetorical question, counter to your
assertion that if a client uses the tools available to it, it can know
the differences in hierarchy behaviour.
So a UI for "Create subfolder" has to ask two questions: "What name
would you like?" and "Do you want subfolders or messages?". Even though
for many servers, there's no distinction, and it's somewhat confusing
for users.
Really there ought to be a CAPABILITY string or a NAMESPACE extension
string (or both) explaining what the hierarchy is actually capable of.
Dave.
Mark Crispin
> But for a 6+ digit message count, the number of octets adds up very
> quickly indeed even for a THREAD response (It's 683k or so for 100,000
> message mailboxes), and there's no quick-fix like ESEARCH to reduce it,
> nor any method for chopping it into manageable chunks.
True, but do you really want to thread 100K messages from a mobile client?
My experience is that most clients fail to scale with much smaller message
counts, having nothing to do with protocol or network traffic issues.
I suppose that you know that you can thread a range of messages; there is
no requirement that you thread all messages. You do run the risk of your
subset message range thread being different that a subset of the total
thread.
> True. But it can only change the thread tree for one "thread". (It may
> merge two threads, of course).
It can merge more than two threads. The level of change can be
spectacular...
> As it stands, I think THREAD is too expensive for mobile clients,
> useless for disconnected mode clients, and only useful for client which
> aim to provide a snapshot view of a mailbox - basically meaning webmail
> clients.
Huh? THREAD works quite well for online clients!
> This may sound a little harsh, but the fact is that those users likely
> to want a threaded view are also likely to have large mailboxes.
Yes, but few MUAs today work well with 4-digit message counts, even fewer
work well with 5-digit message counts, and even fewer work at all with 6+
digit message counts.
Of course, what constitutes "large" has changed in the past (I remember
when 3-digit was "large") and it will doubtless change in the future.
However, I believe that the speed at which constitutes "large" number of
message count has grown has slowed down, and has been almost at a
standstill for at least a decade.
My observations suggest that people split up large message count mailboxes
into multiple smaller mailboxes long before they approach the message
count limits of their MUA.
Mailboxes have grown enormously, but almost entirely in the vector of
average message size as opposed to number of messages.
>> You can accomplish the same user experience with a separate mailbox in
>> which you stash the data that you would have placed in an annotation.
>> The same thing holds true for address books and configuration; it may seem
>> to be a "poor man's" way of doing things, but it actually works remarkably
>> well and doesn't require any additional infrastructure.
> And it's awful over low bandwidth.
What constitutes "low"?
I have yet to find a mobile client that I can tolerate, but I've used Pine
over 1xRTT, and before than over CDPD. It wasn't all that long ago that I
finally said goodbye to 14.4k modems. I won't claim that it was a speed
demon, but it was faster than some other programs using broadband.
> I use ACAP for that
I'm glad to see that there's still some life in ACAP; but I fear the jury
is still out. Sad to say, LDAP seems to have won the address book war.
> If a client wishes to create a mailbox foo, with the potential
> intention of creating inferior mailboxes *and* having foo contain
> messages, is there any way of knowing a priori if this is possible?
No.
> If I understand your answer correctly, the answer's no, which is what I
> thought - it was largely a rhetorical question, counter to your
> assertion that if a client uses the tools available to it, it can know
> the differences in hierarchy behaviour.
That doesn't really make much of a point. You have no way of knowing
whether you can create any name at all
> So a UI for "Create subfolder" has to ask two questions: "What name
> would you like?" and "Do you want subfolders or messages?". Even though
> for many servers, there's no distinction, and it's somewhat confusing
> for users.
I eschew the ambiguous term "folder" for this reason.
Your mileage may vary, but I find it remarkable that people expect that a
mailbox should contain other mailboxes. I haven't yet heard a groundswell
of support for C language source files to serve a second purpose as a
container of other source files.
> Really there ought to be a CAPABILITY string or a NAMESPACE extension
> string (or both) explaining what the hierarchy is actually capable of.
That is the wrong granularity, since the state is per-mailbox and it is
perfectly possible for a mail store hierarchy to have multiple types of
mailbox objects.
Jeffrey Goldberg
> Compare this to HTTP's inclusion of the (optional) User-Agent header
> field. The bane of server and web developers, alike, it at least offered
> something to check in case modifications were needed to get things
> working *today*. At the beginning of this thread, I was convinced it was
> a perfect example of why a protocol should be user-agent agnostic. But
> then I asked myself, "Why is the user experience for HTTP so much better
> today than it is for email?"
Yikes. I won't respond to that directly. But in another the story of
HTML has persuaded me that MRC is right about something I used to think
he was wrong about. In client and server design the established
philosophy was "be liberal in what you accept, conservative in what you
send." That made plenty of sense to me (and other people). I remember
being bewildered when I first encountered MRC's opposition to that
seemingly common sense idea.
But HTML provides the extreme case of what goes wrong with that. (It
was when I started implementing non-sendmail servers that I started to
realize the nature of the problem, but the story with HTML is so much
clearer).
Early browsers, lynx and mosaic, were a bit liberal about what they
accepted. Lynx, however, displayed a nice little warning that the HTML
was invalid. When the browser wars really began (Mozilla and IE), one
"feature" on which they competed was how well they rendered broken HTML.
From a user's point of view, a browser that did a better job of
guessing what was intended by broken HTML provided a better user
experience. HTML authors tinkered with their HTML until it "worked",
but not until it was right. That is, they were not very conservative in
what they sent. There was no incentive to be conservative in what they
sent. So as browsers got better at parsing broken HTML, more broken
HTML populated the web.
I'm sure that by now the vast majority of the HTML reading code in
browsers is about reading broken HTML. Then we start getting version
compatibility issues. It's easier for a browser to be backwards
compatible with earlier HTML standards, but it's much harder for it to
be backwards compatible with how its prior versions interpreted some
particularly broken HTML document. Also for "web developers" (there'd
be much less need for those specialists if HTML hadn't gone down this
route) there is much more work because making an innocuous change to a
broken document can lead to unpredictable results in how it renders.
If popular browsers stopped accommodating broken HTML, then the problem
would swiftly disappear, and writing web pages and building browsers
would be much easier. But of course that isn't going to happen because
any popular browser that did that would quickly cease to be popular.
With SMTP there was a way out. When SMTP clients and servers that
worked fine with and depended on sendmail's failure to follow the
standards started to fail with sendmail alternatives, the people who
made the policy decisions were informed enough to put pressure where it
belonged.
Let me give an example that I was party to. With Express Outhouse on
win95 (or what it win98?) it sent the machine's hostname with HELO. If
the machine had been named "Jeffs Computer", then it would send "EHLO
jeffs_computer" (at least it was thoughtful enough replace the space
with something, but it would have been nice if it had replaced it with
something that didn't violate standards.)
The MTA that I was working with at the time, exim, gave a 5xx on those
EHLOs. Sendmail took them just fine. A configuration option was
quickly added to exim to accept such broken HELOs (be liberal in what
you accept), but many email administrators, including me, decided not to
enable that option. Sure some of it may have been motivated by an
anti-MS attitude, but whatever our inner motivations were, it lead us to
see the dangers of "be liberal in what you accept". Instead, there was
a feeling that this trend had to be nipped in the bud. Why should we
break exim to imitate how sendmail was broken just to encourage clients
to do broken things?
In that case, and other things with SMTP, we were able to contain the
problem. But that, I believe, is because the choices were made by
people writing and administering SMTP clients and servers. With HTML it
was a different story. The people writing (broken) HTML had little
stake in doing the right thing, and probably less idea of the
implications of accommodating broken HTML. The writers of the browsers
may well have foreseen the consequences, but market forces pushed in the
other direction.
So I think that the current difficulty of writing HTML readers and of
maintaining HTML in the face of browser changes is a direct consequence
of "be liberal in what you accept." I think that the web experience
tells us what we want to avoid, and is hardly an example of success.
Are you by any chance a web designer? I think that only someone's whose
skills are focused on working around the problems erratic HTML could
think that user agent sniffing in HTML is a good model of anything.
I'm sorry to be so harsh, but for me, that kind of User Agent detection
and action epitomizes what has gone wrong with the web. It's not
something that I would ever want to see as a "solution" in protocols.
-j
Mark Crispin
> In client and server design the established philosophy was "be
> liberal in what you accept, conservative in what you send." That made plenty
> of sense to me (and other people). I remember being bewildered when I first
> encountered MRC's opposition to that seemingly common sense idea.
Just as a reminder, I never disagreed with Postel's robustness principle.
In fact, I strongly agree to it -- with the interpretation that Jon Postel
intended.
The older protocols (TELNET, FTP) had a plethora of facilities that hardly
anyone used, but were in the specification because someone thought they
were important. Some of these were mandatory-to-implement as opposed to
options; a good example was the "go ahead" in TELNET which served the
purpose of unlocking the keyboard on an IBM 2741.
Nevertheless, there was a subset in each of these protocols that everybody
implemented and actually used. In TELNET, these were transmit-binary,
echo, and suppress-go-ahead.
Thus, being "conservative in what you send" meant to limit what you send
to this subset. TELNET and FTP may have all these nifty features, but
they weren't commonly used and even if they were implemented they probably
didn't actually work.
Conversely, being "liberal in what you accept" meant that even though some
part of the specification looks totally useless, you should implement and
test it anyway just in case someone tries to use it.
These days, unfortunately, the "be conservative" part of the robustness
principle has been discarded entirely. Worse, the "be liberal" part of
the robustness principle has been twisted to mean something entirely
different: the notion that if it looks something like what the protocol
specifies, and if a human could figure out what was intended, then it
should be interpreted "correctly" even though it's invalid under the
protocol.
The result, typified by HTML, is almost the exact opposite of what the
robustness principle intended: rather than being a subset of the published
protocol, the actual protocol in operational use becomes an unpublished
*superset* of the published protocol.
HTML did not exist when IMAP was designed, but message headers served as
an invaluable lesson of what happened when the protocol allowed excessive
flexibility in syntax and imposed insufficient syntactic rigor. One of
the hopes (imperfectly filled) of IMAP was that servers would do all the
dirty (but necessary) work of dealing with bad syntax (the modern
interpretation of "be liberal") so that clients could be written for a
perfect world.
Dave Cridland
> On Sun, 9 Oct 2005, Dave Cridland wrote:
> > But for a 6+ digit message count, the number of octets adds up very
> > quickly indeed even for a THREAD response (It's 683k or so for 100,000
> > message mailboxes), and there's no quick-fix like ESEARCH to reduce it,
> > nor any method for chopping it into manageable chunks.
>
> True, but do you really want to thread 100K messages from a mobile client?
>
Why not? Polymer already handles 100K message mailboxes perfectly well
over low-bandwidth connections, even in a cold-cache situation. Why on
earth wouldn't I want to thread them too? (I don't see any problem with
handling much higher message counts, either, I just can't be bothered
to build such a mailbox.)
> I suppose that you know that you can thread a range of messages; there is
> no requirement that you thread all messages. You do run the risk of your
> subset message range thread being different that a subset of the total
> thread.
>
Well, yes. I *could* thread only a small portion of the messages. But
what, exactly, does that mean? In general, it means I'll have lots of
partial threads, with no easy way of reconnecting the subthreads.
If I could select the threads matching a criteria, this would be
easier, but I can't - I have to select the messages, then thread them,
instead of threading the mailboxes, and then select the threads I want.
If I could do the latter, it'd be easy - I'd want to display threads
which had a message with a relatively recent internal date, sorted by
highest internal date. Or even better, the last N threads.
> > True. But it can only change the thread tree for one "thread". (It may
> > merge two threads, of course).
>
> It can merge more than two threads. The level of change can be
> spectacular...
>
Yes, true. It's possible that it can merge all threads in the mailbox,
providing a missing, and previously unknown, common ancestor for all
threads.
> > As it stands, I think THREAD is too expensive for mobile clients,
> > useless for disconnected mode clients, and only useful for client which
> > aim to provide a snapshot view of a mailbox - basically meaning webmail
> > clients.
>
> Huh? THREAD works quite well for online clients!
>
Oh, across a LAN, yes.
Perhaps it'd be fairer if I simply said that both SORT and THREAD lack
the level of scalability found elsewhere in the protocol.
> > This may sound a little harsh, but the fact is that those users likely
> > to want a threaded view are also likely to have large mailboxes.
>
> Yes, but few MUAs today work well with 4-digit message counts, even fewer
> work well with 5-digit message counts, and even fewer work at all with 6+
> digit message counts.
>
Polymer gets regularly tested against a mailbox with nearly 130,000
messages in. This was one of the primary motivating factors behind
ESEARCH, although the set-syntax helps almost equally well with 130
mailboxes each with 1000 messages in. I'm confident that Polymer would
perform approximately the same over much higher message counts.
FWIW, I regularly use Polymer on mailboxes with 70k messages in, as do
some of the other users. I do this reasonably often over GPRS, so I'm
putting my money where my code is, as it were.
> My observations suggest that people split up large message count mailboxes
> into multiple smaller mailboxes long before they approach the message
> count limits of their MUA.
>
That's largely because most MUAs rapidly degrade as the message count
increases, and users have been invariably trained to "sweep" messages
across.
> Mailboxes have grown enormously, but almost entirely in the vector of
> average message size as opposed to number of messages.
>
Oh, that's grown too, certainly.
> >> You can accomplish the same user experience with a separate mailbox in
> >> which you stash the data that you would have placed in an annotation.
> >> The same thing holds true for address books and configuration; it may seem
> >> to be a "poor man's" way of doing things, but it actually works remarkably
> >> well and doesn't require any additional infrastructure.
> > And it's awful over low bandwidth.
>
> What constitutes "low"?
>
Well, my personal target is 2400 baud. If I can get Polymer usable over
that I'll be reasonably happy. The "reading your mail" bit now seems to
work fine at that bandwidth (or at least, Polymer's emulation of it),
the "configuration" bit is mostly okay. SORT and THREAD aren't
implemented, and folder listing is currently synchronous, which
effectively means "slow". Connections also take far too long for my
liking - I'm thinking of seeing how practical deploying something like
HIP would be on a mobile phone. On a laptop it's a no brainer, of
course.
> I have yet to find a mobile client that I can tolerate, but I've used Pine
> over 1xRTT, and before than over CDPD. It wasn't all that long ago that I
> finally said goodbye to 14.4k modems. I won't claim that it was a speed
> demon, but it was faster than some other programs using broadband.
>
Well, yes, PINE's handling of IMAP is very good - as I'd expect. But
configuration and addressbook handling is weak, in my opinion, both in
terms of functionality and expressiveness. That's not meant as an
attack on PINE, just a demonstration that a "poor man's" method, while
sometimes "good enough", isn't "as good as it can be".
As an aside, the problem with 14k4 modems wasn't the amount of data
transferred, per-se, but the time spent online, at least in the UK.
Polymer is asbolutely not designed to minimize the time spent online.
It's designed for circumstances where you're charged by transfer.
> > I use ACAP for that
>
> I'm glad to see that there's still some life in ACAP; but I fear the jury
> is still out. Sad to say, LDAP seems to have won the address book war.
>
Everyone liked, and still appears to like, the notion of ACAP. It's
just nobody actually wanted to take the plunge and actually use it -
just one conformant server and four MUAs, two of which are commercial
including Mulberry.
LDAP I'm not so certain has won the addressbook war - I think it's
managed a successful land grab, but failed to consolidate. It's used
heavily for canonical lists of address data, such as global
addressbooks and actual directories, but there seems little motion
toward using it as a personal addressbook. Moreover, I don't think
anyone's seriously considering accessing LDAP over a mobile link.
> > If a client wishes to create a mailbox foo, with the potential
> > intention of creating inferior mailboxes *and* having foo contain
> > messages, is there any way of knowing a priori if this is possible?
>
> No.
>
> > If I understand your answer correctly, the answer's no, which is what I
> > thought - it was largely a rhetorical question, counter to your
> > assertion that if a client uses the tools available to it, it can know
> > the differences in hierarchy behaviour.
>
> That doesn't really make much of a point. You have no way of knowing
> whether you can create any name at all
>
True, but you have a reasonable idea of whether it's possible, and a NO
response from CREATE should be exceptional rather than normal.
> > So a UI for "Create subfolder" has to ask two questions: "What name
> > would you like?" and "Do you want subfolders or messages?". Even though
> > for many servers, there's no distinction, and it's somewhat confusing
> > for users.
>
> I eschew the ambiguous term "folder" for this reason.
>
Yes, I suppose it'd be cleaner to have "create submailbox" and "create
subcontainer" or something.
> Your mileage may vary, but I find it remarkable that people expect that a
> mailbox should contain other mailboxes. I haven't yet heard a groundswell
> of support for C language source files to serve a second purpose as a
> container of other source files.
>
I like the analogy, forced as it is, but remarkable or not, and even
expectation or not, some servers allow this, some don't, and there's no
easy way for a client to know which short of "suck it and see", which
doesn't strike me as a clean methodology.
> > Really there ought to be a CAPABILITY string or a NAMESPACE extension
> > string (or both) explaining what the hierarchy is actually capable of.
>
> That is the wrong granularity, since the state is per-mailbox and it is
> perfectly possible for a mail store hierarchy to have multiple types of
> mailbox objects.
*nods* True. But in some cases, the behaviour is known on a
per-NAMESPACE or per-server basis.
Dave.
Jorey Bump
news:11kjg6i...@news.supernews.com:
[snip]
> I'm sure that by now the vast majority of the HTML reading code in
> browsers is about reading broken HTML. Then we start getting version
> compatibility issues. It's easier for a browser to be backwards
> compatible with earlier HTML standards, but it's much harder for it to
> be backwards compatible with how its prior versions interpreted some
> particularly broken HTML document. Also for "web developers" (there'd
> be much less need for those specialists if HTML hadn't gone down this
> route) there is much more work because making an innocuous change to a
> broken document can lead to unpredictable results in how it renders.
>
> If popular browsers stopped accommodating broken HTML, then the
> problem would swiftly disappear, and writing web pages and building
> browsers would be much easier. But of course that isn't going to
> happen because any popular browser that did that would quickly cease
> to be popular.
[snip]
> Are you by any chance a web designer? I think that only someone's
> whose skills are focused on working around the problems erratic HTML
> could think that user agent sniffing in HTML is a good model of
> anything.
>
> I'm sorry to be so harsh, but for me, that kind of User Agent
> detection and action epitomizes what has gone wrong with the web.
> It's not something that I would ever want to see as a "solution" in
> protocols.
Jeffrey, everything you wrote is 100% factually correct, and I agree with
it completely. Yes, I am a web developer (among other things), and I
religiously refuse to do any browser detection. Whenever I use JavaScript
to validate form submissions, it is never without a server side
equivalent to both protect the application and provide feedback to the
user who may not be using a JavaScript-enabled browser. I've been doing
this for years, even when there was a time that browser sniffing was a
popular crutch for simply getting pages to render sensibly.
But the User-Agent string has been a clue to other, more serious,
problems, such as the poor handling of SSL, which simply can't be
tolerated in many circumstances. Users and PHBs alike bring pressure to
bear on web developers ("My son at college says the page looks different
in Firefox 1.5 beta than it does on my version of Internet Explorer 4 --
fix it!"). This has put web developers in unique position to deflect that
pressure onto browser developers, and it seems to have had some effect.
Yes, browsers have always seemed to support broken HTML, allowing for
some pretty sloppy coding. But now they also look at the DOCSTRING
(gasp!) and offer some strict modes that never existed before. At one
time, most web developers started out as content providers, and it was
rare to find one that was code-savvy (I had 10 years of programming
experience before I saw HTML, and laughed/cursed myself silly at it).
That's changed quite a bit, but there are times when I ask myself why a
content provider should even be concerned with the arcane technical
details that are necessary to create a valid page that doesn't even meet
some reasonable expectations. I fear it will be impossible to adhere to
some established philosophies about accessibility and graceful
degredation in the face of the popularity of AJAX.
But I am also a mail server administrator, and sorely miss the single
interface, zero configuration benefits inherent in HTTP user interfaces.
There are only a few basic parameters necessary to access one's mail
(IMAP/POP host, SMTP host, login, password, SSL/TLS), but can you name a
single graphical client that puts all of these settings in one simple
screen or dialog box? Or any that let you check mail on the fly without
setting up a permanent account? I won't go into the administrative
backend issues, because I understand and expect there to be some
complexity here, especially as the technology is still evolving. But
changes at the server should be transparent to end users, who will
ideally share a common experience. Mail protocols are hardly in their
infancy, so why has the HTTP user experience *seemed* to outstrip any
email UI available today, in terms of consistency and ease of use?
I'm not really suggesting that the User-Agent header be added to mail
protocols. For all I know, the W3C should take the credit for nudging
browser developers toward the center (albeit slowly). But I'm also not
sure that the strictness of mail protocols has contributed to user-
friendliness as much as it has to USENET flamewars. With the widespread
deployment of SMTP AUTH and STARTTLS, we can now move beyond some
important client issues (for example, who cares if a properly
authenticated Outlook sends an invalid HELO/EHLO?). So I guess the
question is how do we get to a more unified client/server & server/server
communication interface? Is the answer some new layer that sits on top of
HTTP, requiring only a URI (using exisiting HTTP authentication
mechanisms and SSL for encryption)?
Mark Crispin
>> True, but do you really want to thread 100K messages from a mobile client?
> Why not? Polymer already handles 100K message mailboxes perfectly well
> over low-bandwidth connections, even in a cold-cache situation. Why on
> earth wouldn't I want to thread them too?
As does Pine; but the issue isn't a matter of software capability. It's a
matter of user managability. Most users who have that many messages (and
these are typically archived messages, not new messages) break them up
into separate mailboxes, usually named in some way to categorize the
archive (date range is popular, as is topic).
Yes, you can do searches instead. But do you really want to scroll
through 100K messages, and enter in searches through 100K messages, on a
mobile client?
> (I don't see any problem with
> handling much higher message counts, either, I just can't be bothered
> to build such a mailbox.)
OK, so your "can't be bothered" limit is at 100K. Very few people are
there. Most drop out after 10K, with the peak being in the 4 digit range.
>>> As it stands, I think THREAD is too expensive for mobile clients,
>>> useless for disconnected mode clients, and only useful for client which
>>> aim to provide a snapshot view of a mailbox - basically meaning webmail
>>> clients.
>> Huh? THREAD works quite well for online clients!
> Oh, across a LAN, yes.
Or wi-fi.
I think that the GPRS/1xRTT guys are rapidly losing their window of
opportunity. At about the same time that I finally dropped CDPD in favor
of 1xRTT, I observed that my need for WAN wireless had been plummeting
like a paralyzed falcon. So far, I've used WAN wireless *once* this
entire year.
The culprit is wi-fi. It's springing up everywhere. I've found wi-fi to
be ubiquitous and excellent in remote villages of Alaska and Canada where
there is no GPRS or 1xRTT (or even analog cell phone service). Wi-fi
hotspots are everywhere in cities.
You can tell that the mobile phone providers feel threatened, by their
enraged protest against municipal wi-fi networks.
Blackberry faces a different threat in the US. It'll be interested to see
how that turns out.
> Perhaps it'd be fairer if I simply said that both SORT and THREAD lack
> the level of scalability found elsewhere in the protocol.
That claim has been made many times. I disagree. It depends upon how you
look at it. I consider SORT and THREAD to be a form of SEARCH. If you
need the sequence/UID map to resynchronize, UID SEARCH ALL is the fastest
way to do that.
A more credible claim is that it's too costly to reTHREAD when new
messages arrive. That's true; but nobody has yet come up with a good way
to represent a delta thread.
It also needs to be pointed out that SORT and THREAD scale much better
than the previous methods. Note that the NNTP community, which threads
more than anyone else, still depends upon downloading overviews for all
articles!
> Polymer gets regularly tested against a mailbox with nearly 130,000
> messages in.
Tested, yes; but what about usage?
There is a reason that users limit the number of messages in a single
mailbox at a point much lower than their MUA is capable of handling. I
don't think that this reason has received adequate study.
Hierarchies of categorization work better for most humans than a flat
space (even with excellent searching tools).
> FWIW, I regularly use Polymer on mailboxes with 70k messages in, as do
> some of the other users. I do this reasonably often over GPRS, so I'm
> putting my money where my code is, as it were.
Well, yes. A reasonable IMAP client should work well with a 70K message
mailbox over GPRS, even if you start with a zero-state cache (pure online
client). I designed IMAP to work well over 2400 bps, which is what much
of my early work was over.
>> My observations suggest that people split up large message count mailboxes
>> into multiple smaller mailboxes long before they approach the message
>> count limits of their MUA.
> That's largely because most MUAs rapidly degrade as the message count
> increases, and users have been invariably trained to "sweep" messages
> across.
That's true, but I don't think that is the only reason. I think that
there is a degradation of human ability to deal with an excessively large
flat space; and that humans impose hierarchical categorization before
their tools for managing the flat space degrades.
OK, yes, there are still some MUAs that degrade in the 3-digit message
count range. But I don't think that proves your point. If anything, the
fact that such MUAs are still considered worthwhile indicates to me that
there is something else going on.
> Well, my personal target is 2400 baud. If I can get Polymer usable over
> that I'll be reasonably happy.
Yup, that's what I did. The problem is, it's getting very difficult to
justify forcing people to develop for that low a bandwidth, as it's
getting harder and harder to find places which are that low.
Of course, a poorly-performing wireless WAN can have an effective speed
that is that low... :-)
> Well, yes, PINE's handling of IMAP is very good - as I'd expect. But
> configuration and addressbook handling is weak, in my opinion, both in
> terms of functionality and expressiveness. That's not meant as an
> attack on PINE, just a demonstration that a "poor man's" method, while
> sometimes "good enough", isn't "as good as it can be".
Understood. It is impossible in human endeavor to produce anything that
some other person can't produce something better. I personally consider
Pine to be an overall "best" among today's offerings, but that's because I
give almost zero value to GUI. If I'm using a small screen, GUI objects
take away valuable real estate that can present text. What's more, none
of the GUIs that I've seen have been visually compelling.
> As an aside, the problem with 14k4 modems wasn't the amount of data
> transferred, per-se, but the time spent online, at least in the UK.
> Polymer is asbolutely not designed to minimize the time spent online.
> It's designed for circumstances where you're charged by transfer.
I suspect that both pricing models (time online or by transfer) are doomed
in the face of inexpensive flat-rate.
> Everyone liked, and still appears to like, the notion of ACAP. It's
> just nobody actually wanted to take the plunge and actually use it -
> just one conformant server and four MUAs, two of which are commercial
> including Mulberry.
Agreed. Sadly, I feel that ACAP attempted to bite off too much. That
last 2% is invariably 98% of the work.
> LDAP I'm not so certain has won the addressbook war - I think it's
> managed a successful land grab, but failed to consolidate. It's used
> heavily for canonical lists of address data, such as global
> addressbooks and actual directories, but there seems little motion
> toward using it as a personal addressbook. Moreover, I don't think
> anyone's seriously considering accessing LDAP over a mobile link.
The consolidation (and especialy use for personal addressbooks) is going
slower than I predicted, but the lumbering dinosaur is in motion. I'm
going to attend a dinosaur-pushing meeting shortly (and hope that I don't
get buried under dino-dung...) And yes, people are talking about using it
over mobile.
> Yes, I suppose it'd be cleaner to have "create submailbox" and "create
> subcontainer" or something.
Or just follow IMAP's lead, in which you create mailboxes and directories
are created implicitly.
>> Your mileage may vary, but I find it remarkable that people expect that a
>> mailbox should contain other mailboxes. I haven't yet heard a groundswell
>> of support for C language source files to serve a second purpose as a
>> container of other source files.
> I like the analogy, forced as it is, but remarkable or not, and even
> expectation or not, some servers allow this, some don't, and there's no
> easy way for a client to know which short of "suck it and see", which
> doesn't strike me as a clean methodology.
I actually agree; well, not about the analogy being "forced", but
everything else you said. IMAP failed to take a stand and instead tried
to forge a wretched compromise. The two sides of the time have come to
accept that it would have been better to have lost in favor of the other
side.
The reason why I don't think the analogy is forced is that "dual-use"
mailboxes require the assumption that a message (rather than the mailbox)
is the terminal node of the hierarchy. However, a message is not in the
hierarchical namespace; you must open the mailbox first, then use non-name
operations to address the data. There's also no particular reason to call
a message a terminal node rather than a particular body part; the
"dual-use" model is based entirely on the mh/netnews/maildir form of
store.
Those of us who dislike the "dual-use" model point to the fact that there
is no single "open" method for a "dual-use" mailbox. You need to have
separate "open as mailbox" and "open as directory" methods. Whether or
not you accept this argument depends upon whether or not you consider a
single open method to be important.
A better analogy of a dual-use mailbox is the two-fork files of the old
Macintosh operating system, where a name could have both a data and a
resource fork. Even then, a name could not be a folder (which is the
correct term in the Mac world) and have data/resource forks; there was
always a single open method.
>>> Really there ought to be a CAPABILITY string or a NAMESPACE extension
>>> string (or both) explaining what the hierarchy is actually capable of.
>> That is the wrong granularity, since the state is per-mailbox and it is
>> perfectly possible for a mail store hierarchy to have multiple types of
>> mailbox objects.
> *nods* True. But in some cases, the behaviour is known on a
> per-NAMESPACE or per-server basis.
Alas, not on the namespaces and servers where that knowledge matters most
to the client.
Mark Crispin
informative and productive discussion, with a signal to noise ratio that
is unusually high for netnews. :-)
Jorey Bump
> Your mileage may vary, but I find it remarkable that people expect
> that a mailbox should contain other mailboxes. I haven't yet heard a
> groundswell of support for C language source files to serve a second
> purpose as a container of other source files.
This betrays your perspective. You are intimately aware with what a mailbox
actually is (and prefer to limit the definition to flat-file types). But a
user is never presented with this representation. To a user, messages are
individually displayed as files, and mailboxes are indistinguishable from
directories in their representation when compared to file system browsers.
Even Pilot shows files and directories coexisting peacefully in the parent
container, so it shouldn't be a surprise that users expect to see this in
Pine, as well (I don't remember how Pine displays file/message formats, I
currently only use mbox/mbx). Users of graphical clients often try to drag
and drop the familiar icons, with unpredictable results. Faced with this
kind of interface, I find it remarkable that people should be expected to
understand that a mailbox is (sometimes) simply a single text file.
Besides, isn't it true that, in a sense, most source files actually *do*
serve as containers for other source files (#include, import, require, use,
open, etc.)?
Jeffrey Goldberg
> Worse, the "be liberal" part of
> the robustness principle has been twisted to mean something entirely
> different: the notion that if it looks something like what the protocol
> specifies, and if a human could figure out what was intended, then it
> should be interpreted "correctly" even though it's invalid under the
> protocol.
That, of course, is exactly what I was talking about. The robustness
principle continues to make sense *within the protocol*, but now it is
used to justify accepting protocol violations.
> The result, typified by HTML, is almost the exact opposite of what the
> robustness principle intended: rather than being a subset of the
> published protocol, the actual protocol in operational use becomes an
> unpublished *superset* of the published protocol.
Exactly. And for a time, to a much lesser degree, sendmail's behavior
was the unpublished superset of 821. With HTML there are several
competing unpublished supersets of the language.
-j
Jeffrey Goldberg
> Jeffrey, everything you wrote is 100% factually correct, and I agree with
> it completely. Yes, I am a web developer (among other things), and I
> religiously refuse to do any browser detection.
I'm glad to hear that. I had misunderstood what you'd been saying, and
I sure that you understand why that misunderstand would send me off on a
long winded rant.
> But the User-Agent string has been a clue to other, more serious,
> problems, such as the poor handling of SSL, which simply can't be
> tolerated in many circumstances.
More on that later.
> Users and PHBs alike bring pressure to bear on web developers ("My
> son at college says the page looks different in Firefox 1.5 beta than
> it does on my version of Internet Explorer 4 -- fix it!"). This has
> put web developers in unique position to deflect that pressure onto
> browser developers, and it seems to have had some effect.
When you say "looks different" do you mean the behavior that was
deliberately designed into HTML (and which many PHB's, users, and some
web developers don't understand), or are you referring to something
which is more of a problem?
> Yes, browsers have always seemed to support broken HTML, allowing for
> some pretty sloppy coding. But now they also look at the DOCSTRING
> (gasp!) and offer some strict modes that never existed before.
I love it. Now, at least, I can tell the client which of the several
published standards my HTML is written in. But that doesn't change the
fact that many web developers continue to design for competing
unpublished standards.
> At one
> time, most web developers started out as content providers, and it was
> rare to find one that was code-savvy (I had 10 years of programming
> experience before I saw HTML, and laughed/cursed myself silly at it).
I know the feeling.
> That's changed quite a bit, but there are times when I ask myself why a
> content provider should even be concerned with the arcane technical
> details that are necessary to create a valid page that doesn't even meet
> some reasonable expectations.
I see the problem as an instance of the failure of structural mark-up to
catch on with content providers. Whatever the reasons, structural
mark-up is simply not the way that content providers want to think about
things, even when they are handed tools that allow them to do structural
mark-up without having to know the innards of a language. If you give a
typical content provider a choice between using "strong" or "bold", they
will pick "bold" every time.
> But I am also a mail server administrator, and sorely miss the single
> interface, zero configuration benefits inherent in HTTP user
> interfaces. There are only a few basic parameters necessary to access
> one's mail (IMAP/POP host, SMTP host, login, password, SSL/TLS), but
> can you name a single graphical client that puts all of these
> settings in one simple screen or dialog box?
Fair point. For a while I thought that Apple's Mail.app did ok with
that, but then I looked over the shoulder of someone who isn't a
mail-admin setting up an account. Users do need the assistance of
people with some expertise to set up their mail clients. They don't
need that for webmail. It is an enormous difference.
> Or any that let you check mail on the fly without
> setting up a permanent account? I won't go into the administrative
> backend issues, because I understand and expect there to be some
> complexity here, especially as the technology is still evolving. But
> changes at the server should be transparent to end users, who will
> ideally share a common experience. Mail protocols are hardly in their
> infancy, so why has the HTTP user experience *seemed* to outstrip any
> email UI available today, in terms of consistency and ease of use?
I think that the age of the mail protocols is the answer to that riddle.
When the protocols, clients, and servers for mail were being developed
and deployed, these were on multi-user machines. An admin could say to
all of the users, "Pine is what you use for reading mail on our system",
and often with a centrally administered site default configuration.
Users could use different clients if they wished (depending on the
site), but in those cases it was the deviant users' responsibility to
configure their client.
But HTTP really took off in a different environment. Yes there was a
time we had a whole bunch of clients on Windows 3.11 running Mosaic
through the Hummingbird X11 server, and we had better central control,
but those days didn't last very long. So web clients developed in an
environment in which there wasn't a specialist fixing (in both senses of
the word) the client configurations.
If mail were to start now, then most client configuration issues would
be done through something like DHCP. But of course we are no longer in
a world where one's access provider (and so likely DHCP server admin) is
going to be your email service provider.
> But I'm also not
> sure that the strictness of mail protocols has contributed to user-
> friendliness as much as it has to USENET flamewars.
I guess that I see this as a sort of slipper slope, and so (to mix
metaphors) wish to nip it in the bud. That is, if we hadn't been
fighting to keep the protocols adhered to, we would have ended up in a
much worse place. The story of HTML does provide a frightening lesson,
if that leads some people (like me) to over-react, that does help
provide a needed corrective.
> With the widespread
> deployment of SMTP AUTH and STARTTLS, we can now move beyond some
> important client issues (for example, who cares if a properly
> authenticated Outlook sends an invalid HELO/EHLO?).
I never really understood what the point of the HELO argument was
anyway. We only started using it as some kind of weak authentication
check in the late 1990s. My example wasn't to show that the utility of
HELO was being undermined, but how inadvertent liberality in what is
accepted can lead to compatibility problems later. I'm sure that
sendmail developers didn't sit down and thing, "oh, we should allow
underscores in domain names, even though the standard says otherwise
just to be liberal." It simply was the way that it happened to be
coded. Likewise the Outhouse developers weren't saying, "Oh, let's
violate the standards in our translation of spaces in hostnames."
Instead they did what happened to work with existing MTAs. But once the
problem became apparent (though other MTAs that didn't happen to have
the same particular liberality) then it became a question. I could tell
our Win98 users to change their machine names to something without
spaces (which I did), or I could have configured exim to accept
underscores in the HELO hostname. If I and others had let that slide,
we'd now have a larger unpublished superset of the SMTP protocol on our
hands. It was a tough choice at the time, and I don't know how most
people felt about it. Those who spoke up mostly didn't liberalize their
MTAs. But I'm sure that plenty others did, but just kept quiet about it
given the self-righteousness of people like me.
Anyway with sendmail and outhouse, it was a case of unintended
liberalization and dependence on the liberalization. But even that
caused compatibility problems down the road. Once you have deliberate
acceptance of a superset of the specification, things get much worse
much faster than the trivial case with HELO.
> So I guess the question is how do we get to a more unified
> client/server & server/server communication interface? Is the answer
> some new layer that sits on top of HTTP, requiring only a URI (using
> exisiting HTTP authentication mechanisms and SSL for encryption)?
I hadn't really thought about this until this discussion. I don't know.
-j
Dave Cridland
Mark Crispin wrote:
> On Mon, 10 Oct 2005, Dave Cridland wrote:
> >> True, but do you really want to thread 100K messages from a mobile client?
> > Why not? Polymer already handles 100K message mailboxes perfectly well
> > over low-bandwidth connections, even in a cold-cache situation. Why on
> > earth wouldn't I want to thread them too?
>
> As does Pine; but the issue isn't a matter of software capability. It's a
> matter of user managability. Most users who have that many messages (and
> these are typically archived messages, not new messages) break them up
> into separate mailboxes, usually named in some way to categorize the
> archive (date range is popular, as is topic).
>
I maintain that's learned behaviour, based on the inability of many
IMAP clients (and servers) to scale reasonably well. I'm also not
convinced that "most" is even accurate.
> Yes, you can do searches instead. But do you really want to scroll
> through 100K messages, and enter in searches through 100K messages, on a
> mobile client?
>
Well, I'm certainly not suggesting that a user of a mobile client is
likely to want to scroll through all 100k messages, but equally, you're
implying that organization of email into "chunks" by mailbox is a
workaround for the capabilities and user interfaces of clients.
> > (I don't see any problem with
> > handling much higher message counts, either, I just can't be bothered
> > to build such a mailbox.)
>
> OK, so your "can't be bothered" limit is at 100K. Very few people are
> there. Most drop out after 10K, with the peak being in the 4 digit range.
>
No, 100k is my "can't be bothered" limit for a test case. I suppose I
could just COPY 1:* to the same mailbox a few times, but that gives me
a very dull test mailbox.
> >>> As it stands, I think THREAD is too expensive for mobile clients,
> >>> useless for disconnected mode clients, and only useful for client which
> >>> aim to provide a snapshot view of a mailbox - basically meaning webmail
> >>> clients.
> >> Huh? THREAD works quite well for online clients!
> > Oh, across a LAN, yes.
>
> Or wi-fi.
>
Or indeed high bandwidth links in general.
> I think that the GPRS/1xRTT guys are rapidly losing their window of
> opportunity. At about the same time that I finally dropped CDPD in favor
> of 1xRTT, I observed that my need for WAN wireless had been plummeting
> like a paralyzed falcon. So far, I've used WAN wireless *once* this
> entire year.
>
I use it quite a bit, strangely - I find that the cost of WiFi access
points is quite high, and moreover they're not all that common here, in
the wilds of Wales.
Where GPRS wins out isn't so much in the laptop market, although that's
still important, but in the "email enabled mobile phone" market.
> > Perhaps it'd be fairer if I simply said that both SORT and THREAD lack
> > the level of scalability found elsewhere in the protocol.
>
> That claim has been made many times. I disagree. It depends upon how you
> look at it. I consider SORT and THREAD to be a form of SEARCH. If you
> need the sequence/UID map to resynchronize, UID SEARCH ALL is the fastest
> way to do that.
>
Actually, no. Polymer does use UID SEARCH for sequence/UID mappings,
but does so in small chunks, using a successive approximation technique
to find out which chunk most likely holds the UID it wants.
> A more credible claim is that it's too costly to reTHREAD when new
> messages arrive. That's true; but nobody has yet come up with a good way
> to represent a delta thread.
>
Yes, it's a tricky problem.
> It also needs to be pointed out that SORT and THREAD scale much better
> than the previous methods. Note that the NNTP community, which threads
> more than anyone else, still depends upon downloading overviews for all
> articles!
>
Oh, there's no argument there. It just doesn't scale well enough, yet.
> > Polymer gets regularly tested against a mailbox with nearly 130,000
> > messages in.
>
> Tested, yes; but what about usage?
>
> There is a reason that users limit the number of messages in a single
> mailbox at a point much lower than their MUA is capable of handling. I
> don't think that this reason has received adequate study.
>
> Hierarchies of categorization work better for most humans than a flat
> space (even with excellent searching tools).
>
The OSAF have done some work in that area. I'm not sure how much
academic rigour has gone into it, but certainly the rise of multiple
hierarchy models, tagging, and attribute matching, does appear to
suggest that while a single taxonomy - a folder tree - works well for
technical types, this is a learned skill much harder to acquire than
simple tagging, which has, after all, been present in IMAP for over a
decade.
> > FWIW, I regularly use Polymer on mailboxes with 70k messages in, as do
> > some of the other users. I do this reasonably often over GPRS, so I'm
> > putting my money where my code is, as it were.
>
> Well, yes. A reasonable IMAP client should work well with a 70K message
> mailbox over GPRS, even if you start with a zero-state cache (pure online
> client). I designed IMAP to work well over 2400 bps, which is what much
> of my early work was over.
>
I know - and indeed you know - at least one person who regularly reads
email over a 9600bps connection, in fact. He's using the free minutes
on his mobile to avoid bothering with a DSL line.
> > Well, my personal target is 2400 baud. If I can get Polymer usable over
> > that I'll be reasonably happy.
>
> Yup, that's what I did. The problem is, it's getting very difficult to
> justify forcing people to develop for that low a bandwidth, as it's
> getting harder and harder to find places which are that low.
>
> Of course, a poorly-performing wireless WAN can have an effective speed
> that is that low... :-)
>
I'm not thinking that anyone's ever likely to use Polymer, or its
underlying library, over 2400bps.
However, I figure that if it's usable over 2400bps, then using it over
9600bps - a GSM data call, about the most reliably obtainable bandwidth
- is going to be painless, and using it on GPRS is going to be
excellent.
> > Well, yes, PINE's handling of IMAP is very good - as I'd expect. But
> > configuration and addressbook handling is weak, in my opinion, both in
> > terms of functionality and expressiveness. That's not meant as an
> > attack on PINE, just a demonstration that a "poor man's" method, while
> > sometimes "good enough", isn't "as good as it can be".
>
> Understood. It is impossible in human endeavor to produce anything that
> some other person can't produce something better. I personally consider
> Pine to be an overall "best" among today's offerings, but that's because I
> give almost zero value to GUI. If I'm using a small screen, GUI objects
> take away valuable real estate that can present text. What's more, none
> of the GUIs that I've seen have been visually compelling.
>
I have been toying with the idea of a TTY variant. Oddly enough, my
biggiest sticking point is a name. :-)
> > As an aside, the problem with 14k4 modems wasn't the amount of data
> > transferred, per-se, but the time spent online, at least in the UK.
> > Polymer is asbolutely not designed to minimize the time spent online.
> > It's designed for circumstances where you're charged by transfer.
>
> I suspect that both pricing models (time online or by transfer) are doomed
> in the face of inexpensive flat-rate.
>
No, by-transfer is showing a marked increase in the UK. GPRS is charged
entirely by transfer, and some DSL models charge by transfer over a
certain limit.
> > Everyone liked, and still appears to like, the notion of ACAP. It's
> > just nobody actually wanted to take the plunge and actually use it -
> > just one conformant server and four MUAs, two of which are commercial
> > including Mulberry.
>
> Agreed. Sadly, I feel that ACAP attempted to bite off too much. That
> last 2% is invariably 98% of the work.
>
MAKECONTEXT with DEPTH is the tricky one. That and by-class/byowner
namespace duality. The rest is actually quite easy. Even making it
scalable is, essentially, easy, I just haven't had the time. (Oh, I
tell a lie - it's quite tricky figure out which datasets you need to
hold a lock on for a STORE.)
> > LDAP I'm not so certain has won the addressbook war - I think it's
> > managed a successful land grab, but failed to consolidate. It's used
> > heavily for canonical lists of address data, such as global
> > addressbooks and actual directories, but there seems little motion
> > toward using it as a personal addressbook. Moreover, I don't think
> > anyone's seriously considering accessing LDAP over a mobile link.
>
> The consolidation (and especialy use for personal addressbooks) is going
> slower than I predicted, but the lumbering dinosaur is in motion. I'm
> going to attend a dinosaur-pushing meeting shortly (and hope that I don't
> get buried under dino-dung...) And yes, people are talking about using it
> over mobile.
>
Really? Randall Gellens has some interesting data about that, relating
to OTAPA.
> > Yes, I suppose it'd be cleaner to have "create submailbox" and "create
> > subcontainer" or something.
>
> Or just follow IMAP's lead, in which you create mailboxes and directories
> are created implicitly.
>
Yes, true, but that's a pain to do in a GUI as well.
> >> Your mileage may vary, but I find it remarkable that people expect that a
> >> mailbox should contain other mailboxes. I haven't yet heard a groundswell
> >> of support for C language source files to serve a second purpose as a
> >> container of other source files.
> > I like the analogy, forced as it is, but remarkable or not, and even
> > expectation or not, some servers allow this, some don't, and there's no
> > easy way for a client to know which short of "suck it and see", which
> > doesn't strike me as a clean methodology.
>
> I actually agree; well, not about the analogy being "forced", but
> everything else you said. IMAP failed to take a stand and instead tried
> to forge a wretched compromise. The two sides of the time have come to
> accept that it would have been better to have lost in favor of the other
> side.
>
> The reason why I don't think the analogy is forced is that "dual-use"
> mailboxes require the assumption that a message (rather than the mailbox)
> is the terminal node of the hierarchy. However, a message is not in the
> hierarchical namespace; you must open the mailbox first, then use non-name
> operations to address the data. There's also no particular reason to call
> a message a terminal node rather than a particular body part; the
> "dual-use" model is based entirely on the mh/netnews/maildir form of
> store.
Yes, sure. But IMAP allows dual-use to happen, and yet cannot signal
the client as to what is going to happen.
C: A01 CREATE foo/
S: A01 OK I did something, now guess what.
> >>> Really there ought to be a CAPABILITY string or a NAMESPACE extension
> >>> string (or both) explaining what the hierarchy is actually capable of.
> >> That is the wrong granularity, since the state is per-mailbox and it is
> >> perfectly possible for a mail store hierarchy to have multiple types of
> >> mailbox objects.
> > *nods* True. But in some cases, the behaviour is known on a
> > per-NAMESPACE or per-server basis.
>
> Alas, not on the namespaces and servers where that knowledge matters most
> to the client.
Hmmm.
C: A01 CREATE foo ((TYPE DUAL))
S: A01 NO Cannot create dual-use mailboxes.
But in any case, for servers that only support either all-dual or
no-dual, a CAPABILITY string or NAMESPACE string would avoid the wasted
command.
Dave.
Mark Crispin
> I never really understood what the point of the HELO argument was anyway. We
> only started using it as some kind of weak authentication check in the late
> 1990s.
The purpose of HELO (and the Received: header line) was to fix a problem
that went away with the NCP->TCP transition.
NCP had no addressing in the host-host protocol; hosts were addressed
entirely by what we now call the MAC address. What's more, hosts on the
ARPAnet were connected to a smart switch, called an IMP, which actually
did the routing.
The host-IMP protocol was a mirror image of the IMP-host protocol; and
there was only a single host field. What this meant is that an IMP-host
packet of "from host A" was identical to a host-IMP packet of "to host A".
A loopback on the IMP port for a host would therefore cause all packets
sent to that host address to be sent to the originating host as if it came
from the destination host.
The NCP mail protocol was a command in FTP that only had the destination
(no source) and only the local part (no domain).
I think that you're starting to see what the problem was.
Suppose mail to "fred" on host A was forwarded to host B. But if host B
was currently in loopback, the mail would go right back to host A, with no
way for host A to know that it is talking to itself.
The replacement of NCP with TCP/IP was enough to fix the problem, but
people felt strongly about making this never happen again. Hence the
creation of SMTP to replace FTP-based mail. SMTP added:
. the SMTP client identified itself (HELO), and you were allowed to barf
if the HELO claimed to be yourself since that meant that the network
was in loopback.
. there was a return path (MAIL FROM) to identify origin, rather than
obliging the MTA to parse the headers to send a bounce.
. the destination had a domain (RCPT TO), again to prevent loops; but
also to allow relaying (which at that time was a feature not a bug).
Times have changed quite a bit in the 22+ years since NCP and FTP-based
mail went away!
> My example wasn't to show that the utility of HELO was being
> undermined, but how inadvertent liberality in what is accepted can lead to
> compatibility problems later.
It is a good example.
Mark Crispin
> I use it quite a bit, strangely - I find that the cost of WiFi access
> points is quite high, and moreover they're not all that common here, in
> the wilds of Wales.
Yes, but for how long will that remain the case?
Unless the UK has government regulation to hinder the deployment of WiFi,
WiFi is going to win.
> Where GPRS wins out isn't so much in the laptop market, although that's
> still important, but in the "email enabled mobile phone" market.
Yes, but...
> No, by-transfer is showing a marked increase in the UK. GPRS is charged
> entirely by transfer, and some DSL models charge by transfer over a
> certain limit.
I doubt that UK GPRS providers will enjoy much success with a usage-based
charging model. It'll collapse the moment one of them breaks rank and
offers flat-rate. The only question is whether it will happen fast enough
to save GPRS from the WiFi onslaught.
Yiorgos Adamopoulos
> I have been toying with the idea of a TTY variant. Oddly enough, my
> biggiest sticking point is a name. :-)
monomer?
Dave Cridland
Jeffrey Goldberg wrote:
> Jorey Bump wrote:
> > But I am also a mail server administrator, and sorely miss the single
> > interface, zero configuration benefits inherent in HTTP user
> > interfaces. There are only a few basic parameters necessary to access
> > one's mail (IMAP/POP host, SMTP host, login, password, SSL/TLS), but
> > can you name a single graphical client that puts all of these
> > settings in one simple screen or dialog box?
>
> Fair point. For a while I thought that Apple's Mail.app did ok with
> that, but then I looked over the shoulder of someone who isn't a
> mail-admin setting up an account. Users do need the assistance of
> people with some expertise to set up their mail clients. They don't
> need that for webmail. It is an enormous difference.
No, they absolutely do need a mail administrator to setup their
webmail, it just happens that it can be done remotely, without the user
really being aware, and in fact, webmail pretty much has to be
preconfigured by the administrator.
It's similar with Polymer - an administrator can quite easily setup
Polymer without the user being aware, or present. In principle, the
administrator needn't even be aware that it's Polymer being used,
however, in practise, Polymer remains the only MUA which uses these
specifications. In Polymer's case, it doesn't require preconfiguration,
though. (It does, however, ask for all the basic mail settings in two
dialogs).
Of course, I don't do that on my ACAP server for everyone, because I
don't know the details, but perhaps I should preconfigure some
anonymous access IMAP servers, and suchlike, for demonstration
purposes.
Dave.
Jeffrey Goldberg
> Jeffrey Goldberg wrote:
>> Users do need the assistance of
>> people with some expertise to set up their mail clients. They don't
>> need that for webmail. It is an enormous difference.
> No, they absolutely do need a mail administrator to setup their
> webmail, it just happens that it can be done remotely, without the user
> really being aware, and in fact, webmail pretty much has to be
> preconfigured by the administrator.
I was talking about from the user's point of view.
> In principle, the
> administrator needn't even be aware that it's Polymer being used,
> however, in practise, Polymer remains the only MUA which uses these
> specifications.
I have to confess that I'd entirely forgotten about ACAP until this
discussion. Yes, that is a way forward.
> In Polymer's case, it doesn't require preconfiguration,
> though. (It does, however, ask for all the basic mail settings in two
> dialogs).
And how do novice users do when completely unassisted in that task?
Some of them can follow the instructions, typically given in MUA
specific versions from their email providers. But I think that you'll
agree this is still a hurdle for many users in a way that setting up a
webmail account isn't.
Is Polymer exclusively an IMAP client? If so, then if a user happens to
have a POP-only account some place then users can't have all their mail
handled by one MUA. And if it does do POP then the IMAP vs POP question
will appear in the dialogue (unless Polymer does some probing of the
server). That is question leads to a "what the hell is this?" response
from novice users. I guess I should finish installing polymer myself
and test it out to see how those dialogues are handled.
> Of course, I don't do that on my ACAP server for everyone, because I
> don't know the details,
Exactly. Usually only the user has the details, but an insufficient
understanding to have all of the answers.
-j
kael
> Mark Crispin wrote:
>
>>There is a reason that users limit the number of messages in a single
>>mailbox at a point much lower than their MUA is capable of handling. I
>>don't think that this reason has received adequate study.
>>
>>Hierarchies of categorization work better for most humans than a flat
>>space (even with excellent searching tools).
The problem is that there is no efficient tool for SEARCHing in several
mailboxes in the IMAP protocol.
Using KEYWORDs with few mailboxes is an alternative way to organise
emails - but the problem seem to be scalability.
With Polymer I use two mailboxes, one for personal mails and the other
for mailing-lists, each list is tagged (thanks to the SIEVE IMAP flags
extension) with at least two KEYWORDs and I use views (based on the
SEARCH syntax) to filter mails by KEYWORD. There are 70,000 mails in
this mailbox and I'm wondering what level it can reach - and more
generally with SORT and THREAD.
It may be interesting to have a _Virtual Mailboxes extension_ to allow
SEARCHes in a tree and more generally to create permanent user-defined
VIEWs on an IMAP tree. This extension would be very useful in mobile
environment.
The MSGFILTER draft
<http://ietfreport.isoc.org/idref/draft-wener-lemonade-msgfilter> seemed
interesting but has expired unfortunately.
> The OSAF have done some work in that area. I'm not sure how much
> academic rigour has gone into it, but certainly the rise of multiple
> hierarchy models, tagging, and attribute matching, does appear to
> suggest that while a single taxonomy - a folder tree - works well for
> technical types, this is a learned skill much harder to acquire than
> simple tagging, which has, after all, been present in IMAP for over a
> decade.
Tagging doesn't exclude a folder tree necessarily, although tags have
been used into flat spaces. Tagging also allowed the emergence of
folksonomy, what is called a /bottom-up/ (sic) taxonomy.
Tags are a way to highlight a specific metadata among others. This
metadata can be an original or an ad hoc one.
Sharing tags is another aspect of the folksonomy and the tagging
experience. Sharing KEYWORDs via shared mailboxes can be very useful,
specially for archives.
But there is no supply - as usual with IMAP - of UIs for the STORE
command and the FLAGS and PERMANENTFLAGS responses. As far as I know,
Polymer is the most advanced MUA as regards those capabilities.
--
kael
those who know me have no need of my name
>Mark Crispin wrote:
>> On Fri, 7 Oct 2005, Dave Cridland wrote:
>>> Note that the last entry is "Other Servers" -
>>> I've not put UW-IMAP on that list, because it
>>> happens that I don't have an account on such a server.
>>
>> Can't you install it on your own system?
>
>Only when I find a spare system. :-) All my servers have IMAP
>implementations on them at the moment.
uw-imap works fine on a port other than 143. if you happen to choose 993
you'll need to start ssl/tls negotiation immediately.
/etc/services:
uwimap 144/tcp
/etc/inetd.conf:
uwimap stream tcp nowait root /path/to/imapd imapd
/etc/xinetd.d/uwimap:
service uwimap
{
socket_type = stream
wait = no
user = root
server = /path/to/imapd
}
--
a signature
Dave Cridland
> Dave Cridland wrote:
> > Jeffrey Goldberg wrote:
>
> >> Users do need the assistance of
> >> people with some expertise to set up their mail clients. They don't
> >> need that for webmail. It is an enormous difference.
>
>
> > No, they absolutely do need a mail administrator to setup their
> > webmail, it just happens that it can be done remotely, without the user
> > really being aware, and in fact, webmail pretty much has to be
> > preconfigured by the administrator.
>
> I was talking about from the user's point of view.
>
Sure. But webmail doesn't hold a monopoly on this, nor is there
anything intrinsic about webmail which makes it impossible to do
elsewhere.
> > In principle, the
> > administrator needn't even be aware that it's Polymer being used,
> > however, in practise, Polymer remains the only MUA which uses these
> > specifications.
>
> I have to confess that I'd entirely forgotten about ACAP until this
> discussion. Yes, that is a way forward.
>
Don't worry - *everyone* forgot about ACAP. :-)
>
> > In Polymer's case, it doesn't require preconfiguration,
> > though. (It does, however, ask for all the basic mail settings in two
> > dialogs).
>
> And how do novice users do when completely unassisted in that task?
> Some of them can follow the instructions, typically given in MUA
> specific versions from their email providers. But I think that you'll
> agree this is still a hurdle for many users in a way that setting up a
> webmail account isn't.
>
No, I don't.
The sole reason that webmail doesn't have this problem is because it's
impossible to deploy webmail without preconfiguring it.
> Is Polymer exclusively an IMAP client? If so, then if a user happens to
> have a POP-only account some place then users can't have all their mail
> handled by one MUA. And if it does do POP then the IMAP vs POP question
> will appear in the dialogue (unless Polymer does some probing of the
> server). That is question leads to a "what the hell is this?" response
> from novice users. I guess I should finish installing polymer myself
> and test it out to see how those dialogues are handled.
>
Polymer doesn't probe, which is rather poor, and I do intend correcting
that. Polymer only handles IMAP, ESMTP, ACAP, and Managesieve,
protocol-wise.
> > Of course, I don't do that on my ACAP server for everyone, because I
> > don't know the details,
>
> Exactly. Usually only the user has the details, but an insufficient
> understanding to have all of the answers.
Sure. There's not much that can be done to simplify the user's
first-time config. There's been suggestions to use SRV records, and
that might well help, but the most obvious solution is
preconfiguration.
Dave.
kael
> It may be interesting to have a _Virtual Mailboxes extension_ to allow
> SEARCHes in a tree and more generally to create permanent user-defined
> VIEWs on an IMAP tree. This extension would be very useful in mobile
> environment.
There's an amazing draft I discovered via
<imap://cyrus.watson.org/lists.ietf.announce?SUBJECT%20imap>:
_IMAP CREATE/RENAME parameters_
<http://www.ietf.org/internet-drafts/draft-melnikov-imap-createparams-01.txt>
which presages of very interesting extensions (like virtual mailboxes):
"2.2. TYPE parameter to CREATE command
[...]
This document defines the following initial set of mailbox types:
- "CONTACT" - can contain MIME messages containing text/plain (?)
and vCARD
- "CALENDAR" - can contain vCALENDAR objects as described in <<>>
- "VOICE"
- "IMAGE"
- "VIDEO"
- "JUNK" - messages identified by the user as junk. Messages MAY
be deleted automatically from such mailbox after some period of
time.
- "VIEW" - "virtual mailbox", created using a persistent mailbox
search mechanism.
[...]
2.3. SHAREDFLAGS parameter to CREATE command
[...]
SHAREDFLAGS parameter allows to specify which system flags and
user defined keywords should be shared for the mailbox. It also
allows to "precreate" some user defined keyword".
--
kael
David Magda
> With Polymer I use two mailboxes, one for personal mails and the
> other for mailing-lists, each list is tagged (thanks to the SIEVE
> IMAP flags extension) with at least two KEYWORDs and I use views
> (based on the SEARCH syntax) to filter mails by KEYWORD. There are
> 70,000 mails in this mailbox and I'm wondering what level it can
> reach - and more generally with SORT and THREAD.
Just curious, but which IMAP server are you using?
--
David Magda <dmagda at ee.ryerson.ca>
Because the innovator has for enemies all those who have done well under
the old conditions, and lukewarm defenders in those who may do well
under the new. -- Niccolo Machiavelli, _The Prince_, Chapter VI
kael
> kael <ka...@alussinan.org> writes:
>
>>With Polymer I use two mailboxes, one for personal mails and the
>>other for mailing-lists, each list is tagged (thanks to the SIEVE
>>IMAP flags extension) with at least two KEYWORDs and I use views
>>(based on the SEARCH syntax) to filter mails by KEYWORD. There are
>>70,000 mails in this mailbox and I'm wondering what level it can
>>reach - and more generally with SORT and THREAD.
>
> Just curious, but which IMAP server are you using?
I use FastMail which runs a Cyrus server.
--
kael