Syntax to access mail folders on a remote machine; problem making secure connection
Adam H. Kerman
I check an inbox via IMAP on a remote host. That inbox is in Maildir
format (separate file for each message). I have messages stored in files
that are still in Berkeley Mail format in a directory named mail on that
remote machine. I cannot figure out the syntax to use to access these
files. {example.com/user=someuser}~someuser/mail/[] doesn't cut it. I
receive a truncated error message that on a Maildir++ format, all folders
must be I (and that's it). Is there a restriction that all mail folders on
one host, whether for an inbox or saved messages, must be in one format?
Also, I'm unable to make a secure connection to that remote machine when
checking the remote inbox. Keeps insisting I add "notls". The Help text
says that Pine must be linked to a library to use secure connections. Is
this a compile-time option, or an option that can be set after Pine has
been compiled?
Mark Crispin
> I check an inbox via IMAP on a remote host. That inbox is in Maildir
> format (separate file for each message). I have messages stored in files
> that are still in Berkeley Mail format in a directory named mail on that
> remote machine. I cannot figure out the syntax to use to access these
> files. {example.com/user=someuser}~someuser/mail/[] doesn't cut it. I
> receive a truncated error message that on a Maildir++ format, all folders
> must be I (and that's it). Is there a restriction that all mail folders on
> one host, whether for an inbox or saved messages, must be in one format?
You will need to contact the system management of your IMAP server for
help on this. Only then can tell if your if your IMAP server supports
traditional mailbox format, and if so how to reference them by name.
> Also, I'm unable to make a secure connection to that remote machine when
> checking the remote inbox. Keeps insisting I add "notls". The Help text
> says that Pine must be linked to a library to use secure connections. Is
> this a compile-time option, or an option that can be set after Pine has
> been compiled?
What is the EXACT error message that you got? If Pine is telling you that
you have to use /notls, it supports secure connections. Either the server
does not, or the server's certificate isn't right. But without the exact
text of the error message, I can't determine the problem.
-- Mark --
http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
Adam H. Kerman
>On Sat, 21 May 2005, Adam H. Kerman wrote:
>>I check an inbox via IMAP on a remote host. That inbox is in Maildir
>>format (separate file for each message). I have messages stored in files
>>that are still in Berkeley Mail format in a directory named mail on that
>>remote machine. I cannot figure out the syntax to use to access these
>>files. {example.com/user=someuser}~someuser/mail/[] doesn't cut it. I
>>receive a truncated error message that on a Maildir++ format, all
>>folders must be I (and that's it). Is there a restriction that all mail
>>folders on one host, whether for an inbox or saved messages, must be in
>>one format?
>You will need to contact the system management of your IMAP server for
>help on this. Only then can tell if your if your IMAP server supports
>traditional mailbox format, and if so how to reference them by name.
Hm. Where do I tell him to look for the IMAP configuration option? He uses
kind of a minimalist Linux system, with (ugh) qmail as the mailer.
>>Also, I'm unable to make a secure connection to that remote machine when
>>checking the remote inbox. Keeps insisting I add "notls". The Help text
>>says that Pine must be linked to a library to use secure connections. Is
>>this a compile-time option, or an option that can be set after Pine has
>>been compiled?
>What is the EXACT error message that you got? If Pine is telling you
>that you have to use /notls, it supports secure connections. Either the
>server does not, or the server's certificate isn't right.
Why would the server give me a different certificate when I use IMAP than
when I use ssh/telnet? Pine wouldn't store its own key somewhere else,
would it? If it does, can I just delete it and get a fresh key?
>But without the exact text of the error message, I can't determine the
>problem.
I commented out the entry in my .pinerc, and re-added the remote inbox
from the folder list screen. During the add, I received this error:
There was an SSL/TLS failure for the server
The reason for the failure was
SSL negotiation failed
This is just an informational message. With the current setup, SSL/TLS
will not work. If this error re-occurs every time you run Pine, your
current setup is not compatible with the configuration of your mail
server. You may want to add the option
/notls
to the name of the mail server you are attempting to access. In other
words, wherever you see the characters
in your configuration, replace those characters with
Type RETURN to continue.
Mark Crispin
>> You will need to contact the system management of your IMAP server for
>> help on this. Only then can tell if your if your IMAP server supports
>> traditional mailbox format, and if so how to reference them by name.
> Hm. Where do I tell him to look for the IMAP configuration option? He uses
> kind of a minimalist Linux system, with (ugh) qmail as the mailer.
There are IMAP servers which work with qmail and Maildir format, but I can
not speak authoritatively about the function of any of these.
> Why would the server give me a different certificate when I use IMAP than
> when I use ssh/telnet?
ssh is not SSL/TLS. ssh is a completely separate mechanism. ssh does not
use certificates; it uses host keys.
A certificate is a signed statement, either self-signed or issued by a
certificate authority (CA), that the server is in fact the entity it
claims to be.
Certificates are used by POP, IMAP, HHTP (web browsers),...basically
everything except for ssh.
> Pine wouldn't store its own key somewhere else,
> would it? If it does, can I just delete it and get a fresh key?
You're thinking about host keys, which are used with ssh. SSL/TLS is a
different mechanism.
> There was an SSL/TLS failure for the server
> example.com
> The reason for the failure was
> SSL negotiation failed
I assume that you substituted "example.com" for the real name. There's
not much point to concealing the real name, and doing so just means that I
can not directly verify the problem.
"SSL negotiation failed" means that the IMAP server didn't negotiate SSL
or TLS encryption at all, even though it claimed that it could do so.
Generally, this is because no certificate is installed, but you would have
to ask the system manager of the site. If he doesn't have the slightest
clue of what you're talking about, that pretty much confirms that there is
no certificate installed.
Adam H. Kerman
>On Sat, 21 May 2005, Adam H. Kerman wrote:
>>Why would the server give me a different certificate when I use IMAP than
>>when I use ssh/telnet?
>ssh is not SSL/TLS. ssh is a completely separate mechanism. ssh does
>not use certificates; it uses host keys.
My error; assumed IMAP used the same method of authentification as ssh.
Thank you for clarifying that.
>>There was an SSL/TLS failure for the server
>> example.com
>>The reason for the failure was
>>SSL negotiation failed
>I assume that you substituted "example.com" for the real name. There's
>not much point to concealing the real name, and doing so just means that
>I can not directly verify the problem.
I'm confused. Is the purpose of the certificate to verify the identity of
the host and it's available to anyone not a user on that host?
What protocol is used during IMAP that avoids sending user and password
information as plain text?
>"SSL negotiation failed" means that the IMAP server didn't negotiate SSL
>or TLS encryption at all, even though it claimed that it could do so.
>Generally, this is because no certificate is installed, but you would
>have to ask the system manager of the site. If he doesn't have the
>slightest clue of what you're talking about, that pretty much confirms
>that there is no certificate installed.
I'll ask. Thank you for the advice.
Mark Crispin
> I'm confused. Is the purpose of the certificate to verify the identity of
> the host and it's available to anyone not a user on that host?
> What protocol is used during IMAP that avoids sending user and password
> information as plain text?
The SSL/TLS mechanism for IMAP is the same as on secure web pages.
Essentially, what is presented is data that says "I am so-and-so" and that
data is digitally signed, either as a self-signed certificate or by a
well-known certificate authority (CA).
CA certificates are much better than self-signed certificates, because in
effect a self-signed certificate is simply someone saying, "I am Joe
Mooch, here's what my signature looks like"; validating that certificate
in the future does nothing more than validate that it's the same signature
as was presented by the guy who claimed to be Joe Mooch.
With a CA certificate, an authority states "I verified that this is the
real Joe Mooch", so the very first time you're handed the signature you
can believe that it's Joe Mooch.
The difference here is that instead of Joe Mooch, it may be amazon.com,
bankofamerica.com, etc.
The signatures of CAs are well-published and registered on all Windows
computers and on many UNIX computers.
After validating the server's signature, you know that you are talking to
the real server and not an imposter. Your session is also encrypted so
that bad guys can not spy on it. In that encrypted session, you send your
password. So the server does get your user name and password as plain
text, unless you use something like Kerberos.
If you use /novalidate-cert, then the session is still encrypted but you
don't try to see if the certificate is authentic. So you can be talking
to a bad guy, and giving your user name and password to the bad guy.
And if you don't use TLS or SSL at all, then anyone can eavesdrop on you
while you're sending your user name and password.
NM Public
> At 8:38am -0700, 05/21/05, Mark Crispin <m...@CAC.Washington.EDU> wrote:
>> On Sat, 21 May 2005, Adam H. Kerman wrote:
>
>>>
>>> Is there a restriction that all mail
>>> folders on one host, whether for an inbox or saved messages, must be in
>>> one format?
>
>> You will need to contact the system management of your IMAP server for
>> help on this. Only then can tell if your if your IMAP server supports
>> traditional mailbox format, and if so how to reference them by name.
>
> Hm. Where do I tell him to look for the IMAP configuration option? He uses
> kind of a minimalist Linux system, with (ugh) qmail as the mailer.
Hm. If you need to tell your system administrator something like
this, you might want to get a better IMAP service provider!
Anyway, to answer your question, no, there is no restriction
about the number of mailbox formats supported by an IMAP server.
For example, Binc IMAP uses the term "IMAPdir" for its IMAP store
and...
"IMAPdir, on the other hand, supports all sorts of things,
including mixing mbox folders in with Maildir folders"
This quote is an excerpt from this Binc mailing-list message:
<http://article.gmane.org/gmane.mail.imap.binc.general/3518>
If you decide you want to get a different IMAP service provider,
check out my IMAP Service Providers page, which is here:
<http://www.ii.com/internet/messaging/imap/isps/>
It includes reviews of more than 400 providers.
Good luck with Pine and IMAP,
Nancy
--
Nancy McGough ~ <http://deflexion.com/>
--> Please keep the discussion in the group <--
Adam H. Kerman
>On 21 May 2005 Adam H. Kerman (a...@chinet.com) wrote:
>>At 8:38am -0700, 05/21/05, Mark Crispin <m...@CAC.Washington.EDU> wrote:
>>>On Sat, 21 May 2005, Adam H. Kerman wrote:
>>>>Is there a restriction that all mail folders on one host, whether for
>>>>an inbox or saved messages, must be in one format?
>>>You will need to contact the system management of your IMAP server for
>>>help on this. Only then can tell if your if your IMAP server supports
>>>traditional mailbox format, and if so how to reference them by name.
>>Hm. Where do I tell him to look for the IMAP configuration option? He
>>uses kind of a minimalist Linux system, with (ugh) qmail as the mailer.
>Hm. If you need to tell your system administrator something like this,
>you might want to get a better IMAP service provider!
He's a friend doing this as a hobby, sharing a T1 with another friend of
his who administers it for his employer, so I won't insult him by
demanding better service. As I don't have a "big pipe" to the Internet
myself, I'm attempting to learn a bit about system administration
vicariously. I'll raise the issue with him, trying to point him toward a
solutions, and see if he's willing to implement it. Yes, I know I'll never
master it if I'm not directly responsible for administering a server.
It's nice of Mark Crispin, who was the main programmer of the first
working implementation of IMAP, to hang around here on this newsgroup
waiting to answer the same questions he's heard a thousand times before. I
appreciate it, and I owe him beers.
Nancy, you've been nice enough to answer users' basic questions over the
years too; you are also owed beers.
btw, I did check the IMAP site looking for information about the
relationship between SSL and TLS and IMAP4, but didn't notice anything
useful. I did find an introduction to these issues in the Wikipedia at
http://en.wikipedia.org/wiki/Secure_Sockets_Layer
>Anyway, to answer your question, no, there is no restriction
>about the number of mailbox formats supported by an IMAP server.
Thank you for the information. I'll ask him what implementation of IMAP
he's running.
Does IMAP need to be told explicitly that there are Berkeley Mail-style
mailboxes as well as Maildir++ boxes? Or can it be concluded that I messed
up the syntax on my end?
Mark Crispin
> It's nice of Mark Crispin, who was the main programmer of the first
> working implementation of IMAP, to hang around here on this newsgroup
> waiting to answer the same questions he's heard a thousand times before. I
> appreciate it, and I owe him beers.
It's part of my job. I hope that I do better at it in the future than I
have in the past.
> Does IMAP need to be told explicitly that there are Berkeley Mail-style
> mailboxes as well as Maildir++ boxes? Or can it be concluded that I messed
> up the syntax on my end?
IMAP is completely neutral on the subject of mailbox formats; this is
entirely a server internal implementation issue. Consequently, only
someone knowledgable about the server's implementation can answer your
question.
In general, servers try to hide these implementation details from clients.
NM Public
>
>>> Hm. Where do I tell him to look for the IMAP configuration option? He
>>> uses kind of a minimalist Linux system, with (ugh) qmail as the mailer.
>
>> Hm. If you need to tell your system administrator something like this,
>> you might want to get a better IMAP service provider!
>
> He's a friend doing this as a hobby, sharing a T1 with another friend of
> his who administers it for his employer, so I won't insult him by
> demanding better service. As I don't have a "big pipe" to the Internet
> myself, I'm attempting to learn a bit about system administration
> vicariously. I'll raise the issue with him, trying to point him toward a
> solutions, and see if he's willing to implement it. Yes, I know I'll never
> master it if I'm not directly responsible for administering a server.
Is this IMAP server used by other people or just by you and your
friend? If it's used by others, my guess is that he's not going
to want to switch to an IMAP server that supports simultaneous
mbox- and maildir-formatted mailboxes (because I think the only
IMAP server that does that (testing version of Binc) is in a
super alpha state). I think what most people do who are in a
situation like you are, is convert all their mbox-formatted
mailboxes to maildir-format and then access them either via IMAP
clients (e.g., Pine) or via clients that can directly access
maildir mailboxes (e.g., Mutt). One way to do the conversion is
to use mailutil, which I've written about here:
<http://www.ii.com/internet/messaging/pine/pc/#mailutil>
> It's nice of Mark Crispin, who was the main programmer of the first
> working implementation of IMAP, to hang around here on this newsgroup
> waiting to answer the same questions he's heard a thousand times before. I
> appreciate it, and I owe him beers.
I appreciate it too and often think of Mark as one of the Unsung
Heroes of the Net.
> Nancy, you've been nice enough to answer users' basic questions over the
> years too; you are also owed beers.
Thank you for the beers Adam, they are appreciated. I do this not
because it's my job, but because it makes me feel good to help
people. And also -- actually mainly -- because I like to help
people switch to standards-based systems (IMAP, Pine, etc.). The
more people who know about and use this stuff, the more it will
spread, at least that is the hope.
Adam H. Kerman
>On 22 May 2005, Adam H. Kerman wrote:
>>>>Hm. Where do I tell him to look for the IMAP configuration option? He
>>>>uses kind of a minimalist Linux system, with (ugh) qmail as the
>>>>mailer.
>>>Hm. If you need to tell your system administrator something like this,
>>>you might want to get a better IMAP service provider!
>>He's a friend doing this as a hobby, sharing a T1 with another friend of
>>his who administers it for his employer, so I won't insult him by
>>demanding better service. As I don't have a "big pipe" to the Internet
>>myself, I'm attempting to learn a bit about system administration
>>vicariously. I'll raise the issue with him, trying to point him toward a
>>solutions, and see if he's willing to implement it. Yes, I know I'll never
>>master it if I'm not directly responsible for administering a server.
>Is this IMAP server used by other people or just by you and your friend?
>If it's used by others, my guess is that he's not going to want to switch
>to an IMAP server that supports simultaneous mbox- and maildir-formatted
>mailboxes (because I think the only IMAP server that does that (testing
>version of Binc) is in a super alpha state).
Mark seems to disagree with you on that.
>I think what most people do who are in a situation like you are, is
>convert all their mbox-formatted mailboxes to maildir-format . . .
I wouldn't bother to do that. I'd just move the mailboxes to a different
host that expects Berkeley Mail format.
>>Nancy, you've been nice enough to answer users' basic questions over the
>>years too; you are also owed beers.
>Thank you for the beers Adam, they are appreciated.
I'm in beer deficit on Usenet.
>I do this not because it's my job, but because it makes me feel good to
>help people. And also -- actually mainly -- because I like to help people
>switch to standards-based systems (IMAP, Pine, etc.).
Heh. As opposed to standards-breaking Windows clients and servers.
Eduardo Chappa
:) {example.com/user=someuser}~someuser/mail/[] doesn't cut it. I receive
:) a truncated error message that on a Maildir++ format, all folders must
:) be I (and that's it). Is there a restriction that all mail folders on
:) one host, whether for an inbox or saved messages, must be in one
:) format?
Maildir++ is an "extension" of maildir format. It is not maildir, there
are some important differences. It really sounds like your friend
installed Courier as the server.
I have a patch in my web page (address below) that allows you to read
maildir format folders, and you can have a collection of folders where
maildir folders are in the same collection as mbox folders.
I hope this helps.
--
Eduardo
Patches/Help: http://www.math.washington.edu/~chappa/pine/
XML/RSS feed: http://www.math.washington.edu/~chappa/pine/pine.xml
Please send spam to webmaster@localhost
NM Public
>
>> Is this IMAP server used by other people or just by you and your friend?
>> If it's used by others, my guess is that he's not going to want to switch
>> to an IMAP server that supports simultaneous mbox- and maildir-formatted
>> mailboxes (because I think the only IMAP server that does that (testing
>> version of Binc) is in a super alpha state).
>
> Mark seems to disagree with you on that.
I don't think Mark has said anything about IMAP servers that
support simultaneous mbox- and maildir-formatted mailboxes, has
he? I would love to know about IMAP servers that support this
because this is definitely an IMAP FAQ. I know Dovecot can serve
mbox-format mailboxes *or* maildir-format mailboxes, but I don't
think it can serve both to the same user (but I'm not sure about
this). Dovecot is the default IMAP server for some Linux
distributions so it's becoming pretty popular.
UW IMAP can serve mbox, MBX, mh, MMDF, and some other mailbox
formats simultaneously to the same user, but not maildir-format.
There are maildir patches for UW IMAP, but most people who know
about these things say to avoid those patches. (And actually most
IMAP aficionados suggest not using maildir-format mailboxes at
all.)
>> I think what most people do who are in a situation like you are, is
>> convert all their mbox-formatted mailboxes to maildir-format . . .
>
> I wouldn't bother to do that. I'd just move the mailboxes to a different
> host that expects Berkeley Mail format.
That's probably what I would do too.
>>> Nancy, you've been nice enough to answer users' basic questions over the
>>> years too; you are also owed beers.
>
>> Thank you for the beers Adam, they are appreciated.
>
> I'm in beer deficit on Usenet.
Well here's a beer for you for being one of the people I remember
from the good ol' days of Usenet!
>> I do this not because it's my job, but because it makes me feel good to
>> help people. And also -- actually mainly -- because I like to help people
>> switch to standards-based systems (IMAP, Pine, etc.).
>
> Heh. As opposed to standards-breaking Windows clients and servers.
Heh, Pine is a Windows IMAP client too and there are decent IMAP
servers for Windows, so this isn't an anti-Windows thing.
Adam H. Kerman
>On 22 May 2005, Adam H. Kerman wrote:
>>>Is this IMAP server used by other people or just by you and your
>>>friend? If it's used by others, my guess is that he's not going to want
>>>to switch to an IMAP server that supports simultaneous mbox- and
>>>maildir-formatted mailboxes (because I think the only IMAP server that
>>>does that (testing version of Binc) is in a super alpha state).
>>Mark seems to disagree with you on that.
>I don't think Mark has said anything about IMAP servers that support
>simultaneous mbox- and maildir-formatted mailboxes, has he?
He said that the protocol itself doesn't care about the mailbox format. I
suppose I'm not phrasing it correctly as IMAP itself isn't a server.
Just checked something: On the remote host, messages I save (into an
archive created since my user account was moved to that machine) are in a
Berkeley mailbox. I don't even know if saving a message is a client or
server issue. If saving a message is internal to Pine, that would explain
why it's saved in mbox format. But if a message is saved based on
instructions from client to server, then the server handles both formats.
>>>I do this not because it's my job, but because it makes me feel good to
>>>help people. And also -- actually mainly -- because I like to help
>>>people switch to standards-based systems (IMAP, Pine, etc.).
>>Heh. As opposed to standards-breaking Windows clients and servers.
>Heh, Pine is a Windows IMAP client too and there are decent IMAP servers
>for Windows, so this isn't an anti-Windows thing.
I knew which author of standards-breaking clients and servers you were
referring to that was going unmentioned... Certainly it's possible to
write compliant clients for Windows.
Eduardo Chappa
:) UW IMAP can serve mbox, MBX, mh, MMDF, and some other mailbox formats
:) simultaneously to the same user, but not maildir-format. There are
:) maildir patches for UW IMAP, but most people who know about these
:) things say to avoid those patches. (And actually most IMAP aficionados
:) suggest not using maildir-format mailboxes at all.)
I have read all maildir patches and tried many of them. I myself wrote
one. I know what are their strengths and weaknesses. I can tell you that
many of them have big bugs, and as far as I know they are not even kept up
to date. As far as your comment goes, it's very accurate for those
patches. I do not think your comment applies to my patch. I can tell you
many good things about my patch, and encourage you to try it.
The problem with maildir and IMAP access is that the format itself does
not lend itself for IMAP access. For example, it is not specified how to
add keywords to a message, or even worse, a UID. The format is fast in
other aspects though (getting some metadata of the folder is extremely
fast), but my opinion is that making it IMAP compatible is an internal
issue of the server, which means that if a server wants to support IMAP
and maildir at the same time, then it is the job of the server to add
somehow that support (e.g. through some kind of index file), and so you
will see that many servers have their own extension of the maildir format.
I do not consider myself a fan of maildir format. I do not consider it
better or worse than mbox format. Each format has it weakenesses and
strengths. I just think of maildir format as another storage format.
Mark Crispin
> The problem with maildir and IMAP access is that the format itself does
> not lend itself for IMAP access.
This has been my conclusion as well.
> if a server wants to support IMAP
> and maildir at the same time, then it is the job of the server to add
> somehow that support (e.g. through some kind of index file), and so you
> will see that many servers have their own extension of the maildir format.
The problem with an index file is that such a thing is not part of
maildir, and if the index file is not NFS-safe by maildir's standards then
it violates maildir.
Unfortunately, it does not seem to be possible to have a reasoned
technical discussion about this issues with members of the Church of
Maildir. This is particularly troublesome for me, since I'm totally
dedicated to the Church of IMAP :-) and any technical solution that I
endorse must be fully compliant with IMAP.
Other people are willing to cut corners with IMAP and/or maildir, and then
turn around and claim that there is no problem. The Courier server is a
good example of this; it flagrantly violates IMAP in multiple ways and I
think that it also violates maildir. You can watch the ongoing
performance of Courier's author on comp.mail.misc to see why I choose not
to communicate with him.
Liam Greenwood
> On Mon, 23 May 2005, Eduardo Chappa wrote:
>> The problem with maildir and IMAP access is that the format itself does
>> not lend itself for IMAP access.
>
> This has been my conclusion as well.
Heh, I think this is the first time I've see you two in such accord :-)
>
> Other people are willing to cut corners with IMAP and/or maildir, and then
> turn around and claim that there is no problem. The Courier server is a
> good example of this; it flagrantly violates IMAP in multiple ways and I
> think that it also violates maildir. You can watch the ongoing
> performance of Courier's author on comp.mail.misc to see why I choose not
> to communicate with him.
So what IMAP servers are 'good'? I am about to set up a machine with
email services, and as it's just for a few people I have total control.
The machine will mostl likely be running Solaris 10, but could just as
well run a Linux or a BSD if there was a compelling reason to.
BTW, thank you for dreaming up IMAP. It's made life much simpler for
me, email-wise, over multiple machines/multiple accounts/copy mailboxes
that life used to be about.
Cheers, Liam
Eduardo Chappa
:) > if a server wants to support IMAP and maildir at the same time, then
:) > it is the job of the server to add somehow that support (e.g. through
:) > some kind of index file), and so you will see that many servers have
:) > their own extension of the maildir format.
:)
:) The problem with an index file is that such a thing is not part of
:) maildir, and if the index file is not NFS-safe by maildir's standards
:) then it violates maildir.
Agree.
:) Other people are willing to cut corners with IMAP and/or maildir, and
:) then turn around and claim that there is no problem. The Courier
:) server is a good example of this; it flagrantly violates IMAP in
:) multiple ways and I think that it also violates maildir. You can watch
:) the ongoing performance of Courier's author on comp.mail.misc to see
:) why I choose not to communicate with him.
Courier violates maildir by being compliant to maildir++, created by the
same author of Courier. It is a tradeoff that is done to improve
performance of the server (e.g. not having to stat() every single file
upon access to the folder).
It is a tough problem to extend maildir format to be compliant with IMAP.
I do not think it is impossible, though. The main problem that I have with
maildir implementations is that all of them extend maildir in a different
way to overcome the problems that maildir has, and I really wish that
everyone agreed on just one extension and that this extension be
compatible with IMAP.
Mark Crispin
> So what IMAP servers are 'good'?
UW imapd (which I wrote) and Cyrus (from CMU) come to mind right away.
UW imapd has by far the easiest installation and setup, but to get it to
perform well in medium to large installations you have to do some tuning
(don't use the default traditional UNIX mailbox format!).
Cyrus provides you with much more administrative control, and it has a
very fast mailbox format, but its installation and setup is complex and
for small installations it's overkill.
Perhaps the best book on the subject is "Managing IMAP", by Dianna Mullet
& Kevin Mullet, published by O'Reilly, ISBN 0-596-00012-X.
Mark Crispin
> same author of Courier. It is a tradeoff that is done to improve
> performance of the server (e.g. not having to stat() every single file
> upon access to the folder).
I also believe that it's not NFS-safe, which pretty much obliviates the
entire point of maildir.
More to the point; if you're going to use an external index with a
one-file/one-message format, why not just run Cyrus and get an IMAP server
that complies with the IMAP specification (and whose authors are active
participants in the IMAP community)?
Eduardo Chappa
:) More to the point; if you're going to use an external index with a
:) one-file/one-message format, why not just run Cyrus and get an IMAP
:) server that complies with the IMAP specification (and whose authors are
:) active participants in the IMAP community)?
My experience in math is that a new proof of an old result always sheds
light on the nature of the theorem (even theories have been created by
trying to simplify proofs!). Therefore, my answer to your question is that
I see a lot of value in reinventing the wheel (not in everything, but this
is one of the places where reinventing it has some value).
Liam Greenwood
> On Mon, 24 May 2005, Liam Greenwood wrote:
>> So what IMAP servers are 'good'?
>
> UW imapd (which I wrote) and Cyrus (from CMU) come to mind right away.
>
> UW imapd has by far the easiest installation and setup, but to get it to
> perform well in medium to large installations you have to do some tuning
> (don't use the default traditional UNIX mailbox format!).
So my 5 or so users will have no issues :-). Plus I am guessing that
pine run on the box won't have any issues with the mailbox format.
>
> Perhaps the best book on the subject is "Managing IMAP", by Dianna Mullet
> & Kevin Mullet, published by O'Reilly, ISBN 0-596-00012-X.
Thank you for the pointer.
Cheers, Liam
Mark Crispin
>> UW imapd has by far the easiest installation and setup, but to get it to
>> perform well in medium to large installations you have to do some tuning
>> (don't use the default traditional UNIX mailbox format!).
> So my 5 or so users will have no issues :-). Plus I am guessing that
> pine run on the box won't have any issues with the mailbox format.
Correct. Internally, the same code is used by both Pine and UW imapd.
So, even if you change the mailbox format used by UW imapd (e.g. mbx
format), Pine will handle it just fine.
Adam H. Kerman
>Other people are willing to cut corners with IMAP and/or maildir, and
>then turn around and claim that there is no problem. The Courier server
>is a good example of this; it flagrantly violates IMAP in multiple ways
>and I think that it also violates maildir. You can watch the ongoing
>performance of Courier's author on comp.mail.misc to see why I choose not
>to communicate with him.
Turns out my friend is using Binc and not Courier, as Eduardo had guessed.
Not to choose sides or anything, I never thought that the Berkeley Mail
format was broken, although it could benefit from a specification
describing the boundary.
Eduardo Chappa
:) At 9:33am -0700, 05/23/05, Mark Crispin <m...@CAC.Washington.EDU> wrote:
:)
:) >Other people are willing to cut corners with IMAP and/or maildir, and
:) >then turn around and claim that there is no problem. The Courier
:) >server is a good example of this; it flagrantly violates IMAP in
:) >multiple ways and I think that it also violates maildir. You can
:) >watch the ongoing performance of Courier's author on comp.mail.misc to
:) >see why I choose not to communicate with him.
:)
:) Turns out my friend is using Binc and not Courier, as Eduardo had
:) guessed.
Ok, good to know.
:) Not to choose sides or anything, I never thought that the Berkeley Mail
:) format was broken, although it could benefit from a specification
:) describing the boundary.
How did you conclude that. I guess I don't understand what you are
referring to.
Adam H. Kerman
>Adam H. Kerman wrote:
>>Not to choose sides or anything, I never thought that the Berkeley Mail
>>format was broken, although it could benefit from a specification
>>describing the boundary.
>How did you conclude that. I guess I don't understand what you are
>referring to.
Perhaps if you hadn't cut out the quote that I was referring to...
Are you saying that there IS a specification describing the Berkeley Mail
message boundary?
Eduardo Chappa
:) At 3:07pm -0700, 05/23/05, Eduardo Chappa <cha...@math.washington.edu> wrote:
:) >Adam H. Kerman wrote:
:)
:) >>Not to choose sides or anything, I never thought that the Berkeley
:) >>Mail format was broken, although it could benefit from a
:) >>specification describing the boundary.
:)
:) >How did you conclude that. I guess I don't understand what you are
:) >referring to.
:)
:) Perhaps if you hadn't cut out the quote that I was referring to...
I did not cut anything out, I quoted your message and quote "as it was".
Again, I am not removing any of the text you quoted.
:) Are you saying that there IS a specification describing the Berkeley
:) Mail message boundary?
No. I am asking how you drew your conclusion. I still can't figure what
did this have to do with maildir format.
I'm really curious to know how you drew your conclusion.
NM Public
>
> Just checked something: On the remote host, messages I save (into an
> archive created since my user account was moved to that machine) are in a
> Berkeley mailbox. I don't even know if saving a message is a client or
> server issue. If saving a message is internal to Pine, that would explain
> why it's saved in mbox format. But if a message is saved based on
> instructions from client to server, then the server handles both formats.
It sounds like you might not be using Pine as an IMAP client, but
instead are accessing the "remote host" using local file system
calls. Can you post what you have specified for the following
Pine variables:
inbox-path
incoming-folders
folder-collections
This will help us to figure out what's going on.
In answer to your question about whether saving is a client or
server issue: It's a server issue if you are accessing the
mailstore with IMAP, it's a client issue if you are accessing the
mailstore with local file system calls.
Adam H. Kerman
>>>>Not to choose sides or anything, I never thought that the Berkeley
>>>>Mail format was broken, although it could benefit from a
>>>>specification describing the boundary.
>>>How did you conclude that. I guess I don't understand what you are
>>>referring to.
>>Perhaps if you hadn't cut out the quote that I was referring to...
>I did not cut anything out, I quoted your message and quote "as it was".
Fine. It seems I didn't put the context in.
>Again, I am not removing any of the text you quoted.
>>Are you saying that there IS a specification describing the Berkeley
>>Mail message boundary?
>No. I am asking how you drew your conclusion. I still can't figure what
>did this have to do with maildir format.
>I'm really curious to know how you drew your conclusion.
In conclusion, I didn't chose one format or the other (let alone a
proprietary extension to Maildir to make it work with IMAP). One complaint
about mboxes is the unspecified delimiter (usually copied from ENVELOPE
FROM) as it may contain a time stamp from the original server that posted
the message that's not in a standard format, or it may have a nonstandard
method of identifying the message's author. The boundary needs to be
parsed to be recognized; some parsing isn't capable of recognizes some of
the common varieties out there. Furthermore, some lazy parsers look for
nothing more than "From " following two new lines.
Still, it should be possible to write a parser to catch nearly all
variations of what's put in the boundary.
Now, if someone were to write a spec that influenced new versions of
programs, this issue would go away.
Mail storage formats like maildir eliminate the boundary issue by using
separate files for each message.
Adam H. Kerman
>On 23 May 2005, Adam H. Kerman wrote:
>>Just checked something: On the remote host, messages I save (into an
>>archive created since my user account was moved to that machine) are in
>>a Berkeley mailbox. I don't even know if saving a message is a client or
>>server issue. If saving a message is internal to Pine, that would
>>explain why it's saved in mbox format. But if a message is saved based
>>on instructions from client to server, then the server handles both
>>formats.
>It sounds like you might not be using Pine as an IMAP client, but instead
>are accessing the "remote host" using local file system calls.
Yes. To check that, I had ssh'ed into the remote machine and used Pine
locally to see what kind of mailbox I'd get when I stored a message.
>Can you post what you have specified for the following Pine variables:
From .pinerc on the local machine:
>inbox-path
Not set.
>incoming-folders
incoming-folders=remote {example.com/user=[user]/notls}inbox
IMAP to the Maildir++ inbox works with this syntax.
"notls" due to unrelated problem of missing certificate, mentioned
earlier in the thread.
>folder-collections
remote_collection {example.com/user=[user]/notls}~[user]/mail/[]
This one gave me a truncated error message
With a Maildir++ depot, you must create all mailboxes under I
when attempting to make this entry on the Setup List screen.
Say, Mark, what is that full error message supposed to be?
I also tried variations on the syntax, leaving off the home directory,
giving the full path, leaving off the final slash, etc. All resulted in
same error message. Those archived folders are mboxes.
> This will help us to figure out what's going on.
As I told Eduardo, he's using Binc's version of imapd.
Eduardo Chappa
:) >>Are you saying that there IS a specification describing the Berkeley
:) >>Mail message boundary?
:)
:) >No. I am asking how you drew your conclusion. I still can't figure
:) >what did this have to do with maildir format.
:)
:) >I'm really curious to know how you drew your conclusion.
:)
:) In conclusion, I didn't chose one format or the other (let alone a
:) proprietary extension to Maildir to make it work with IMAP). One
:) complaint about mboxes is the unspecified delimiter (usually copied
:) from ENVELOPE FROM) as it may contain a time stamp from the original
:) server that posted the message that's not in a standard format, or it
:) may have a nonstandard method of identifying the message's author. The
:) boundary needs to be parsed to be recognized; some parsing isn't
:) capable of recognizes some of the common varieties out there.
:) Furthermore, some lazy parsers look for nothing more than "From "
:) following two new lines.
:)
:) Still, it should be possible to write a parser to catch nearly all
:) variations of what's put in the boundary.
:)
:) Now, if someone were to write a spec that influenced new versions of
:) programs, this issue would go away.
:)
:) Mail storage formats like maildir eliminate the boundary issue by using
:) separate files for each message.
I do agree with most things you say, but there is one thing you are
missing. A mailbox format is an internal thing of a server or a client, it
is not a standard. A client may choose to save internal information about
a mailbox in the mailbox itself, for example. There's no requirement that
the mailbox can be read by many clients.
If I recall correctly, Kmail had its own format and could not read folders
created by Pine, so it was suggested many years ago that at least folders
read by Pine could be read by Kmail. C-Client modifies the usual Berkeley
format to store folder metadata, specially by adding an internal message,
making this some king of "C-client mbox format".
Therefore you are probably not going to find standards about formats, but
explanations of them.
Now in terms of separator lines, there are certain restrictions on what
Pine parses as a From separator line, but there is no "standard" about
what such line must contain.
Mark Crispin
> This one gave me a truncated error message
> With a Maildir++ depot, you must create all mailboxes under I
> when attempting to make this entry on the Setup List screen.
> Say, Mark, what is that full error message supposed to be?
I don't know. You can ask the author of the IMAP server. The complete
text may also be in the .pine-debug[1-4] file.
Adam H. Kerman
I deliberately refrained from using the word "standard" so as not to imply
that the specification would be appropriate for submission to the RFC
editor proposed for standards track. No, proprietary mailbox formats are
not an internetwork issue.
But what does that have to do with writing a specification for it?
Someone, perhaps a programmer still around from the time that mail (the
mail client) was written could write a specification for the boundary. It
seems like a good idea as so Mail clients written subsequently claim to
recognize it, if not implement it. Once the spec is written, then newer
versions of various Mail clients would have something to comply with.
>If I recall correctly, Kmail had its own format and could not read
>folders created by Pine, so it was suggested many years ago that at least
>folders read by Pine could be read by Kmail. C-Client modifies the usual
>Berkeley format to store folder metadata, specially by adding an internal
>message, making this some king of "C-client mbox format".
Didn't know that about Kmail; thanks.
>Therefore you are probably not going to find standards about formats, but
>explanations of them.
Is not a spec an explanation?
>Now in terms of separator lines, there are certain restrictions on what
>Pine parses as a From separator line, but there is no "standard" about
>what such line must contain.
Pine seems to make more of an effort to parse it than other clients do..
Eduardo Chappa
:) >Therefore you are probably not going to find standards about formats,
:) >but explanations of them.
:)
:) Is not a spec an explanation?
I guess we use the different words for the same thing.
:) >Now in terms of separator lines, there are certain restrictions on
:) >what Pine parses as a From separator line, but there is no "standard"
:) >about what such line must contain.
:)
:) Pine seems to make more of an effort to parse it than other clients
:) do..
True.
NM Public
>
> remote_collection {example.com/user=[user]/notls}~[user]/mail/[]
>
> This one gave me a truncated error message
> With a Maildir++ depot, you must create all mailboxes under I
My guess is that the error is something like this:
With a Maildir++ depot, you must create all mailboxes under INBOX
and one of the following will probably work in your Pine
folder-collections list:
{example.com/user=[user]/notls}[]
{example.com/user=[user]/notls}INBOX.[]
{example.com/user=[user]/notls}INBOX/[]
Let us know which, if any, of these work.
Hope this helps,
Nancy
--
Nancy McGough ~ <http://www.ii.com/> ~ <http://deflexion.com/>
IMAP, pine, procmail, message deflexion, infinity, and more
Adam H. Kerman
> On 24 May 2005, Adam H. Kerman wrote:
>>remote_collection {example.com/user=[user]/notls}~[user]/mail/[]
>>This one gave me a truncated error message
>>With a Maildir++ depot, you must create all mailboxes under I
>My guess is that the error is something like this:
>With a Maildir++ depot, you must create all mailboxes under INBOX
>and one of the following will probably work in your Pine
>folder-collections list:
>{example.com/user=[user]/notls}[]
>{example.com/user=[user]/notls}INBOX.[]
>{example.com/user=[user]/notls}INBOX/[]
I guess I don't understand how this syntax would inform Pine to look for a
collection of mboxes in a completely different path. The only Maildir++
folder is the inbox itself.
Adam H. Kerman
Tried 'em each. No joy. The second one gave me an error message. The first
and third showed no folders. Claimed to create folders, but I never saw
them.
NM Public
>
>>> My guess is that the error is something like this:
>
>>> With a Maildir++ depot, you must create all mailboxes under INBOX
>
>>> and one of the following will probably work in your Pine
>>> folder-collections list:
>
>>> {example.com/user=[user]/notls}[]
>>> {example.com/user=[user]/notls}INBOX.[]
>>> {example.com/user=[user]/notls}INBOX/[]
>
>> I guess I don't understand how this syntax would inform Pine to look for a
>> collection of mboxes in a completely different path. The only Maildir++
>> folder is the inbox itself.
This syntax isn't necessarily pointing at mbox-formatted
mailboxes. It is pointing at the personal IMAP mailstore of
'[user]'. The mailbox format depends on how the Binc IMAP server
was configured. From the error message you posted, it seems that
this Binc IMAP server is configured to serve Maildir++ formatted
mailboxes. As I said in an earlier message, there are testing
releases of Binc IMAP that can serve mbox-formatted mailboxes,
but it seems unlikely that your system is using that unreleased
(and buggy) version of Binc IMAP.
>>> Let us know which, if any, of these work.
>
> Tried 'em each. No joy. The second one gave me an error message. The first
> and third showed no folders. Claimed to create folders, but I never saw
> them.
Here are some links that might help you figure out what's going
on:
How can I tell Binc IMAP where my mailboxes/folders are?
<http://bincimap.org/bincimap-faq.html#q12>
Terminology section of IMAP Service Providers page, especially
the term 'IMAP path or IMAP root or IMAP prefix or IMAP namespace'
<http://www.ii.com/internet/messaging/imap/isps/#terms>
The Binc mailing list, which you can read in Pine by typing:
G {news.gmane.org/nntp}#news.gmane.mail.imap.binc.general
^
Pine's GotoFldr command
Also, ask other people who access this system with IMAP what they
have specified as the 'IMAP path' or 'IMAP root' or 'IMAP prefix'
or 'IMAP namespace.' You might also want to try using another
IMAP client to access this server -- this might help you to
figure out the 'IMAP path' and hierarchy separator (delimiter).
Nancy
--
Nancy McGough ~ <http://www.ii.com> ~ <http://deflexion.com>
IMAP, pine, procmail, data deflexion, infinity, and more
Adam H. Kerman
>Here are some links that might help you figure out what's going on:
> How can I tell Binc IMAP where my mailboxes/folders are?
> <http://bincimap.org/bincimap-faq.html#q12>
I'll see if I can get him to read this. Yes, that would seem to answer the
question, but doesn't explain how to tell it that user-created mailboxes
are mbox versus Maildir.
Interestly enough, the next question is "How do I get SSL to work?" which
is the other question I asked in this thread!
>Also, ask other people who access this system with IMAP what they have
>specified as the 'IMAP path' or 'IMAP root' or 'IMAP prefix' or 'IMAP
>namespace.'
It seems that I am the only one trying to use IMAP.