Webmail considerations...
Przemyslaw Wegrzyn
relatively high loads. I know, there are many open source web-mail systems
, but they doesn't satisfy me. Almost every falls into one of two
cateogries: php based, using imap; perl cgi based, using IMAP or direct
filesystem access...
I'd like someone experienced with such systems help me with the following:
1. Almost every available webmail system uses the following way of handling
(rreceiving, in this example) attachements: load the whole message body
from IMAP server or message file, decode it and send to the client.
The _whole_ attachement gets loaded into server's RAM. Isn't it waste of
resources/ killing the server ?
I think it should read/decode/send the attachement on a line-by-line
manner. Am I right ?
2. Which one is better - accessing maildirs directly, or using IMAP ?
I can see that IMAP seems to be more scalable / universal... Maildirs
probably can be much faster to work with directly, but probably less
secure...
Any other pros/contras ?
3. I'm going to develop the front-end using Apache::ASP or php, not
decided yet, and access the mails through the middle-tier daemon.
The question is - is it a good way to use persistent IMAP connections ? If
so, there will be no overhead of authentication on every operation, but
there can be many open IMAP connections at the same time.
Which strategy is better ?
4. Are there any libraries similar to c-client, maybe some C++ ones ?
5. Does c-client library allow to retrieve the message body (attachements)
partialy (see 1) ?
I've seen some /tmp access in its source code - does it dowload whole
message to /tmp ?
Hmm, that's all for now...
TIA
-=Czaj-nick=-
Sam
Hash: SHA1
In article <9g030f$59k$1...@panorama.wcss.wroc.pl>,
Przemyslaw Wegrzyn <cza...@czajsoft.pl> writes:
> 2. Which one is better - accessing maildirs directly, or using IMAP ?
Accessing maildirs directly.
> I can see that IMAP seems to be more scalable / universal... Maildirs
> probably can be much faster to work with directly, but probably less
> secure...
There's nothing inherently secure about maildirs. The only thing that IMAP
will give you is the ability to have the actual mail stored on a remote
server. That doesn't necessarily translate into better or worse security.
If you fail to secure your web server, and leave an old FTP server running
for no good reason, you'll get r00ted whether or not you use maildirs or
not.
ALso, note that many people don't bother to explicitly log out of their
webmail account, and they simply close their browser. This means that you
will always have a bunch of idle IMAP sessions hanging around until they
time out due to inactivity.
> Any other pros/contras ?
If you use IMAP as an intermediary, you will now have to maintain two
separate applications, and you will have to do regression testing each time
you upgrade either one -- twice as much work.
You always should use the right tool for the right job. IMAP is not the
right tool for every situation.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.5 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt
iD8DBQE7I5yG+3BFaxHnGY0RAiE+AKCyJLcsfjRwh9IECfjVm5sSA8sx9wCgsigq
/bRffHBDET8rcMKCJ2kyy4k=
=Nf6r
-----END PGP SIGNATURE-----
Przemyslaw Wegrzyn
> There's nothing inherently secure about maildirs. The only thing that
> IMAP will give you is the ability to have the actual mail stored on a
> remote
> server.
I can't ignore this ability - I still have in mind the posiblity of
scaling by using load balancing.
But also I could use maildirs mounted from webservers with NFS, am I right ?
> That doesn't necessarily translate into better or worse security.
IMHO It does someway - if I use my coustom soft to access maildirs, I play
with security my self. If I use IMAP, my imap daemon becomes the main
source of possible insecurities -it's the only software that directly plays
with underlying filesystem. Also imapd is managed and used by many people,
so several bugs gets corrected by others..
I have to use the imapd anyway for my clients - so why not to use it by
myself ?
> If you fail to secure your web server, and leave an old FTP server running
> for no good reason, you'll get r00ted whether or not you use maildirs or
> not.
Of course !
> ALso, note that many people don't bother to explicitly log out of their
> webmail account, and they simply close their browser. This means that you
> will always have a bunch of idle IMAP sessions hanging around until they
> time out due to inactivity.
Using some middleware helps to manage such situations...
> If you use IMAP as an intermediary, you will now have to maintain two
> separate applications, and you will have to do regression testing each
> time you upgrade either one -- twice as much work.
That's right. But as mentioned above - I _do_ need imap also for customers,
and also I'm not alone working with imapd, so possible breaks are usually
corrected quickly.
> You always should use the right tool for the right job. IMAP is not the
> right tool for every situation.
That's the reason of my posts - I want to read some post from people
experienced with such installations...
Anyone here knows what is used at (for example) hotmail ?
-=Czaj-nick=-
Sam
Hash: SHA1
In article <9g0dk8$7ti$1...@panorama.wcss.wroc.pl>,
Przemyslaw Wegrzyn <cza...@czajsoft.pl> writes:
> Sam wrote:
>
>> There's nothing inherently secure about maildirs. The only thing that
Slippery fingers -- ^H^Hinherently insecure.
>> IMAP will give you is the ability to have the actual mail stored on a
>> remote
>> server.
> I can't ignore this ability - I still have in mind the posiblity of
> scaling by using load balancing.
> But also I could use maildirs mounted from webservers with NFS, am I right ?
Pretty much. Big boys do it all the time.
>> That doesn't necessarily translate into better or worse security.
> IMHO It does someway - if I use my coustom soft to access maildirs, I play
> with security my self. If I use IMAP, my imap daemon becomes the main
> source of possible insecurities -it's the only software that directly plays
If your web server is r00ted, it doesn't matter how the mail gets read.
> with underlying filesystem. Also imapd is managed and used by many people,
> so several bugs gets corrected by others..
> I have to use the imapd anyway for my clients - so why not to use it by
> myself ?
Because, as I mentioned, you now have an extra layer of complexity that
forces you into regression testing any time either application is
reconfigured/ugpraded.
>> You always should use the right tool for the right job. IMAP is not the
>> right tool for every situation.
> That's the reason of my posts - I want to read some post from people
> experienced with such installations...
>
> Anyone here knows what is used at (for example) hotmail ?
Home-brewed slop.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.5 (GNU/Linux)
Comment: http://www.geocities.com/SiliconValley/Peaks/5799/GPGKEY.txt
iD8DBQE7I72Y+3BFaxHnGY0RAtYOAKCcq8Ga7s9X0yQXSZtyh4hN60YllwCeJp2p
2Rupvp3N8iFFo60KMDOchkw=
=7J3h
-----END PGP SIGNATURE-----
Anil Madhavapeddy
>
> That's the reason of my posts - I want to read some post
> from people experienced with such installations...
I suspect that the optimisations (e.g. partial attachment fetching) you are
talking about would merely shift your costs away from hardware layout to
development time.
From our experiences with Horde/IMP, which has reported installations with
millions of users, it _is_ inefficient to run IMAP servers, and
reauthenticate every time a user clicks on URL, but it works just fine. It
scales pretty well just by using more powerful hardware, and has the
advantage that most of the components come straight off the shelf, and the
architecture is simple. In operational terms, I value this more than
getting every one of those components being amazingly efficient.
FWIW, we are slowly working on an IMAP proxy which can 'accelerate' an IMAP
server by maintaining persistent client connections ... this will remove the
major cause of load on large IMP installations. Disclaimer: No public
release yet, and no estimated date.
The other interesting point is the inefficiencies in c-client itself, which
affects how well PHP can interact with an IMAP server. There's a thread
about this on the Horde-dev list:
http://marc.theaimsgroup.com/?t=99115062200001&w=2&r=1 . If you really want
to develop something, a more streamlined IMAPv4 implementation for PHP would
go a long way :-)
PHP also recently imported a mail parsing module using a chunk of Sam's code
from maildrop. That would come in useful if you want to parse maildirs
directly.
--
Anil Madhavapeddy, <an...@recoil.org>