Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Big local mailboxes (mbox format) -> fast index ?

65 views
Skip to first unread message

rackamx

unread,
Apr 10, 2003, 10:40:19 AM4/10/03
to
Hi,

Are there any patches out there to get a better handling of big local
mailboxes (in mbox format) ? I happen to have some ( almost 50 Mo ). I
know this is bad, and that I should split them ... but I don't want to
do that. I have searched the web but didn't find anything this far. Does
anyone know if someone ever tried to have mutt store the state of local
mbox in faster index files, so that on the next loading of such a
mailbox mutt doesn't have to parse the whole file again ?

--
Rackx

Sven Guckes

unread,
Apr 10, 2003, 7:42:19 PM4/10/03
to
* rackamx <rac...@netcourrier.com>:

> Are there any patches out there to get a better
> handling of big local mailboxes (in mbox format)?
> I happen to have some ( almost 50 Mo ).

"50 Mo"?

my mutt folder here has 52,000+ messages and mutt
reads it in and creates an index in about a minute.

if you need the real fast version which skips almost
all headers but the usual (From/To/Cc/Date/Subject)
then get mutt 2.1.3 which does it blindingly fast.

> Does anyone know if someone ever tried to have mutt
> store the state of local mbox in faster index files,
> so that on the next loading of such a mailbox mutt
> doesn't have to parse the whole file again ?

index files are for wimps. mutt doesn't need this.
all speed patches are only improved when the
other mailers improved their speed a little.

i hear that mutt 3.x will guess the index without
looking at the folder at all. however, this requires
the AI patch which should be out real soon now...

all this info is in the manual, of course.

Sven [fzvyrlf? zr?]

rackamx

unread,
Apr 11, 2003, 12:30:41 PM4/11/03
to
Dans l'article <2003-04-1...@guckes.net>, Sven Guckes a écrit :
> * rackamx <rac...@netcourrier.com>:
>> Are there any patches out there to get a better
>> handling of big local mailboxes (in mbox format)?
>> I happen to have some ( almost 50 Mo ).

> "50 Mo"?

Well, sorry, there are only 6,000+ messages in my mbox, but it takes
almost 20 sec to get it running, whereas in a wimp I will not mention it
takes less than 1 sec.

> my mutt folder here has 52,000+ messages and mutt
> reads it in and creates an index in about a minute.

isn't that long ? I mean, couldn't it be faster by using some space to
store the index on disk ?

> if you need the real fast version which skips almost
> all headers but the usual (From/To/Cc/Date/Subject)
> then get mutt 2.1.3 which does it blindingly fast.

yeah, where can I find it ?

>> Does anyone know if someone ever tried to have mutt
>> store the state of local mbox in faster index files,
>> so that on the next loading of such a mailbox mutt
>> doesn't have to parse the whole file again ?

> index files are for wimps. mutt doesn't need this.
> all speed patches are only improved when the
> other mailers improved their speed a little.

wasn't there a GUI for mutt at a certain point ? Why should wimps be
faster than mutt on loading big mbox ? there are patches to cache
headers for maildir, so, I just wanna know if someone has made the same
concerning mbox.

> i hear that mutt 3.x will guess the index without
> looking at the folder at all. however, this requires
> the AI patch which should be out real soon now...

> all this info is in the manual, of course.

> Sven [fzvyrlf? zr?]

yeah ... [VJGO]

--
It's time to kick ass and chew bubble gum, and I'm all outta gum!
Duke nukem

Sven Guckes

unread,
Apr 13, 2003, 2:55:01 PM4/13/03
to
* rackamx <rac...@netcourrier.com>:

> Dans l'article <2003-04-1...@guckes.net>, Sven Guckes a écrit :
>> * rackamx <rac...@netcourrier.com>:
>>> Are there any patches out there to get a better
>>> handling of big local mailboxes (in mbox format)?
>>> I happen to have some ( almost 50 Mo ).
>
>> "50 Mo"?
>
> Well, sorry, there are only 6,000+ messages in my
> mbox, but it takes almost 20 sec to get it running

processor, OS, mailbox or maildir?

> whereas in a wimp I will not mention it takes less than 1 sec.

"wimp"?

>> my mutt folder here has 52,000+ messages and mutt
>> reads it in and creates an index in about a minute.
>
> isn't that long ? I mean, couldn't it be faster
> by using some space to store the index on disk?

it could always be faster when the data you want
is already available directly. but if it isn't
then some program has to compute at some time...

>> if you need the real fast version which skips almost
>> all headers but the usual (From/To/Cc/Date/Subject)
>> then get mutt 2.1.3 which does it blindingly fast.
>
> yeah, where can I find it ?

google for the announcement..

>>> Does anyone know if someone ever tried to have mutt
>>> store the state of local mbox in faster index files,
>>> so that on the next loading of such a mailbox mutt
>>> doesn't have to parse the whole file again ?
>
>> index files are for wimps. mutt doesn't need this.
>> all speed patches are only improved when the
>> other mailers improved their speed a little.
>
> wasn't there a GUI for mutt at a certain point?

yes - and the mutt GUI stuff was dropped.

> Why should wimps be faster than mutt on loading big mbox?
> there are patches to cache headers for maildir, so, I just
> wanna know if someone has made the same concerning mbox.

"maybe". i never tried those patches.

ps: if you must use text in the attribution then please
adjust it to the language of the rest of the message.

Sven

Jens Kubieziel

unread,
Apr 13, 2003, 4:52:19 PM4/13/03
to
Sven Guckes <guc...@math.fu-berlin.de> wrote:
> my mutt folder here has 52,000+ messages and mutt
> reads it in and creates an index in about a minute.

Is there a way to count all mail in all mailboxes? If yes, how do I
realise that?

Thanks
--
Jens Kubieziel http://www.kubieziel.de
When a person goes on a diet, the first thing he loses is his temper.

Sven Guckes

unread,
Apr 14, 2003, 5:55:00 AM4/14/03
to
* Jens Kubieziel <spa...@kuwest.de>:

> Is there a way to count all mail in all mailboxes?

yes

> If yes, how do I realise that?

put all mails into one big folder:

cat folder1 folder2 ... folderN > big_folder

and then count the From_ lines:
grep -c 'From ' big_folder

or let mutt parse the folder
mutt -f big_folder
and then see what is says.

ps: please learn to post a new message
and *not* follow up to others' posts.

Sven

Jens Kubieziel

unread,
Apr 14, 2003, 9:40:05 AM4/14/03
to
Sven Guckes <guc...@math.fu-berlin.de> wrote:
> * Jens Kubieziel <spa...@kuwest.de>:
>> [Counting all e-mails]

>> If yes, how do I realise that?
>
> put all mails into one big folder:
> and then count the From_ lines:

OK, I guess a better solution is to write a small shell script, which
walks through all folders and greps those data.

> ps: please learn to post a new message and *not* follow up to others'
> posts.

It was a question which came up while reading your message. IMHO a follow
up with subject change was a better solution here.


--
Jens Kubieziel http://www.kubieziel.de

FORTRAN rots the brain.
-- John McQuillin

Oliver Braun

unread,
Apr 14, 2003, 12:43:26 PM4/14/03
to
> Sven Guckes <guc...@math.fu-berlin.de> wrote:
> > * Jens Kubieziel <spa...@kuwest.de>:
> >> [Counting all e-mails]
> >> If yes, how do I realise that?

> > put all mails into one big folder:
> > and then count the From_ lines:

* Jens Kubieziel <spa...@kuwest.de>:


> OK, I guess a better solution is to write a small shell script, which
> walks through all folders and greps those data.

Use chk4mail: http://freshmeat.net/projects/chk4mail/?topic_id=28%2C35

example:

# chk4mail IN/incoming
incoming - new of total 98

Regards,
Olli
--
Just Nothing :: forall a. Maybe (Maybe a)

0 new messages