Mu index on MacOS: sometimes very fast, sometimes very slow

[trois....@gmail.com]

Hi everybody,

I'm using mu 0.9.16 on MacOS (10.11.6), with Emacs 25.1.1. Sometimes, updating the index of my 28000 messages takes 1 second, sometimes it is far slower (20 seconds). Nothing else seems to be using the processor.

I noticed that when the index is fast, further indexes will be fast too.

Do you know how I could get rid of the slow updates?

Here are the logs (for the same Maildir directory and same messages):

Fast update:

2016-11-13 23:42:03 logging enabled
2016-11-13 23:42:17 -> cmd:index path:"/Users/me/.mail" my-addresses:m...@me.org
2016-11-13 23:42:17 * Received 58 byte(s)
2016-11-13 23:42:17 <- (:info index :status running :processed 0 :updated 0)
2016-11-13 23:42:17 * Received 488 byte(s)
2016-11-13 23:42:17 <- (:info index :status running :processed 1000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 2000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 3000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 4000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 5000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 6000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 7000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 8000 :updated 0)
2016-11-13 23:42:17 * Received 247 byte(s)
2016-11-13 23:42:17 <- (:info index :status running :processed 9000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 10000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 11000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 12000 :updated 0)
2016-11-13 23:42:17 * Received 186 byte(s)
2016-11-13 23:42:17 <- (:info index :status running :processed 13000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 14000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 15000 :updated 0)
2016-11-13 23:42:17 * Received 496 byte(s)
2016-11-13 23:42:17 <- (:info index :status running :processed 16000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 17000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 18000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 19000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 20000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 21000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 22000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 23000 :updated 0)
2016-11-13 23:42:17 * Received 248 byte(s)
2016-11-13 23:42:17 <- (:info index :status running :processed 24000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 25000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 26000 :updated 0)
2016-11-13 23:42:17 <- (:info index :status running :processed 27000 :updated 0)
2016-11-13 23:42:17 * Received 77 byte(s)
2016-11-13 23:42:17 <- (:info index :status complete :processed 27697 :updated 0 :cleaned-up 0)
2016-11-13 23:42:25 logging disabled

Slow update:

2016-11-22 15:48:39 logging enabled
2016-11-22 15:48:54 -> cmd:index path:"/Users/me/.mail" my-addresses:m...@me.org
2016-11-22 15:48:54 * Received 58 byte(s)
2016-11-22 15:48:54 <- (:info index :status running :processed 0 :updated 0)
2016-11-22 15:48:55 * Received 61 byte(s)
2016-11-22 15:48:55 <- (:info index :status running :processed 1000 :updated 2)
2016-11-22 15:48:57 * Received 61 byte(s)
2016-11-22 15:48:57 <- (:info index :status running :processed 2000 :updated 2)
2016-11-22 15:48:58 * Received 61 byte(s)
2016-11-22 15:48:58 <- (:info index :status running :processed 3000 :updated 2)
2016-11-22 15:49:00 * Received 61 byte(s)
2016-11-22 15:49:00 <- (:info index :status running :processed 4000 :updated 2)
2016-11-22 15:49:02 * Received 61 byte(s)
2016-11-22 15:49:02 <- (:info index :status running :processed 5000 :updated 2)
2016-11-22 15:49:04 * Received 61 byte(s)
2016-11-22 15:49:04 <- (:info index :status running :processed 6000 :updated 2)
2016-11-22 15:49:05 * Received 61 byte(s)
2016-11-22 15:49:05 <- (:info index :status running :processed 7000 :updated 2)
2016-11-22 15:49:06 * Received 61 byte(s)
2016-11-22 15:49:06 <- (:info index :status running :processed 8000 :updated 2)
2016-11-22 15:49:08 * Received 61 byte(s)
2016-11-22 15:49:08 <- (:info index :status running :processed 9000 :updated 2)
2016-11-22 15:49:08 * Received 62 byte(s)
2016-11-22 15:49:08 <- (:info index :status running :processed 10000 :updated 2)
2016-11-22 15:49:09 * Received 62 byte(s)
2016-11-22 15:49:09 <- (:info index :status running :processed 11000 :updated 2)
2016-11-22 15:49:10 * Received 62 byte(s)
2016-11-22 15:49:10 <- (:info index :status running :processed 12000 :updated 2)
2016-11-22 15:49:11 * Received 62 byte(s)
2016-11-22 15:49:11 <- (:info index :status running :processed 13000 :updated 2)
2016-11-22 15:49:13 * Received 62 byte(s)
2016-11-22 15:49:13 <- (:info index :status running :processed 14000 :updated 2)
2016-11-22 15:49:14 * Received 62 byte(s)
2016-11-22 15:49:14 <- (:info index :status running :processed 15000 :updated 2)
2016-11-22 15:49:15 * Received 62 byte(s)
2016-11-22 15:49:15 <- (:info index :status running :processed 16000 :updated 2)
2016-11-22 15:49:16 * Received 62 byte(s)
2016-11-22 15:49:16 <- (:info index :status running :processed 17000 :updated 2)
2016-11-22 15:49:18 * Received 62 byte(s)
2016-11-22 15:49:18 <- (:info index :status running :processed 18000 :updated 2)
2016-11-22 15:49:19 * Received 62 byte(s)
2016-11-22 15:49:19 <- (:info index :status running :processed 19000 :updated 2)
2016-11-22 15:49:21 * Received 62 byte(s)
2016-11-22 15:49:21 <- (:info index :status running :processed 20000 :updated 2)
2016-11-22 15:49:22 * Received 62 byte(s)
2016-11-22 15:49:22 <- (:info index :status running :processed 21000 :updated 2)
2016-11-22 15:49:23 * Received 62 byte(s)
2016-11-22 15:49:23 <- (:info index :status running :processed 22000 :updated 2)
2016-11-22 15:49:25 * Received 62 byte(s)
2016-11-22 15:49:25 <- (:info index :status running :processed 23000 :updated 2)
2016-11-22 15:49:26 * Received 62 byte(s)
2016-11-22 15:49:26 <- (:info index :status running :processed 24000 :updated 2)
2016-11-22 15:49:28 * Received 62 byte(s)
2016-11-22 15:49:28 <- (:info index :status running :processed 25000 :updated 2)
2016-11-22 15:49:30 * Received 62 byte(s)
2016-11-22 15:49:30 <- (:info index :status running :processed 26000 :updated 2)
2016-11-22 15:49:31 * Received 62 byte(s)
2016-11-22 15:49:31 <- (:info index :status running :processed 27000 :updated 2)
2016-11-22 15:49:31 * Received 62 byte(s)
2016-11-22 15:49:31 <- (:info index :status running :processed 28000 :updated 2)
2016-11-22 15:50:15 * Received 77 byte(s)
2016-11-22 15:50:15 <- (:info index :status complete :processed 28225 :updated 2 :cleaned-up 0)

Cheers,
--
3S

[Dirk-Jan C. Binnema]

On Tuesday Nov 22 2016, trois singes wrote:

> Hi everybody,
>
> I'm using mu 0.9.16 on MacOS (10.11.6), with Emacs 25.1.1. Sometimes,
> updating the index of my 28000 messages takes 1 second, sometimes it is far
> slower (20 seconds). Nothing else seems to be using the processor.
>
> I noticed that when the index is fast, further indexes will be fast too.
>
> Do you know how I could get rid of the slow updates?

With a slightly newer mu/mu4e (the development version, 0.9.17, which
should soon be crowned the next release 0.9.18), you can try:

(setq
mu4e-index-cleanup nil ;; don't do a full cleanup check
mu4e-index-lazy-check t) ;; don't consider up-to-date dirs

which should make indexing quite a bit faster (see the manual for the
trade-offs). If you're still seeing slow indexing with that, please file
an issue in github with all details you can provide, thanks!

Kind regards,
Dirk.

--
Dirk-Jan C. Binnema Helsinki, Finland
e:dj...@djcbsoftware.nl w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C

[Marcin Borkowski]

On 2016-11-22, at 21:38, Dirk-Jan C. Binnema <dj...@djcbsoftware.nl> wrote:

> (setq
> mu4e-index-cleanup nil ;; don't do a full cleanup check
> mu4e-index-lazy-check t) ;; don't consider up-to-date dirs

BTW, there's a typo in mu4e-index-cleanup's docstring. Instead of
"face" it should spell "phase".

Best, and thanks again for mu/mu4e!

--
Marcin Borkowski

[Joost Kremers]

On Wed, Nov 23 2016, Marcin Borkowski wrote:
> BTW, there's a typo in mu4e-index-cleanup's docstring. Instead
> of
> "face" it should spell "phase".

Actually, that's not the only typo in the doc string. :-) There's
a "the" that should be "there" in the next sentence.

> Best, and thanks again for mu/mu4e!

Hear, hear!

--
Joost Kremers
Life has its moments