mu4e locks emacs when updating (mu4e update not asynchronous)

[pow...@gmail.com]

Greetings,

I have been fighting an extremely annoying issue, and I haven't been able to work-around it.

Basically, when mu4e updates, it locks everything up (I can't type, etc.) and it displays "Searching..." in the message view.

I have the following variables set:

        mu4e-get-mail-command true

        mu4e-update-interval 60

        mu4e-index-cleanup nil

        mu4e-index-lazy-check t

        mu4e-index-update-in-background t

My setup is to have mbsync executed outside of emacs to sync with the servers every 10 minutes. This works fine. All I want is mu4e to update its status with the mbsync data. Right now, for testing this issue, it checks every minute. Of course, only at 10 minute intervals might something have actually changed.

However, every minute, the header view (usually) switches to "Searching..." and for ~1-5 seconds, emacs is synchronously blocked. This is incredibly frustrating while typing messages, etc. Typically, the 9 updates between mbsync updates result in only a very minor synchronous block. 

This with mu4e 1.0 C and is on GNU emacs 26.0.91 on Mac OSX compiled from homebrew. I built mu and used mu4 from HEAD-7d6c30f on Feb. 4.

Any ideas as to why and how to fix this? I'm not sure what else to check. I appreciate your help.

Cheers,

Brian

[Eduardo Mercovich]

Hi Brian.

> [...] when mu4e updates, it locks everything up (I can't type,

> etc.) and it displays "Searching..." in the message view.

I have a 20 Gb maildir with ~180.000 mails and the lock when
indexing never tooks more than a second or so (a small % is marked
as not indexable anymore since they are old accounts).

How big is your maildir?

> [...] Any ideas as to why and how to fix this? I'm not sure what
> else to check.

I don't know neither, but just to measure the impact in seconds of
your indexing, try to mark some maildir directories with a
.noupdate and see if it is reduced.
It's a simple check, costs nothing and can give us a glimpse if is
the real indexing or there is something else there...


--
Eduardo Mercovich

Donde se cruzan tus talentos
con las necesidades del mundo,
ahí está tu vocación.
(Anónimo)

[pow...@gmail.com]

Eduardo, thank you for your response. I have been trying a variety of tests and scenarios, and after a week of testing, I am no closer to a resolution.

My maildir is over 30Gb with over 200k messages.

I ran a variety of tests from the command line:

% for i in {1..10}; do time mu index --quiet --maildir=mail --lazy-check --nocleanup; sleep 10 ; done

The mean values were:

real 0m0.799s

user 0m0.665s

sys 0m0.085s

So, on the command-line it takes very little time to update.

Unfortunately, on the mu4e side, the issue remains. I reconfigured my setup to look for mail storage optimizations, but that didn't really make any difference. I tried using .noupdate files, but with the update being so fast (as per above), it only served to make the system unworkable as messages went missing, etc.

I am using mu from git ID starting with 7d6c30f.

I tried a number of lisp issues to get to the bottom of what is happening. I used the profiler extensively, but it didn't seem to provide any helpful information on why it was blocking. I have byte-compiled the mu4e directory to see if that improves things, but there wasn't a change.

Basically, the issue remains: on the mu4e-update-interval time, the header view is replaced with "Loading..." for 2-5 seconds, and I cannot edit, etc. in emacs. I have explicitly set it not to call mbsync on update with lazy-check and no cleanup (as per the command-line above) as shown below. 

The other odd behavior that has started and drives me crazy: the first time a view a message, it does not use auto-fill-mode or show the images despite their being enabled. If I then review the same image, it uses auto-fill-mode and displays the images...

Cheers,

Brian

  (setq mu4e-maildir (expand-file-name "~/mail")

        mu4e-get-mail-command 'true

        mu4e-mu-binary "/usr/local/bin/mu"

        mu4e-update-interval 300

        mu4e-html-renderer 'w3m

        mu4e-html2text-command 'mu4e-shr2text

        mu4e-view-html-plaintext-ratio-heuristic 99999

        mu4e-change-filenames-when-moving t

        mu4e-sent-messages-behavior 'delete

        mu4e-use-fancy-chars nil

        mu4e-headers-unread-mark (purecopy '("u" . "✉"))

        mu4e-view-show-images t

        mu4e-hide-index-messages t

        mu4e-show-images t

        mu4e-view-image-max-width 800

        mu4e-view-image-max-height 800

        mu4e-attachment-dir  "~/Downloads"

        mu4e-headers-leave-behavior 'apply

        mu4e-headers-skip-duplicates t

        mu4e-headers-visible-lines 15

        mu4e-headers-full-search nil

        mu4e-headers-include-related nil

        mu4e-headers-results-limit 600

        mu4e-headers-auto-update t

        mu4e-sent-messages-behavior 'delete

        mu4e-index-cleanup nil

        mu4e-index-lazy-check t

        message-citation-line-function nil

        message-kill-buffer-on-exit t

        mu4e-split-view 'horizontal

        mu4e-compose-dont-reply-to-self t

        mu4e-compose-signature-auto-include nil

        mu4e-compose-context-policy 'ask-if-none

        mu4e-context-policy 'pick-first

        mu4e-index-update-in-background t

        mu4e-confirm-quit nil

        gnus-dired-mail-mode 'mu4e-user-agent

        mu4e-view-show-addresses t

        mail-from-style 'angles

        url-mail-command 'compose-mail

        mail-user-agent 'mu4e-user-agent)

[Eduardo Mercovich]

Hi.

> Eduardo, thank you for your response. I have been trying a
> variety of tests and scenarios, and after a week of testing, I

> am no closer to a resolution. [...]

Sadly, I also can't help much more... :(

Anyone else can jump to this issue with other clues?

[Dirk-Jan C. Binnema]

> (setq mu4e-maildir (expand-file-name "~/mail")
> mu4e-get-mail-command 'true

^^
"true" (which also the default, see
the docstring).
Kind regards,
Dirk.

--
Dirk-Jan C. Binnema Helsinki, Finland
e:dj...@djcbsoftware.nl w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C

[pow...@gmail.com]

Thank you Dirk. I actually don't set it, I added that blurb when I pasted it in for "clarity" but I failed. The truth is that I leave the value at the default...

e:d...@djcbsoftware.nl           w:www.djcbsoftware.nl