Notmuch vs MU

[Samuel Loury]

Hi.

I am a regular user of notmuch and I just discovered mu. It seems to me
they are both very similar tools: mail indexers with command line tools to
control them.

It appears that notmuch started in 2010 while mu started in 2011.

I then wonder why the mu project came to life? I mean, why creating a new
project instead of contributing to notmuch? Are they features not provided by
notmuch that mu provides? Or are they dedicated to different categories of
users? Looking at the http://www.djcbsoftware.nl/code/mu/ page, I do not
understand how both project differ. Could you explain me?

I don't want to launch any kind of troll, I am a user wanting to read mails in
emacs that found out a new way of doing that and wanting to understand why it
could be different.

Thank you for your attention.
--
Konubinix

[Abdó Roig]

I then wonder why the mu project came to life? I mean, why creating a new 
project instead of contributing to notmuch? Are they features not provided by 
notmuch that mu provides?

Hi,

I am a notmuch user that recently moved to mu/mu4e. Here are my reasons:

1) The emacs client for mu is quite different from the notmuch one. For some reason I never felt comfortable with notmuch. mu4e fits my workflow much better. This is an entirely personal matter I guess. Moreover the mu4e code is very clean and well structured, and offers a lot of extension potential. After having gone through gnus and notmuch, configuring mu4e the way I wanted was a pleasure.

However to be fair there are some features I had in notmuch that are not yet available for mu: identities, message threading, unread counts on the main view...

2) mu indexes email addresses from the messages, providing autocompletion out of the box. For me, this almost renders email databases like bbdb unnecessary. 

3) mu supports tagging messages via an extra header on the messages (X-Keywords), which fits better with my setup than what notmuch does, storing tags entirely in the database. A downside of this is that I had to write my little script to change tags, as mu only indexes / searches them.

4) A consequence of 2 is that all the metadata for messages can be recovered from the Maildir alone. This makes it much easier syncing the Maildir between different machines without losing data (in my case home, work and laptop). My setup with notmuch was a nightmare...

Abdó.

[Mandar Mitra]

Samuel Loury wrote (Mon, Oct 29, 2012 at 01:49:00PM +0100):
> I don't want to launch any kind of troll, I am a user wanting to read mails in
> emacs that found out a new way of doing that and wanting to understand why it
> could be different.

Many thanks for asking this question. I'm new to both notmuch and mu4e, and
this discussion is bound to be useful to others like me, who're looking for
the "best" alternative.

0. I currently use mutt running in a multi-term within emacs, with
emacsclient as my pager (customised view-mode) + editor (customised
message-mode). I've been using mutt for many years, so I'm comfortable
with this setup.

Recently, I've added both mu and notmuch-emacs to my setup: mu for
searching from within mutt, and mu-cfind + notmuch-address for tab
completion of addresses. I'm sure this can be streamlined.

Main drawbacks: haven't been able to figure out how to make mutt +
offlineimap play well with Gmail's labels / folders.


1. I tried notmuch, and was able to set it up to be fairly close to the
mutt interface that I'm accustomed to, but I found it slow. Far too
often, I was having to wait longer than I would like for notmuch to
respond. Also, deleting / moving email requires an additional program
called afew.


2. I've just started trying mu4e. Things I haven't properly explored yet:
- whether it's possible to sort threads by most recent message received
- whether it's possible to sort threads / messages by time of receipt
- whether it's possible to save messages to mbox files that I have had
for a long time (e.g. ~/courses/xyz/mail), or to maildirs that are
outside of the directory hierarchy known to mu
- ...

But from the discussion on the mailing list, and Abdo Roig's patches for
mu4e + Offlineimap, it seems like mu4e is the way to go as far as
integration with Gmail + Offlineimap is concerned.

Hope others contribute to this discussion. If this turns out to be a
comprehensive comparison, perhaps djcb could include a link ("notmuch vs.
mu4e") from the website?

Thanks,
Mandar.

[Joost Kremers]

[changed subject because this isn't a contribution to the mu vs. notmuch debate)

On Mon, Oct 29 2012, Mandar Mitra <mandar...@gmail.com> wrote:
> - whether it's possible to save messages to mbox files that I have had
> for a long time (e.g. ~/courses/xyz/mail), or to maildirs that are
> outside of the directory hierarchy known to mu

there is a program called archivemail that can handle, well, archiving
old mail. it'll move messages to archive folders based on any number of
criteria, for example on how old it is. (in fact, i assume there are
other such projects, but archivemail is the one i have experience with.)


--
Joost Kremers
Life has its moments

[Mandar Mitra]

archivemail seems to be primarily date driven. I was thinking of
moving the current message. I don't know if you've ever used mutt,
but I had something like mutt's save-message function in mind. Usually,
one types 's' and mutt asks for the name of a maildir or mbox file, and
moves the current message to the specified mbox / maildir.

I suppose refiling is alternative parlance for this, but I was thinking
of actually moving a message to a different mailbox/dir instead of
simply changing labels.

mandar.

[Dirk-Jan C. Binnema]

Hi Samuel,

On Mon, Oct 29 2012, Samuel Loury wrote:

> Hi.
>
> I am a regular user of notmuch and I just discovered mu. It seems to me
> they are both very similar tools: mail indexers with command line tools to
> control them.
>
> It appears that notmuch started in 2010 while mu started in 2011.

> I then wonder why the mu project came to life? I mean, why creating a new
> project instead of contributing to notmuch? Are they features not provided by
> notmuch that mu provides? Or are they dedicated to different categories of
> users? Looking at the http://www.djcbsoftware.nl/code/mu/ page, I do not
> understand how both project differ. Could you explain me?

mu actually started in 2008; only the emacs frontend is from 2011. These
frontends take a quite different approach.

For more details, see e.g.
http://emacs-fu.blogspot.fi/2012/08/introducing-mu4e-for-email.html
or read the mu4e manual.

> I don't want to launch any kind of troll, I am a user wanting to read mails in
> emacs that found out a new way of doing that and wanting to understand why it
> could be different.

Well, just try both and see which one you like best.

Best wishes,
Dirk.

--
Dirk-Jan C. Binnema Helsinki, Finland
e:dj...@djcbsoftware.nl w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C

[Dirk-Jan C. Binnema]

If it's about hiding from mu indexing, you can use '.noindex' and/or
'.noupdate'; see the mu-index man page.

[Jason Lewis]

There is another tool that I use called cleanup-maildir

http://svn.houseofnate.net/unix-tools/trunk/cleanup-maildir

of which there are a few forks in github.

It will allow you to save your mails to wherever you like. Runs nicely
from cron.

Jason

--
Jason Lewis
http://emacstragic.net

[Stephen Eglen]

I looked at both notmuch and mu4e before deciding on mu4e.  The key thing I found missing from notmuch was how to reliably synchronise tags across different machines.  This was a year ago (roughly), has that been solved yet?

I do like the idea of tags, but the search of mu4e is powerful enough that I don't find I really need tags much.

Also, perhaps another thing I prefer is that you can easily follow and contribute to mu4e development, as Dirk is so responsive.  By contrast, notmuch is a bigger project, and it has many people contributing, but that seems to entail lots of discussion on each patch, which makes it less interesting for the outsider to follow on the mailing list.

Stephen

[Mandar Mitra]

Stephen Eglen wrote (Tue, Oct 30, 2012 at 03:08:16PM -0700):
> Also, perhaps another thing I prefer is that you can easily follow and
> contribute to mu4e development, as Dirk is so responsive. By contrast,
> notmuch is a bigger project, and it has many people contributing, but that
> seems to entail lots of discussion on each patch, which makes it less
> interesting for the outsider to follow on the mailing list.

Also, for those seeking to migrate to mu4e from mutt, it's reassuring to
know that Dirk was also a mutt user for a very long time.


mandar.

[Samuel Loury]

Thank you for your answer.

> 3) mu supports tagging messages via an extra header on the messages
> (X-Keywords), which fits better with my setup than what notmuch does,
> storing tags entirely in the database. A downside of this is that I had to
> write my little script to change tags, as mu only indexes / searches them.

This is in fact very interesting. It is THE FEATURE that makes me
hesitate between notmuch and mu.

> 4) A consequence of 2 is that all the metadata for messages can be
> recovered from the Maildir alone. This makes it much easier syncing the
> Maildir between different machines without losing data (in my case home,
> work and laptop). My setup with notmuch was a nightmare...

I guess you mean a consequence of 3.

How do you manage to make the synchronization between your machines? I
use offlineimap to synchronize my mails via my gmail account and it
looks like it does not update a mail when it has been edited.

To test that, I manually edited a mail and added the header "X-Flag:
test" into it. After running offlineimap, I looked at the mail into my
gmail account and the modification was not there...

Regards

--
Konubinix

[Abdó Roig-Maranges]

Hi,

>> 4) A consequence of 2 is that all the metadata for messages can be
>> recovered from the Maildir alone. This makes it much easier syncing the
>> Maildir between different machines without losing data (in my case home,
>> work and laptop). My setup with notmuch was a nightmare...
>

> I guess you mean a consequence of 3.

Yes :)

> How do you manage to make the synchronization between your machines? I
> use offlineimap to synchronize my mails via my gmail account and it
> looks like it does not update a mail when it has been edited.

That's right. I use a patched offlineimap to keep tags in sync with
gmail labels, as if they were flags. This takes care of syncing tags
with gmail, letting me use them on the web interface, mobile, etc.

http://permalink.gmane.org/gmane.mail.imap.offlineimap.general/5943

Then I sync data between my machines (including the maildir) using
unison.

http://www.cis.upenn.edu/~bcpierce/unison/

Technically, to sync tags between multiple machines, you don't need
unison. With the patched offlineimap you get it, but as I already sync
everything else with unison, I use it with the maildir too. The patched
offlineimap is responsible to sync with gmail only.

That I can sync the maildirs with unison without losing data, relies on
the fact that everything mu stores in the database comes from the
messages. This way, I can sync maildirs without syncing the mu database
(the db is too large to be synced constantly), and then reindexing new
messages on every machine. This I could not do with notmuch, as tags
lived only on the database.

Abdó.

[Samuel Loury]

First of all, I would like to thank you all for your quick answers.

> I looked at both notmuch and mu4e before deciding on mu4e. The key thing I
> found missing from notmuch was how to reliably synchronise tags across
> different machines. This was a year ago (roughly), has that been solved
> yet?

In fact, I guess we should tags into two classes:
1. tags that we can compute from the content of a mail like
mailing_list, sent, rss, work_mail ...
2. tags that we cannot compute from the content of the mail like
read, replied, spam...

When wanting to synchronize mails between several computers, I like the
idea of using an outside IMAP server and use offlineimap to synchronize
it with a local maildir. In fact, I don't really know other ways to do
that. I will then try to sum up how to synchronize tags with the
offlineimap + IMAP server and the problem each solution brings.

Abdó mentioned also a way using a folder synchronization tool such as
unison to synchronize mails. This assumes the user has access to both
computer at the same time or that he uses a usb stick to store the
mails. For that reason, this method does not quite fit with my
situation.

The first category of tags does not really need to be stored into the
mail content because in am able to re create the tags from a new copy of
maildir. Then notmuch and mu can both do that very well.

To synchronize tags from the second part, there are several approaches:
1. insert the tag into the mail headers like mu does. This, however,
needs offlineimap to be patched to work.
2. use the dedicated maildir tags like T, S, F, D... Those are really
limited and a patch in offlineimap should be needed to synchronize
other letters (for instance W for spams) as well. This is described
here
http://lists.gnu.org/archive/html/info-gnus-english/2011-10/msg00052.html
3. use a maildir directory and have offlineimap synchronize those with
imap labels. This causes the mails to be duplicated for each tag it
possesses.
4. Patch offlineimap to make it create hardlinks of mails instead of
copy them.

None of these solutions seems to be perfect. 3 and 4 work with notmuch
and mu but 3 is definitively not good because there are too many
duplications of mails and 4 seems like difficult to implement. 1 work
out of the box with mu but it needs a patch to be applied to
offlineimap. In the case of 2, a monkey patch must be applied in
offlineimap and both notmuch and mu must also be patched to make it
work.

As a conclusion, there does not seem to be any solution out of the box
using mu or notmuch to easily and efficiently synchronize mails. 1 is a
bit easier with mu.

Is there any one with a fifth solution?

> Also, perhaps another thing I prefer is that you can easily follow and
> contribute to mu4e development, as Dirk is so responsive. By contrast,
> notmuch is a bigger project, and it has many people contributing, but that
> seems to entail lots of discussion on each patch, which makes it less
> interesting for the outsider to follow on the mailing list.

I totally agree with this one. I really like the responsiveness and
kindness of Dirk on the mailing list.
--
Konubinix

[Samuel Loury]

Hi,
Your patch is really interesting. Thanks!
--
Konubinix

[Dirk-Jan C. Binnema]

Hi Samuel,

Thanks for the write-up! some comments below.

On Fri, Nov 02 2012, Samuel Loury wrote:

> 2. use the dedicated maildir tags like T, S, F, D... Those are really
> limited and a patch in offlineimap should be needed to synchronize
> other letters (for instance W for spams) as well. This is described
> here
> http://lists.gnu.org/archive/html/info-gnus-english/2011-10/msg00052.html

mu only checks for the Maildir-standard[1] tag-letters; more letters
could be added of course, but I think some agreement should be reached
between Maildir-implementations on their meaning.

[1] http://cr.yp.to/proto/maildir.html

> 3. use a maildir directory and have offlineimap synchronize those with
> imap labels. This causes the mails to be duplicated for each tag it
> possesses.
> 4. Patch offlineimap to make it create hardlinks of mails instead of
> copy them.
>
> None of these solutions seems to be perfect. 3 and 4 work with notmuch
> and mu but 3 is definitively not good because there are too many
> duplications of mails and 4 seems like difficult to implement. 1 work
> out of the box with mu but it needs a patch to be applied to
> offlineimap. In the case of 2, a monkey patch must be applied in
> offlineimap and both notmuch and mu must also be patched to make it
> work.
>
> As a conclusion, there does not seem to be any solution out of the box
> using mu or notmuch to easily and efficiently synchronize mails. 1 is a
> bit easier with mu.

The real problem is that the Gmail <-> IMAP mapping is imperfect. It's
possible to hack around this to some extent, but I think it'll be very
hard to not always have some problematic corner cases.

Personally, at least for some of my accounts, I simply use Gmail-POP,
and don't care about synchronizing changes back to Gmail. At least for
me that works -- of course, I don't get all the tagging/labeling in the
Gmail web interface then, but search seems to be good enough for me.

For some accounts where I do use Gmail/IMAP and offlineimap, as tags I
use the Maildir-flags mentioned above, and for the rest I have folders -
after all, folders are equivalent to single tags. I don't sync the All
Mail folders, so there's no duplication.

So, the problem case is where you want multiple labels per message, and
want to sync those with Gmail.

> Is there any one with a fifth solution?
>
>> Also, perhaps another thing I prefer is that you can easily follow and
>> contribute to mu4e development, as Dirk is so responsive. By contrast,
>> notmuch is a bigger project, and it has many people contributing, but that
>> seems to entail lots of discussion on each patch, which makes it less
>> interesting for the outsider to follow on the mailing list.
>
> I totally agree with this one. I really like the responsiveness and
> kindness of Dirk on the mailing list.

Thanks for the nice words! Hope I can keep it that way.

[Samuel Loury]

Hi,

> For some accounts where I do use Gmail/IMAP and offlineimap, as tags I
> use the Maildir-flags mentioned above, and for the rest I have folders -
> after all, folders are equivalent to single tags. I don't sync the All
> Mail folders, so there's no duplication.

I used to do that also. I encountered unfortunately two problems that
made this solution complicated, they are more related to the
implementation of the solution than the solution itself though:
1. with offlineimap, syncing multiple folders takes a lot of time, I
guess because it has to make a new connection for each folder.
2. Some times, I don't know why, some mails in gmail have their labels
removed. When not syncing "All Mails", we assume that all the mails
not put into a dedicated folder are at least into the folder INBOX. I
recently tried to get to an old mail that would normally be in INBOX
and realized that it was not in my local mails. Looking at the gmail
webmail, I found out this mail was no more into INBOX. Thus, it had
no label associated to it. The only way to retrieve it via
offlineimap was to sync with the "All Mails" folder.

Have you ever experienced some similar problems?

Regards
--
Konubinix
GPG Key: 7439106A

[Dirk-Jan C. Binnema]

Hi Samuel,

On Mon, Nov 05 2012, Samuel Loury wrote:

> Hi,
>
>> For some accounts where I do use Gmail/IMAP and offlineimap, as tags I
>> use the Maildir-flags mentioned above, and for the rest I have folders -
>> after all, folders are equivalent to single tags. I don't sync the All
>> Mail folders, so there's no duplication.
>
> I used to do that also. I encountered unfortunately two problems that
> made this solution complicated, they are more related to the
> implementation of the solution than the solution itself though:
> 1. with offlineimap, syncing multiple folders takes a lot of time, I
> guess because it has to make a new connection for each folder.

Selectively getting folders helps; there are some tricks that can be
done with offlineimap and some python scripts.

> 2. Some times, I don't know why, some mails in gmail have their labels
> removed. When not syncing "All Mails", we assume that all the mails
> not put into a dedicated folder are at least into the folder INBOX. I
> recently tried to get to an old mail that would normally be in INBOX
> and realized that it was not in my local mails. Looking at the gmail
> webmail, I found out this mail was no more into INBOX. Thus, it had
> no label associated to it. The only way to retrieve it via
> offlineimap was to sync with the "All Mails" folder.
>
> Have you ever experienced some similar problems?

Yeah, occasionaly. Not sure if there's much we can do about it. Maybe
some Gmail setting?