Mu4e and Gmail, my current setup

[Magnus Therning]

Hi all,

This is just my attempt to document some of my experience with using
/mu4e/ on a Gmail account. Hopefully it'll be useful to someone. It'd be
even better if it results in me getting suggestions for improving my
setup :)

At first I configured /offlineimap/ to sync all my mail folders, i.e. I
had a folder for each Gmail label on top of the basic ones (Inbox, All
Mail, Drafts, etc.). This quickly led to strange situations where /mu4e/
would get confused about what was read, what was tagged, and so on. It
took a while for me to realise that all of my irritation was caused by
Gmail serving up duplicate emails. It's a strange thing Google does, so
it's not too surprising that /mu4e/ doesn't cope that well with having
multiple copies of the same email in many places. It doesn't manage to
execute its actions on all copies. Leaving me irritated with read emails
turning up as unread, deleted emails as undeleted, etc.

So, I thought I'd try the obvious approach to this problem: I'll avoid
having copies.

I removed all traces of my previous use of /offlineimap/ and /mu/ and
change my config to only sync the following four folders:

- All Mail
- Bin
- Drafts
- Spam

#+BEGIN_EXAMPLE
[Repository HomeRemote]
type = Gmail
remoteuser = XXX
remotepass = XXX
folderfilter = lambda folder: folder.startswith('[Gmail]/') and \
folder[8:] in ['All Mail', 'Bin', 'Drafts', 'Spam']
trashfolder = [Gmail]/Bin
maxconnections = 3
realdelete = no
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
#+END_EXAMPLE

I've then set /mu4e/ use those four folders like this

#+BEGIN_SRC elisp
(setq mu4e-sent-folder "/[Gmail].All Mail"
mu4e-drafts-folder "/[Gmail].Drafts"
mu4e-trash-folder "/[Gmail].Bin"
mu4e-refile-folder "/[Gmail].All Mail")
#+END_SRC

I now don't have an =INBOX= as such instead I've defined a few handy
bookmarks

#+BEGIN_SRC elisp
(setq mu4e-bookmarks `(("\\\\Inbox" "Inbox" ?i)
("flag:flagged" "Flagged messages" ?f)
(,(concat "flag:unread AND "
"NOT flag:trashed AND "
"NOT maildir:/[Gmail].Spam AND "
"NOT maildir:/[Gmail].Bin")
"Unread messages" ?u)))
#+END_SRC

The remaining problem is to propagate to Gmail that I've refiled
(archived) a message. I also noticed that flagging a message in /mu4e/
doesn't immediately mean the message is starred on the Gmail side. With
the recently added ~mu4e-mark-execute-pre-hook~ it becomes possible to
whip Gmail into doing my bidding :)

#+BEGIN_SRC elisp
(add-hook 'mu4e-mark-execute-pre-hook
(lambda (mark msg)
(cond ((member mark '(refile trash)) (mu4e-action-retag-message msg "-\\Inbox"))
((equal mark 'flag) (mu4e-action-retag-message msg "\\Starred"))
((equal mark 'unflag) (mu4e-action-retag-message msg "-\\Starred")))))
#+END_SRC

I've been using this for about a week now and I'm rather liking this setup so far.

It wasn't until after a couple of days that I realised this makes /mu4e/
more like Gmail than it was with me first setup, i.e. all email is in a
single folder and I use searching exclusively to find stuff :-)

/M

--
Magnus Therning OpenPGP: 0x927912051716CE39
email: mag...@therning.org jabber: mag...@therning.org
twitter: magthe http://therning.org/magnus

The ultimate goal of all computer science is the program. The
performance of programs was once the noblest function of computer
science, and computer science was indispensable to great programs.
Today, programming and computer science exist in complacent isolation,
and can be [rescued only] by conscious coöperation and collaboration
of all programmers.

[Nan Zhong]

Thanks for sharing this! I've been using mu4e with gmail for a while now and reading about your setup has definitely made my experience better. Previously, I had been going about in the opposite direction where I've set up bookmarks to ignore the "All Mail" folder and only search the other folders, but I was still syncing and indexing the "All Mail" folder so that I could search all my mail.

After realizing that I never actually use my other folders (INBOX, etc.) and solely rely on my bookmarks, your approach seemed much more sensible. Switching over to your approach has also drastically reduced the time of my syncs, since essentially half of the mail no longer gets synced.

Just wanted to share my appreciation for your description of your setup.

[zeltak brisbane]

Thx magnus

i also tried your setup and it seems to work well for me apart from two weird issues. deleted items dont seem to be deleted in the gmail app and the bookmark "inbox" (bi) leads to a view with "no matching messages found".

any clue?

thx

Z

[Nan Zhong]

I was able to solve these issues by ensuring that Magnus' 'mu4e-mark-execute-pre-hook was set, and that I was using the latest HEAD build of mu since that hook doesn't seem to exist in the latest release.

You might also need to make sure your config for your imap syncer is correctly syncing the gmail labels. For offlineimap, it would be:

synclabels = yes
labelsheader = X-Keywords

configured in your account section (but I do believe these are the default values).

Hope this helps.

[zeltak brisbane]

thx i added the  hook and the config options but still, after i delete and execute, sync with offline imap the deleted messages come back both in mu4e and gmail

any clue how i can debug this
thx

z

[Magnus Therning]

On 8 Oct 2016 9:27 a.m., "zeltak brisbane" <ikl...@gmail.com> wrote:
>
> thx i added the  hook and the config options but still, after i delete and execute, sync with offline imap the deleted messages come back both in mu4e and gmail
>
> any clue how i can debug this
> thx

I noticed the same after a recent upgrade :(

I haven't had time to look into it yet. I'm no wizard at elisp so I suspect I'll end doing the equivalent of printf-debugging.

/M

[Dirk-Jan C. Binnema]

I'd recommend isolating the problem -- ie., what happens when you
manually move a message (using 'mv'), then resync (with offlineimap).

Offlineimap / mbsync have a few idiosyncracies; mu4e tries to work
around those a bit, but I wouldn't be surprised if there were cases
where it doesn't work. In any case, isolating the problems would help.

Kind regards,
Dirk.

--
Dirk-Jan C. Binnema Helsinki, Finland
e:dj...@djcbsoftware.nl w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C

[Magnus Therning]

Magnus Therning <mag...@therning.org> writes:

> On 11 October 2016 at 22:35, Dirk-Jan C. Binnema <dj...@djcbsoftware.nl> wrote:
>>
>> On Tuesday Oct 11 2016, Magnus Therning wrote:
>>
>>> On 8 Oct 2016 9:27 a.m., "zeltak brisbane" <ikl...@gmail.com> wrote:
>>>>
>>>> thx i added the hook and the config options but still, after i delete
>>> and execute, sync with offline imap the deleted messages come back both in
>>> mu4e and gmail
>>>>
>>>> any clue how i can debug this
>>>> thx
>>>
>>> I noticed the same after a recent upgrade :(
>>>
>>> I haven't had time to look into it yet. I'm no wizard at elisp so I suspect
>>> I'll end doing the equivalent of printf-debugging.
>>
>> I'd recommend isolating the problem -- ie., what happens when you
>> manually move a message (using 'mv'), then resync (with offlineimap).
>>
>> Offlineimap / mbsync have a few idiosyncracies; mu4e tries to work
>> around those a bit, but I wouldn't be surprised if there were cases
>> where it doesn't work. In any case, isolating the problems would help.
>

> I still haven't had time to look at this in detail, but in the past
> the issue has always been around tag/labels. For example a local 'mv'
> to the rubbish bin, without a related change in labels, will result in
> the email re-appearing from the rubbish after two synchronizations
> with offlineimap. I place the blame squarely on Google for
> implementing their IMAP so strangely (IMAP *has* the concept of
> labels, but they've decided to not use it), and all I can hope is that
> mu/mu4e offers the functionality to make

I've had a quick look at it seems that my `mu4e-mark-execute-pre-hook`
isn't working as it used to. Currently it's set to the following

~~~
((lambda (mark msg)
(cond ((equal mark 'refile) (mu4e-action-retag-message msg "-\\Inbox"))
((equal mark 'trash) (mu4e-action-retag-message msg "-\\Inbox -\\Starred"))
((equal mark 'flag) (mu4e-action-retag-message msg "-\\Inbox \\Starred"))

((equal mark 'unflag) (mu4e-action-retag-message msg "-\\Starred")))))

~~~

When deleting the \Inbox tag isn't removed any longer. How can I check
if `mu4e-mark-execute-pre-hook` is run at all?

/M

--
Magnus Therning OpenPGP: 0x927912051716CE39
email: mag...@therning.org jabber: mag...@therning.org
twitter: magthe http://therning.org/magnus

Computer Science: "In low-level languages like C"
Computer Engineering: "In high-level languages like C"

[Dirk-Jan C. Binnema]

On Thursday Oct 20 2016, Magnus Therning wrote:

> ~~~
> ((lambda (mark msg)
> (cond ((equal mark 'refile) (mu4e-action-retag-message msg "-\\Inbox"))
> ((equal mark 'trash) (mu4e-action-retag-message msg "-\\Inbox -\\Starred"))
> ((equal mark 'flag) (mu4e-action-retag-message msg "-\\Inbox \\Starred"))
> ((equal mark 'unflag) (mu4e-action-retag-message msg "-\\Starred")))))
> ~~~
>
> When deleting the \Inbox tag isn't removed any longer. How can I check
> if `mu4e-mark-execute-pre-hook` is run at all?

You can add another function to that hook, e.g.
(message "Captain Hook")

[Magnus Therning]

Dirk-Jan C. Binnema <dj...@djcbsoftware.nl> writes:

> On Thursday Oct 20 2016, Magnus Therning wrote:
>
>> ~~~
>> ((lambda (mark msg)
>> (cond ((equal mark 'refile) (mu4e-action-retag-message msg "-\\Inbox"))
>> ((equal mark 'trash) (mu4e-action-retag-message msg "-\\Inbox -\\Starred"))
>> ((equal mark 'flag) (mu4e-action-retag-message msg "-\\Inbox \\Starred"))
>> ((equal mark 'unflag) (mu4e-action-retag-message msg "-\\Starred")))))
>> ~~~
>>
>> When deleting the \Inbox tag isn't removed any longer. How can I check
>> if `mu4e-mark-execute-pre-hook` is run at all?
>
> You can add another function to that hook, e.g.
> (message "Captain Hook")

Thanks for the help.

It showed that the hook is indeed run as it should. However, when I read
the documentation on `mu4e-action-retag-message` I found that the tags
should be comma separated (earlier it worked fine with space
separation). So now that problem is solved :)

/M

--
Magnus Therning OpenPGP: 0x927912051716CE39
email: mag...@therning.org jabber: mag...@therning.org
twitter: magthe http://therning.org/magnus

I have steadily endeavored to keep my mind free, so as to give up any
hypothesis, however much beloved — and I cannot resist forming one
on every subject — as soon as facts are shown to be opposed to it.
— Charles Darwin (1809-1882)

[zeltak brisbane]

Hi Magnus

can you share your revised code with the new tag separated code?

thx

Z

[Magnus Therning]

zeltak brisbane <ikl...@gmail.com> writes:

> Hi Magnus
>
> can you share your revised code with the new tag separated code?

Of course, just separate the tags using commas (',') like this:

~~~
(add-hook 'mu4e-mark-execute-pre-hook

(lambda (mark msg)
(cond ((equal mark 'refile) (mu4e-action-retag-message msg "-\\Inbox"))

((equal mark 'trash) (mu4e-action-retag-message msg "-\\Inbox,-\\Starred"))
((equal mark 'flag) (mu4e-action-retag-message msg "-\\Inbox,\\Starred"))

((equal mark 'unflag) (mu4e-action-retag-message msg "-\\Starred")))))
~~~

/M

--
Magnus Therning OpenPGP: 0x927912051716CE39
email: mag...@therning.org jabber: mag...@therning.org
twitter: magthe http://therning.org/magnus

You do not examine legislation in the light of the benefits it will
convey if properly administered, but in the light of the wrongs it
would do and the harms it would cause if improperly administered.
— Lyndon Johnson

[zeltak brisbane]

Hi

ok so i did all the above but i *Still* cant get it to delete. i use ('D') to mark files for deletion,  then use x to execute and update with U. i can see offlineimap and sync but the email is still in gmail. what am i missing here?

thx

Z

[Jacek Generowicz]

zeltak brisbane <ikl...@gmail.com> writes:

> Hi
>

> ok so i did all the above but i *Still* cant get it to delete. i use ('D')
> to mark files for deletion, then use x to execute and update with U. i can
> see offlineimap and sync but the email is still in gmail. what am i missing
> here?

I forget the exact details, but I think it goes something like this:

+ Gmail notices that any message you delete has mysteriously disappeared
and helpfully puts it back for you.

+ To get gmail to remove a message, you have to *trash* it (using 'd'
rather than 'D').

+ But this still leaves the message in the Googlemail/[Google Mail].*
folder which is a union of *all* messages gmail knows about, including
the trashed ones in Googlemail/[Trashed].

+ So I have mu4e bookmarks which exclude Googlemail/[Google Mail].* to
stop me from seeing the trashed messages.

+ Gmail eventually *really* gets rid of trashed messages, after a month,
I think. Chances are you can change this in your Gmail settings.

This approach was 'good enough for me' when I first encountered this
problem a number of years ago, and I haven't thought about it since
because I really don't care about and don't want to use gmail at all,
though some mailing lists (ahem) that do interest me essentially forced
me into it.

If you are a serious gmail user, then what works for me might be
unsatisfactory for you.

[beo...@gmail.com]

I think the problem could be on the IMAP application side. I recall that
I had a similar problem until I had mbsync/isync sync the Bin[/Trash]
directory first before any of the other directories.

> > email: mag...@therning.org <javascript:> jabber: mag...@therning.org
> > <javascript:>

> > twitter: magthe http://therning.org/magnus
> >
> > You do not examine legislation in the light of the benefits it will
> > convey if properly administered, but in the light of the wrongs it
> > would do and the harms it would cause if improperly administered.
> > — Lyndon Johnson
> >

> --
> You received this message because you are subscribed to the Google Groups "mu-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to mu-discuss+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


--
Benjamin Slade
`(pgp_fp: ,(21BA 2AE1 28F6 DF36 110A 0E9C A320 BBE8 2B52 EE19))
'(sent by mu4e on Emacs running under GNU/Linux . https://gnu.org )
'(Choose Linux, Choose Freedom . https://linux.com )

[Magnus Therning]

'Jacek Generowicz' via mu-discuss <mu-di...@googlegroups.com> writes:

> zeltak brisbane <ikl...@gmail.com> writes:
>
>> Hi
>>
>> ok so i did all the above but i *Still* cant get it to delete. i use
>> ('D') to mark files for deletion, then use x to execute and update
>> with U. i can see offlineimap and sync but the email is still in
>> gmail. what am i missing here?
>
> I forget the exact details, but I think it goes something like this:
>
> + Gmail notices that any message you delete has mysteriously
> disappeared and helpfully puts it back for you.
>
> + To get gmail to remove a message, you have to *trash* it (using 'd'
> rather than 'D').

Yes, that's the key thing. With my settings 'd' will put the mail in
[Gmail].Bin (I'm using English as opposed to American ;) and set tags
properly for Gmail to move the mail to the bin.

> + But this still leaves the message in the Googlemail/[Google Mail].*
> folder which is a union of *all* messages gmail knows about,
> including the trashed ones in Googlemail/[Trashed].
>
> + So I have mu4e bookmarks which exclude Googlemail/[Google Mail].* to
> stop me from seeing the trashed messages.

My bookmarks look like this

~~~

mu4e-bookmarks `(("\\\\Inbox" "Inbox" ?i)
("flag:flagged" "Flagged messages" ?f)
(,(concat "flag:unread AND "
"NOT flag:trashed AND "
"NOT maildir:/[Gmail].Spam AND "
"NOT maildir:/[Gmail].Bin")
"Unread messages" ?u))

~~~

/M

--
Magnus Therning OpenPGP: 0x927912051716CE39
email: mag...@therning.org jabber: mag...@therning.org
twitter: magthe http://therning.org/magnus

In corporate religions as in others, the heretic must be cast out not
because of the probability that he is wrong but because of the
possibility that he is right.
— Antony Jay

[d...@ddoherty.net]

In case anyone is stumbles upon this thread, I found that putting the commas in breaks the

hook in recent versions (0.9.17 for me) of mu/mu4e.  Just separate with spaces and it works again.

[Marc Croteau]

Hi Magnus,

I've just started using mu4e after a long hiatus.

I've set up my system as you suggest. I have four sync'ed folders: All Mail, Spam, Drafts, and Trash (your 'Bin).

I don't have an 'Inbox'.

I'm curious to know what the first line in your bookmarks setting:

               mu4e-bookmarks `(("\\\\Inbox" "Inbox" ?i)

is doing?

In this setup, "bi" will be set to look in an Inbox that doesn't exist.

When I have set my bookmarks like you, I see 'flagged' and 'unread" but never anything in response to 'bi' command.

I apologize for going back to 2016 for this but hope you can still help.

Thank you in advance

Marc

[Paco]

Hi,

Is this still relevant? I'm just asking since the thread is a bit old, but if so, is it possible to have multiple context with this setup? how do you create bookmarks for an specific context?

Thanks!