With GMail send emails end up in both send mail and in "All Mail"; what am I doing wrong?

[Henkjan Gersen]

Hi,

I'm trying to configure mu4e with isync and msmtp in which the email account is GMail.
The combination works and I can sync my email as well as send email through google smtp.

However when I send email this way I always end up having a copy in both the send emails as well as in my "All Mail" folder on GMail.
As I use the "All Mail" folder to archive message that are important enough to keep I consider this unwanted behavior.

Having searched for a solution I have failed to find a clear pointer that tells me what I am doing wrong.
Can someone here explain how can prevent my own send email from appearing under "All Mail" (and as a result in the local archive folder)?

The software versions I'm using are isync(1.2.1); mu (0.9.15); msmtp (1.6.3) and the relevant bits of my config files are below:

Thanks in advance for looking,
Cheers,
Henkjan

======== mu4e ==========
  (setq mu4e-maildir "~/.mail/family"
        mu4e-trash-folder "/trash"
        mu4e-refile-folder "/archive"
        mu4e-get-mail-command "mbsync -a"
        user-mail-address "..snip..@gmail.com"

        mu4e-sent-messages-behavior 'delete
        )

  ;; use msmtp for mail delivery
  (setq message-send-mail-function 'message-send-mail-with-sendmail
        sendmail-program "/usr/bin/msmtp"
        user-full-name "..snip..")

====== mbsyncrc =======
# Default settingS
Create Slave
Expunge Both
SyncState *

IMAPStore family-remote
Host imap.gmail.com
User ...snip....@gmail.com
PassCmd "/usr/bin/pass Email/Family"
SSLType IMAPS
SSLVersions TLSv1.2
CertificateFile /etc/ssl/certs/ca-certificates.crt

MaildirStore family-local
Path ~/.mail/family/
Inbox ~/.mail/family/INBOX

Channel family-inbox
Master ":family-remote:INBOX"
Slave ":family-local:INBOX"

Channel family-archive
Master ":family-remote:[Gmail]/All Mail"
Slave ":family-local:archive"

Channel family-drafts
Master ":family-remote:[Gmail]/Drafts"
Slave ":family-local:drafts"

Channel family-sent
Master ":family-remote:[Gmail]/Sent Mail"
Slave ":family-local:sent"

Channel family-trash
Master ":family-remote:[Gmail]/Bin"
Slave ":family-local:trash"

Channel family-spam
Master ":family-remote:[Gmail]/Spam"
Slave ":family-local:spam"

Group gmail-family
Channel family-trash
Channel family-inbox
Channel family-drafts
Channel family-sent
Channel family-archive
Channel family-spam

====== msmtp =======
# Set default values for all following accounts.
defaults
auth      on
tls       on

# Set a list of trusted CAs for TLS; using a system-wide default file,
tls_trust_file /etc/ssl/certs/ca-certificates.crt

# GMail Family Account
account   gmail-family
host      smtp.gmail.com
port      587
from      ..snip..@gmail.com
user      ..snip
passwordeval pass Email/Family

# Set a default account
account default : gmail-family

[Dirk-Jan C. Binnema]

Hi Henkjan,

On Saturday Jan 09 2016, Henkjan Gersen wrote:

> Hi,
>
> I'm trying to configure mu4e with isync and msmtp in which the email account is GMail.
> The combination works and I can sync my email as well as send email through google smtp.
>
> However when I send email this way I always end up having a copy in
> both the send emails as well as in my "All Mail" folder on GMail. As
> I use the "All Mail" folder to archive message that are important
> enough to keep I consider this unwanted behavior.
>
> Having searched for a solution I have failed to find a clear pointer
> that tells me what I am doing wrong. Can someone here explain how can
> prevent my own send email from appearing under "All Mail" (and as a
> result in the local archive folder)?
>

> The software versions I'm using are isync(1.2.1); mu (0.9.15); msmtp
> (1.6.3) and the relevant bits of my config files are below:

Part of this is simply the way Gmail works -- somewhat like mu4e, it
doesn't really have folders - instead "All Mail", "Inbox", "Important",
"Sent Mail" etc. are queries - so the same messages can be present in
more than one of those.

GMail's IMAP implementation deals with this by turning these "queries"
into IMAP-folders when you download them, and simply giving each of them
a copy of the messages. Since "All Mail" also contains your sent items,
both "Sent Mail" and "All Mail" end up with a copy of the same message
(and "Sent Mail" may even get two).

So, any attempt to get around this necessarily depends on trying to work
around the way GMail works. One trick is using
`mu4e-sent-messages-behavior', that helps to get rid of one copy, and I
see you're already using that.

If you don't care too much about custom folders, you can use GMail's
POP3 interface and simply use it as an automatic backup, while you can
organize your mails locally as you wish.

Kind regards,
Dirk.

--
Dirk-Jan C. Binnema Helsinki, Finland
e:dj...@djcbsoftware.nl w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C

[Henkjan Gersen]

Thanks for the quick response Dirk, I must admit that I never noticed this GMail behavior before as I typically work with very large mailboxes. However I see that the same thing happens if I send an email using GMail's web-interface itself. So indeed expected behavior. :-(

Two remaining questions if I'm allowed:

1) If I delete messages in GMails' sent folder through their web-interface they disappear immediately from the "all mail" folder in GMail as well. In mu4e I have to delete the messages twice; both in the sent folder as well as in the archive folder (the one corresponding to All Mail). Is there a workaround for this, or is that also behavior that is to be expected due to the difference between labels/folders?

2) When I delete messages in my inbox through mu4e I see them correctly disappear in GMail and the Bin folder on GMail is empty. However these messages remain present in the local trash folder in mu4e with the deleted status. Is there an option in mu4e that I'm forgetting to set to make sure that those messages also get removed locally as syncing the Bin folder with the local trash doesn't seem to do that job.

Note that it might simply be that I misunderstand which folders I actually need to synchronize as I find  the IMAP interface from Google rather confusing.

btw) Using pop3 isn't an option for me as I need access to the same large email archive from multiple computers; so IMAP is the better option for now.

Best regards,
Henkjan

[Dirk-Jan C. Binnema]

Hi Henkjan,

On Saturday Jan 09 2016, Henkjan Gersen wrote:

> Thanks for the quick response Dirk, I must admit that I never noticed
> this GMail behavior before as I typically work with very large
> mailboxes. However I see that the same thing happens if I send an
> email using GMail's web-interface itself. So indeed expected
> behavior. :-(
>
> Two remaining questions if I'm allowed:
>
> 1) If I delete messages in GMails' sent folder through their
> web-interface they disappear immediately from the "all mail" folder in
> GMail as well. In mu4e I have to delete the messages twice; both in
> the sent folder as well as in the archive folder (the one
> corresponding to All Mail). Is there a workaround for this, or is that
> also behavior that is to be expected due to the difference between
> labels/folders?

So, from the IMAP perspective, I think GMail should indicate that some
message was deleted in each of the "folders" where it was present. If it
does that, I think your IMAP-sync tool should remove the message locally
as well. This is the theory at least :-), I suppose you have to check
with those tools to see how they do it.

> 2) When I delete messages in my inbox through mu4e I see them
> correctly disappear in GMail and the Bin folder on GMail is
> empty. However these messages remain present in the local trash folder
> in mu4e with the deleted status. Is there an option in mu4e that I'm
> forgetting to set to make sure that those messages also get removed
> locally as syncing the Bin folder with the local trash doesn't seem to
> do that job.

This ultimately seems to be about the same thing as above -- some
messages are deleted on the remote (GMail) side, and this should be
reflected locally as well. If it's not, that's something to check with
the synchronization tool and how GMail maps its folders to IMAP.

mu/mu4e happily deal with whatever is in your local maildirs, but
equally happy leaves the synchronization to other tools.

> Note that it might simply be that I misunderstand which folders I
> actually need to synchronize as I find the IMAP interface from Google
> rather confusing.

Yeah, it is indeed a bit confusing. GMail does not really map to IMAP
perfectly, so you'll get these "glitches in the matrix"
occasionaly. Still nice they provide an IMAP interface of course.

[Henkjan Gersen]

 

This ultimately seems to be about the same thing as above -- some
messages are deleted on the remote (GMail) side, and this should be
reflected locally as well. If it's not, that's something to check with
the synchronization tool and how GMail maps its folders to IMAP.

mu/mu4e happily deal with whatever is in your local maildirs, but
equally happy leaves the synchronization to other tools.

It just was the case of finding the right configuration option, which you have documented but it doesn't appear in blogs or the manual. I'm using mbsync as the synchronization tool, which doesn't like files moving without renaming. As a result the synchronization between local trash and Gmail/Bin failed with a pretty cryptic warning message, which is why the files didn't get removed in the local trash folder.

For those that find this thread: the combo mu4e, mbsync and I assume GMail needs adding (setq mu4e-change-filenames-when-moving t) in the config for deleting files to work as expected.

[Dirk-Jan C. Binnema]

Oh, even better!

Yes, some of the sync-tools have specific expectations. There is
actually a note about mbsync in the FAQ. Had forgotten all about that.

http://www.djcbsoftware.nl/code/mu/mu4e/General.html#General
(#14).

If you think that could be clearer, feel free to send a better
description.

[Kyle Marek-Spartz]

This is not something you are doing, but rather how Gmail works behind
the scenes.

If you want Gmail to work the way you describe, I would create an
Archive folder for what you are describing and not sync the All Mail
folder to your local computer.

--
Kyle Marek-Spartz