Trouble configuring multiple accounts with smtp for sending email with gmail

[Ilja]

Hi! This has been bugging me for a few weeks now, and I've spent a couple of hours in total trying to resolve the issue.

I recently added my personal gmail account to my mu4e setup, as well as my new work gmail account. Syncing using mbsync and this works like a charm, but sending emails still fails for me. Looking up general smtp tips and tricks is complicated by the mu4e multiple account config, so that's why I'm asking here.

I've set up debugging using the following:

(setq smtpmail-debug-info t
      auth-source-debug t
      smtpmail-debug-verb t)

I get the following error messages when I send an email from my gmail or work gmail account (

Sending...
Sending via mail...
auth-source-search: found 1 CACHED results matching (:max 1 :host "smtp.gmail.com" :port "587")
auth-source-search: found 1 CACHED results matching (:host "smtp.gmail.com" :port "587" :user "iko...@hawaii.edu" :max 1 :require nil :create nil)
555-5.5.2 Syntax error, goodbye. Please visit
555-5.5.2  https://support.google.com/a/answer/3221692 and review RFC 5321
555 5.5.2 specifications for more information. z15-20020a170906668f00b0099b76c3041csm865746ejo.7 - gsmtp
Process smtpmail not running: connection broken by remote peer

If I look at the *trace of SMTP session to smtp.gmail.com* I don't see anything that I do not already see in the above, but I'll post it at the end of the email just in case.

I've set up an app password and added that to my ~/.authinfo file:
machine imap.gmail.com login iko...@hawaii.edu password "<app password>" port 993
machine smtp.gmail.com login iko...@hawaii.edu password "<app password>" port 587

I've checked that the composed messages contain the correct way of formatting the To: and From address fields ("name with quotes" <email-in...@domain.com>) and don't find any useful info about this error message.


Any help would be appreciated!


Ilja


Here's the relevant section of my emacs config (full config available at https://github.com/japhir/ArchConfigs/blob/master/myinit.org#mu4e)

(use-package mu4e
  :ensure nil
  :load-path "/usr/share/emacs/site-lisp/mu4e/"
  :commands mu4e
  :bind (("C-c m" . mu4e) ("C-c C-o" . shr-browse-url))
  :hook ((mu4e-view-mode mu4e-compose-mode) . visual-line-mode)
  :config
  (require 'mu4e)  ;; somehow this is needed, because otherwise calling org-store-link doesn't work
  (require 'mu4e-org)
  ;; Use Ivy for mu4e completions (maildir folders, etc)
  (setq mu4e-completing-read-function #'ivy-completing-read)

  ;; send email using smtp
  ;; see https://www.djcbsoftware.nl/code/mu/mu4e/Gmail-configuration.html#Settings
  ;; see also the smtp heading
  (require 'smtpmail)
  (setq mail-user-agent 'mu4e-user-agent
        message-send-mail-function 'smtpmail-send-it
        send-mail-function 'smtpmail-send-it
        ;; this is the same for both mail clients
        smtpmail-default-smtp-server "smtp.gmail.com"
        smtpmail-smtp-service 587
        smtpmail-stream-type 'starttls
        starttls-use-gnutls t
        starttls-gnutls-program "gnutls-cli"
        starttls-extra-arguments nil
        smtpmail-auth-credentials (expand-file-name "~/.authinfo")
        smtpmail-debug-info t
        smtpmail-debug-verb t)

  (setq user-full-name "Ilja J. Kocken")

  (setq mu4e-contexts
        (list
          ;; University of Hawaii
          (make-mu4e-context
           :name "hawaii"
           :enter-func (lambda () (mu4e-message "Switch to Hawaiʻi context"))
           :match-func (lambda (msg)
                         (when msg
                           (string-prefix-p "/hawaii" (mu4e-message-field msg :maildir))))
           :vars '((user-mail-address . "iko...@hawaii.edu")
                   (message-user-organization . "University of Hawaiʻi at Mānoa")
                   ;; sending email
                   (smtpmail-smtp-user . "iko...@hawaii.edu") ; not sure if this is needed?
                   (smtpmail-smtp-server . "smtp.gmail.com")
                   (smtpmail-smtp-service . 587)
                   ;; this seems to be the old way to set it up
                   ;; (smtpmail-starttls-credentials . '(("smtp.gmail.com" 587 nil nil)))
                   ;; (smtpmail-auth-credentials . '(("smtp.gmail.com" 587 "iko...@hawaii.edu" nil)))
                   (mu4e-drafts-folder . "/hawaii/Drafts")
                   (mu4e-sent-folder . "/hawaii/Sent")
                   (mu4e-refile-folder . "/hawaii/Archive")
                   (mu4e-trash-folder . "/hawaii/Trash")
                   (mu4e-compose-signature .
                                           (concat
                                            "Dr. Ilja J. Kocken | Postdoc Researcher at SOEST |\n"
                                            "University of Hawaii at Mānoa | 1000 Pope Road |\n"
                                            "MSB 504 | Honolulu, HI 96822, USA\n"))
                   (mu4e-maildir-shortcuts .
                                           (("/hawaii/Inbox" . ?i)
                                            ("/hawaii/Trash" . ?t)
                                            ("/hawaii/Archive" . ?r)))
                   ;; delete gmail sent items because they're also stored in All Mail/Archive.
                   (mu4e-sent-messages-behavior . delete)))

          ;; old work
          (make-mu4e-context
    ; <config for an outlook smtp.office365.com account that still works for sending!>
    ; <omitted for brevity>
    )
          (make-mu4e-context
    ; <config for my personal gmail config, sending not working>
    ; omitted for brevity>
    )

  (setq mu4e-context-policy 'pick-first) ;; default to hawaii gmail

  (setq mu4e-change-filenames-when-moving t ; important for isync
        mu4e-headers-date-format "%Y-%m-%d %H:%M"
        mu4e-headers-fields
        '((:date          .  17)
          (:flags         .   5)
          (:from          .  22)
          (:subject       .  nil))
        mu4e-get-mail-command "env PATH=/usr/bin:/home/japhir/bin: mbsync -a"
        mu4e-headers-include-related t
        mu4e-compose-format-flowed t  ; plain-text nice to read on phone
        ;; use-hard-newlines t             ; above doesn't work for Outlooks, so disabled for colleagues
        mu4e-confirm-quit nil
        mu4e-view-show-images t)

   ;; if you press a V on an email you open it in the browser
  (add-to-list 'mu4e-view-actions '("View in browser" . mu4e-action-view-in-browser) t)
  (setq mu4e-org-link-query-in-headers-mode nil)

  (add-to-list 'mu4e-bookmarks
               '(:name "All Inboxes"
                 :query "maildir:/hawaii/Inbox OR maildir:/solismail/inbox OR maildir:/gmail/Inbox"
                       :key ?i))
  )


contents of *trace of SMTP session to smtp.gmail.com* buffer:

220 smtp.gmail.com ESMTP z15-20020a170906668f00b0099b76c3041csm865746ejo.7 - gsmtp
250-smtp.gmail.com at your service, [131.211.117.22]
250-SIZE 35882577
250-8BITMIME
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-CHUNKING
250 SMTPUTF8
220 2.0.0 Ready to start TLS
250-smtp.gmail.com at your service, [131.211.117.22]
250-SIZE 35882577
250-8BITMIME
250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-CHUNKING
250 SMTPUTF8
AUTH XOAUTH2 <string of letters, manually omitted>

555-5.5.2 Syntax error, goodbye. Please visit
555-5.5.2  https://support.google.com/a/answer/3221692 and review RFC 5321
555 5.5.2 specifications for more information. z15-20020a170906668f00b0099b76c3041csm865746ejo.7 - gsmtp

Process smtpmail connection broken by remote peer
AUTH PLAIN <omitted>
QUIT

[Dirk-Jan C. Binnema]

Hi,

On Tuesday Oct 03 2023, Ilja wrote:

> Hi! This has been bugging me for a few weeks now, and I've spent a couple
> of hours in total trying to resolve the issue.
>
> I recently added my personal gmail account to my mu4e setup, as well as my
> new work gmail account. Syncing using mbsync and this works like a charm,
> but sending emails still fails for me. Looking up general smtp tips and
> tricks is complicated by the mu4e multiple account config, so that's why
> I'm asking here.
>
> I've set up debugging using the following:
>
> (setq smtpmail-debug-info t
> auth-source-debug t
> smtpmail-debug-verb t)

Don't have a concrete suggestion, but something containing this works
for me:

--8<---------------cut here---------------start------------->8---
(smtpmail-smtp-user . "exa...@gmail.com")
(smtpmail-default-smtp-server . "smtp.gmail.com")
(smtpmail-smtp-server . "smtp.gmail.com")
(send-mail-function . smtpmail-send-it)
(message-send-mail-function . smtpmail-send-it)
(smtpmail-smtp-service . 465)
(smtpmail-stream-type . ssl)
(starttls-use-gnutls . t)
(starttls-gnutls-program . "gnutls-cli")
(starttls-extra-arguments . nil)
--8<---------------cut here---------------end--------------->8---

but the prob may be elsewhere; I'd try with a single setup (no contexts
etc.) first.

The 'syntax error' could be significant. Would useful to understand what
exactly it complains about.

Good luck!
Dirk.


--
Dirk-Jan C. Binnema Helsinki, Finland
e:dj...@djcbsoftware.nl w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C

[Brian Flaherty]

Ilja <iljak...@gmail.com> writes:

> Hi! This has been bugging me for a few weeks now, and I've spent a couple of hours in total trying to resolve the issue.
>
> I recently added my personal gmail account to my mu4e setup, as well as my new work gmail account. Syncing using mbsync and this works like a charm,
> but sending emails still fails for me. Looking up general smtp tips and tricks is complicated by the mu4e multiple account config, so that's why I'm asking
> here.

Hello,

I have several accounts I read with mu4e, but I leave the sending (smtp)
to msmtp. I am using Debian linux and there's a package for it. One
of my accounts is a gmail account.

In my .emacs-mu4e (mu4e specific emacs dotfile), I have:

(setq sendmail-program "/usr/bin/msmtp"
send-mail-function 'smtpmail-send-it
message-send-mail-function 'message-send-mail-with-sendmail)

I also have mu4e-contexts set up for all my different accounts.

msmtp seems smart enough to use the smtp server the corresponds to the
from/user address. I'm not sure which msmtp looks to if they differ.

Then in my .msmtprc file, I have:

defaults
auth on
tls on
### tls_trust_file /usr/local/share/certs/ca-root-nss.crt
logfile /home/user/.msmtp.log

## main
account first
host mail.host.com
user us...@host.com
from us...@host.com
password abc123
port 587

## Gmail
account gmail
host smtp.gmail.com
user us...@gmail.com
from us...@gmail.com
password "abc 123"
port 587

Password for gmail is my app password and enclosed in double quotes.

This set up has worked very well for many months, maybe a couple years?

Hope this helps.

Brian

[Christopher Gray]

Brian Flaherty <gmb...@gmail.com> writes:

> Ilja <iljak...@gmail.com> writes:
>
>> but sending emails still fails for me. Looking up general smtp tips
>> and tricks is complicated by the mu4e multiple account config, so
>> that's why I'm asking here.

> I have several accounts I read with mu4e, but I leave the sending
> (smtp)
> to msmtp. I am using Debian linux and there's a package for it. One
> of my accounts is a gmail account.

> Then in my .msmtprc file, I have:

> password "abc 123"

>
> Password for gmail is my app password and enclosed in double quotes.
>

I’m also using msmtp, and my configuration looks very similar to
Brian’s, except it is in ~/.config/msmtp/config but omits the passwords
from that file because msmtp has native support for gnome-keyring.

--
Christopher Gray
christoph...@gmail.com
gpg: 33B3 BC89 9D14 8852 E874 033E 3217 429B 82FF 02DA

[Adam Hammer]

On Tue, Oct 03 2023, Ilja wrote...

> I recently added my personal gmail account to my mu4e setup, as well
> as my new work gmail account. Syncing using mbsync and this works like
> a charm, but sending emails still fails for me.

Sorry if this older message has already been solved.

Is it possible that there's a mismatch between authorization systems?
You say:

> I've set up an app password and added that to my ~/.authinfo file:

and then the SMTP trace says:

> AUTH XOAUTH2 <string of letters, manually omitted>

I don't think that app passwords and OAUTH2 are the same thing. I
believe the app password should be sent just as if it was a regular
password, not encoded and whatever else the OAUTH2 protocol prescribes.

Do you have any settings that are forcing OAUTH2 to be used?

Adam