I need help with sendmail

[root]

Charter/Spectrum has been my ISP since I switched from dialup
a few decades ago. I remember getting sendmail to run was
a nightmare to me then. I seem to be faced with that nightmare
again.

On Tuesday I am scheduled to get gigabit fiber from AT&T
so I have to switch my smtp server from smtp.charter.net
to smtp.gmail.com. If only it were that simple.

I use /usr/bin/nail to send mail, and getmail to (pop) fetch
my mail from my account at gmail.com. Neither sending
or fetching works.

When I try to send mail I get:
smtp-server: 530 5.7.0 Must issue a STARTTLS command first. v7sm14389077pgv.81 - gsmtp

I have spent several hours today trying to fix this problem. I
started with building what I thought were appropriate certificates
and putting them in /etc/sendmail/certs.

Then I found, and modified sendmail.mc to point to the outgoing
certificate, but the Makefile doesn't work in /etc/sendmail.

I then worked on using getmail to fetch mail from gmail.

I modified my getmailrc to change the server to pop.gmail.com,
leaving type = SimplePOP3Retriever, and adding:
port = 995

but getmail just times out with nothing, not even an error message.

I suspect gmail doesn't support POP3 and that getmail is out-of-date.

I am desperate for any help you can offer.

Thanks.

[Chris Vine]

I am not sure why you are using sendmail if you are also using
mailx/nail. My advice is to ditch sendmail and let mailx deliver the
mail itself, which it is well equipped to do. You include no
information on how you have set up sendmail anyway so that part is
unanswerable.

For sending, TLS is supported by mailx and indeed is required by gmail.
But mailx isn't psychic (neither is sendmail) and cannot intuit that:
you need to set mailx's smtp-use-starttls variable so that it knows to
initiate TLS, say by including 'set smtp-use-starttls' in your mailx
configuration file. If using v14 syntax, presumably you will already
have set the smtp-auth-user and smtp-auth-password variables, so do
what you did with those. I recommend reading the man pages.

gmail also supports the POP3 protocal for receiving. I don't use
getmail, but I would be very surprised if it didn't work OK with that
program provided you tell it to use TLS. For fetchmail, which I do use,
including 'ssl sslproto tls1 sslcertck' in your fetchmail options will
do the job, using port 995.

[Chris Vine]

On re-reading your post I may not have made it clear enough that when I
say "ditch sendmail", I also meant that you should use smtp.gmail.com,
port 587, as your mail relay. Trying to send emails directly using
sendmail without an intermediate relay rarely works these days, as many
recipient ISPs won't accept it. To do use gmail as your relay, as
mentioned above set the smtp-use-starttls, smtp-auth-user and
smtp-auth-password variables, together with the from, smtp and
smtp-auth variables.

[root]

>> gmail also supports the POP3 protocal for receiving. I don't use
>> getmail, but I would be very surprised if it didn't work OK with that
>> program provided you tell it to use TLS. For fetchmail, which I do use,
>> including 'ssl sslproto tls1 sslcertck' in your fetchmail options will
>> do the job, using port 995.
>
> On re-reading your post I may not have made it clear enough that when I
> say "ditch sendmail", I also meant that you should use smtp.gmail.com,
> port 587, as your mail relay. Trying to send emails directly using
> sendmail without an intermediate relay rarely works these days, as many
> recipient ISPs won't accept it. To do use gmail as your relay, as
> mentioned above set the smtp-use-starttls, smtp-auth-user and
> smtp-auth-password variables, together with the from, smtp and
> smtp-auth variables.

Thanks very much for responding. This is my third try
to respond to your two messages. I think I had included
some lines that slrn choked on.

I got outgoing email to work when I send to another account
at gmail. To handle the port 587 would I just use:
set smtp=smtp.gmail.com:587

I will try that and see.

I tried fetchmail, with this .fetchmailrc:

set bouncemail
poll pop.gmail.com protocol POP3 user "MYGMAILNAME" there with password "MYGMAILPASSWORD"

and fetchmail just hangs.

Can you give me an abstracted version of your .fetchmailrc?

Thanks.

[Chris Vine]

My mailx configuration file is:

set smtp=smtp.gmail.com:587
set smtp-auth-user=xx...@gmail.com
set smtp-auth-password=xxxxxxxx
set from=xx...@gmail.com
set smtp-auth=login
set smtp-use-starttls

My fetchmail configuration stanza is:

poll pop.gmail.com with proto pop3 port 995
auth password
user 'xx...@gmail.com' with password 'xxxxxxxx' mda "/usr/bin/procmail" ssl sslproto tls1 sslcertck

The fetchmail configuration only works if you have disabled gmail's
additional security features. If you have enabled those, you can only
get your mail via the gmail apps or web interface. If you don't use
procmail then omit the mda specification.

[Rinaldi]

This has worked for some years now:

poll pop.gmail.com with proto POP3 service 995
user 'username' there with password 'pasword' is 'username' here ssl

[Grant Taylor]

On 7/11/21 12:40 PM, Chris Vine wrote:
> Trying to send emails directly using sendmail without an intermediate
> relay rarely works these days, as many recipient ISPs won't accept it.

You can easily configure Sendmail to use an upstream email server as --
what is called -- a "Smart Host".

Either via the "DS" macro in sendmail.cf or "define(`SMART_HOST',
`...')dnl" in sendmail.mc.

Sendmail methodology strongly recommends using the macro configuration
(mc) file method.

It is possible to configure Sendmail to use Gmail as a Smart Host.

Sendmail /should/ default to using STARTTLS as a client without any
special configuration. If for some reason it's not, let me know.

Let me know if you want more specifics.



--
Grant. . . .
unix || die

[Chris Vine]

On Sun, 11 Jul 2021 17:38:24 -0600
Grant iTaylor <gta...@tnetconsulting.net> wrote:
> On 7/11/21 12:40 PM, Chris Vine wrote:
> > Trying to send emails directly using sendmail without an intermediate
> > relay rarely works these days, as many recipient ISPs won't accept it.
>
> You can easily configure Sendmail

In my experience, which I accept may be defective, nothing in sendmail
is easy to configure. It is also pointless when you use mailx to send
via the gmail smtp relay, which was the case in hand. Mailx can interact
with that relay adequately by itself.

Where you want to use a mta, postfix is in my opinion by far the better
solution, and the one that slackware-current now uses by default
(sendmail now only appears in extra). It is what I use on my systems
and forwarding to the gmail relay is trivial to set up for it (with a
human readable syntax), where you do need your own local mta. sendmail
with its undebuggable m4 macro usage is very much a minority taste
nowadays.

[root]

Thanks for the offer, it seems to work now without sendmail.

[Grant Taylor]

On 7/11/21 7:46 PM, root wrote:
> Thanks for the offer, it seems to work now without sendmail.

You're welcome. I'm glad that you got something working satisfactorily.

[Grant Taylor]

On 7/11/21 6:34 PM, Chris Vine wrote:
> In my experience, ...

In /your/ experience. /My/ experience is obviously different.

> It is also pointless when you use mailx to send via the gmail smtp
> relay, which was the case in hand. Mailx can interact with that relay
> adequately by itself.

In my experience, there is almost always something else on the system
that wants to send email through a local MTA. A service which -- as I
understand it -- mailx doesn't provide. I have found it's always been
better to take the time to configure a proper MTA, even if it only
listens on loopback and accepts / relays email from local processes.

> ... in my opinion ...

...

> sendmail with its undebuggable m4 macro usage is very much a minority
> taste nowadays.

I've never had any problems debugging Sendmail nor m4. I find m4 to be
quite useful and that I am able to make it do lots of different things
with re-text formatting that others say can't be done.

[root]

Rinaldi <r...@nunya.inv> wrote:
>
> poll pop.gmail.com with proto POP3 service 995
> user 'username' there with password 'pasword' is 'username' here ssl
>

Thanks for responding. That's pretty much what we had
worked out, but the service 995 is new. It seems
to work without that.

[root]

I was mired the sendmail stuff before Chris came through. I found
that /etc/mail doesn't even have sendmail.mc and when you find
it, add changes, then the Makefile doesn't work. Patrick must
not like sendmail either.

BTW, I am running 14.2
>

[Chris Vine]

On Sun, 11 Jul 2021 21:13:49 -0600
Grant Taylor <gta...@tnetconsulting.net> wrote:
> In my experience, there is almost always something else on the system
> that wants to send email through a local MTA. A service which -- as I
> understand it -- mailx doesn't provide. I have found it's always been
> better to take the time to configure a proper MTA, even if it only
> listens on loopback and accepts / relays email from local processes.

That can indeed be useful where you have, say, a laptop which relies on
a wireless connection for internet access which may not always be
present. Running your own MTA daemon on port 25 on the laptop enables
you to use the daemon to queue emails from user programs and then
automatically send them when an internet connection is established.
This saves the user having to queue messages in the email client and
having to send them herself from the email client explicitly when a
wireless connection becomes available.

Unfortunately sendmail does not handle this situation: it will not send
email via an IP interface (say, wlan0) which isn't up when the sendmail
daemon is started. Or at least that was the case when I stopped using
sendmail, partly for that reason (its lack of intelligible
documentation and bizarre syntax was another reason). Postfix handles
this situation fine and is also very well documented.

I run postfix on my laptops for that explicit purpose, and I set it up
to forward to smtp.gmail.com when a wireless connection is up. On my
desktop, which always has an interface which is up, I have some
monitoring processes which from time to time send out emails (usually to
my mobile telephone) and those processes forward to smtp.gmail.com
directly using mailx without an intermediate MTA. Each to their own.

[Bit Twister]

On Sun, 11 Jul 2021 17:48:37 -0000 (UTC), root wrote:
> Charter/Spectrum has been my ISP since I switched from dialup
> a few decades ago. I remember getting sendmail to run was
> a nightmare to me then. I seem to be faced with that nightmare
> again.

Yep, I switch ISP anytime I can save money on the monthly bill and
have configured my MTA (postfix) to not care.

> On Tuesday I am scheduled to get gigabit fiber from AT&T
> so I have to switch my smtp server from smtp.charter.net
> to smtp.gmail.com. If only it were that simple.
>
> I use /usr/bin/nail to send mail, and getmail to (pop) fetch
> my mail from my account at gmail.com. Neither sending
> or fetching works.

No experience with nail/sendmail/getmail.

I run Mageia Linux.

Configured my MTA to forward all outbound mail through my
gmail account.

I have separate Linux accounts to pull mail from 7 email accounts.

Each user has a houly cron job running fetchmail to pull down
any email. ~/.fetchmailrc example: cat /accounts/boris/.fetchmailrc
#******************************
# /accounts/boris
#******************************

poll "imap.gmail.com" with proto IMAP
user "sec...@gmail.com" there with password "squirrel"
is boris here
options
ssl # download "seen" and "unseen" messages
fetchall # retrieve old and new messages
stripcr # Strip carriage returns from ends of lines
nokeep # delete new messages after retrieval

# end /accounts/boris/.fetchmailrc

A root hourly cron job uses xmessage to popup a message to
run a script to automagically log into the account that has
mail and run the user's mail client.

[Grant Taylor]

On 7/12/21 3:20 AM, Chris Vine wrote:
> That can indeed be useful where you have, say, a laptop which relies
> on a wireless connection for internet access which may not always be
> present. Running your own MTA daemon on port 25 on the laptop enables
> you to use the daemon to queue emails from user programs and then
> automatically send them when an internet connection is established.
> This saves the user having to queue messages in the email client and
> having to send them herself from the email client explicitly when a
> wireless connection becomes available.

This concept; connection / lack of connection, comes into play in many
different ways. Wireless, or the lack there of, is just a contemporary
example of a perpetual problem with networking.

> Unfortunately sendmail does not handle this situation: it will not
> send email via an IP interface (say, wlan0) which isn't up when
> the sendmail daemon is started.

I've not tested this particular scenario so I don't have first hand
experience. But, that being said, I believe this to be a
(mis)configuration problem.

The first thing that comes to mind is what interface(s) the daemon is
configured to listen on. You can't get a daemon to listen on an
interface / socket (IP & port pair) that is not available when the
daemon starts. However you can tell most daemons to listen on /all/
interfaces / sockets (IP & port pairs) on the system. The subtle nuance
is that the second option; all, will usually allow for IP addresses to
be added and / or removed from the system while the daemon is running.
At least that's been my experience.

The second thing that comes to mind is that you have to specifically
tell Sendmail to use the same IP to send email out as the email came in
on /if/ that's what you want. Which means that Sendmail will receive
and send email on different interfaces by default. As such, Sendmail
should be quite capable of receiving email on loopback and sending from
eth0 / wlan0 / ppp0 without reconfiguring things.

The third thing that comes to mind is that Sendmail doesn't care about
how the kernel does the IP routing, nor does it have any effective way
to influence it. -- Yes, you can get into some really weird minutia /
gyrations to effect some influence, but they are so atypical that they
are effectively a non-issue.

Sendmail, or arguably /any/ daemon, /really/ *SHOULD* be able to run in
the scenario you describe above.

> Or at least that was the case when I stopped using sendmail, partly
> for that reason (its lack of intelligible documentation and bizarre
> syntax was another reason).

I agree that Sendmail's documentation, history, and lore are ... less
than favorable. But that doesn't make it any less capable, just more
difficult to use. ;-)

> Postfix handles this situation fine and is also very well documented.

I question if Postfix and Sendmail had the same type of configuration,
as in what IPs were they listening on. Or if their configurations were
subtly different; Sendmail configured for only specific interfaces while
Postfix was listening to all interfaces.

> I run postfix on my laptops for that explicit purpose, and I set it
> up to forward to smtp.gmail.com when a wireless connection is up.

Do you alter Postfix's state or let it continue to try and fail while
the connection is down then succeed when the connection is up?

> On my desktop, which always has an interface which is up, I have some
> monitoring processes which from time to time send out emails (usually
> to my mobile telephone) and those processes forward to smtp.gmail.com
> directly using mailx without an intermediate MTA. Each to their own.

It has to do with the scope of the solution. A local MTA is effectively
system wide solution. Conversely mailx is only a solution for things
that use mailx. I tend to favor solutions that apply to more things or
at the very least cover all of the things that I need / want to be covered.

[Chris Vine]

On Mon, 12 Jul 2021 10:31:39 -0600
Grant Taylor <gta...@tnetconsulting.net> wrote:
> On 7/12/21 3:20 AM, Chris Vine wrote:
> > Unfortunately sendmail does not handle this situation: it will not
> > send email via an IP interface (say, wlan0) which isn't up when
> > the sendmail daemon is started.
>
> I've not tested this particular scenario so I don't have first hand
> experience. But, that being said, I believe this to be a
> (mis)configuration problem.

[snip]

> Sendmail, or arguably /any/ daemon, /really/ *SHOULD* be able to run in
> the scenario you describe above.

OK, do it - show your sendmail.mc file and I will reinstall sendmail
and test it at my end. When I stopped using sendmail it was a
recognised 'problem'. ('Problem' in quotes because it seemed to be
regarded as a feature, not a bug.)

Note: I am not saying it cannot now be done, possibly it can. It is
easy to test for: bring your interfaces down, start the sendmail
daemon, and bring the interfaces back up again.

[Chris Vine]

For what it is worth, someone else reported the same problem on the
linuxquestions slackware forum a few years ago:
https://www.linuxquestions.org/questions/slackware-14/sendmail-problem-4175617511/