Mailutils+nullmailer: sender full name

[Gertjan Klein]

I am configuring a new VPS, and decided to try nullmailer to send mail.
I don't want to receive mail on the VPS, I just want the mail the system
generates to end up in my mailbox elsewhere. This works, but the mail
from address looks like this: "gklein <gkl...@parvos.nl>". This irks me.
My account has my full name configured; why isn't it used? I am testing
this as follows:

$ echo "this is a test" | mail -s Subject gkl...@parvos.nl

I am unclear about which of GNU mailutils or nullmailer is responsible
for this header. I've tried various things the internet suggested; none
made any difference at all.

While this is more an annoyance than a showstopper for me, I would like
the From address to look like "My name <my@address>". Does anyone here
know which program to persuade, and how to persuade it?

Kind regards,
Gertjan.

[Geert Stappers]

On Tue, Oct 17, 2023 at 05:43:55PM +0200, Gertjan Klein wrote:
> I am configuring a new VPS, and decided to try nullmailer to send mail.
> I don't want to receive mail on the VPS, I just want the mail the system
> generates to end up in my mailbox elsewhere. This works, but the mail
> from address looks like this: "gklein <gkl...@parvos.nl>". This irks me.
> My account has my full name configured;

Where exactly is the full name configured?

> why isn't it used?

That is reason for the above question.

> I am testing this as follows:
>
> $ echo "this is a test" | mail -s Subject gkl...@parvos.nl

Works for me ...

> I am unclear about which of GNU mailutils or nullmailer is responsible
> for this header. I've tried various things the internet suggested; none
> made any difference at all.
>
> While this is more an annoyance than a showstopper for me, I would like
> the From address to look like "My name <my@address>". Does anyone here
> know which program to persuade, and how to persuade it?

|$ command -v mail
|/usr/bin/mail
|$ dpkg -S /usr/bin/mail
|dpkg-query: no path found matching pattern /usr/bin/mail
|$ ls -l /usr/bin/mail
|lrwxrwxrwx 1 root root 22 Oct 12 2012 /usr/bin/mail -> /etc/alternatives/mail
|$ ls -l /etc/alternatives/mail
|lrwxrwxrwx 1 root root 18 Oct 12 2012 /etc/alternatives/mail -> /usr/bin/bsd-mailx
|$ ls -l /usr/bin/bsd-mailx
|-rwxr-xr-x 1 root root 112968 Apr 14 2022 /usr/bin/bsd-mailx
|$ dpkg -S /usr/bin/bsd-mailx
|bsd-mailx: /usr/bin/bsd-mailx
|$

Did it help?


> Kind regards,
> Gertjan.


Groeten
Geert Stappers
--
Silence is hard to parse

[Gertjan Klein]

Op 17-10-2023 om 19:10 schreef Geert Stappers:

> On Tue, Oct 17, 2023 at 05:43:55PM +0200, Gertjan Klein wrote:
>> I am configuring a new VPS, and decided to try nullmailer to send mail.
>> I don't want to receive mail on the VPS, I just want the mail the system
>> generates to end up in my mailbox elsewhere. This works, but the mail
>> from address looks like this: "gklein <gkl...@parvos.nl>". This irks me.
>> My account has my full name configured;
>
> Where exactly is the full name configured?

I meant the user account on that machine:

gklein@parvos:~$ cat /etc/passwd | grep gklein
gklein:x:1000:1000:Gertjan Klein,,,:/home/gklein:/bin/bash

> |$ command -v mail
> |/usr/bin/mail
> |$ dpkg -S /usr/bin/mail
> |dpkg-query: no path found matching pattern /usr/bin/mail
> |$ ls -l /usr/bin/mail
> |lrwxrwxrwx 1 root root 22 Oct 12 2012 /usr/bin/mail -> /etc/alternatives/mail
> |$ ls -l /etc/alternatives/mail
> |lrwxrwxrwx 1 root root 18 Oct 12 2012 /etc/alternatives/mail -> /usr/bin/bsd-mailx
> |$ ls -l /usr/bin/bsd-mailx
> |-rwxr-xr-x 1 root root 112968 Apr 14 2022 /usr/bin/bsd-mailx
> |$ dpkg -S /usr/bin/bsd-mailx
> |bsd-mailx: /usr/bin/bsd-mailx
> |$
>
> Did it help?

That shows that your mail executable is actually bsd-mailx. If you also
use nullmailer, that means mailutils is the culprit here:

gklein@parvos:~$ command -v mail
/usr/bin/mail
gklein@parvos:~$ ls -l /usr/bin/mail
lrwxrwxrwx 1 root root 22 Mar 3 2023 /usr/bin/mail ->
/etc/alternatives/mail
gklein@parvos:~$ ls -l /etc/alternatives/mail
lrwxrwxrwx 1 root root 23 Mar 3 2023 /etc/alternatives/mail ->
/usr/bin/mail.mailutils

My mail is from package mailutils, as described, and it doesn't (by
default) appear to use the full name from /etc/passwd. I had expected
this to 'just work', seeing as "Mailutils is a swiss army knife of
electronic mail handling". I can, of course, switch to a different MTA.
But it's difficult for me to believe this can't be done with this 'Swiss
army knife'.

Kind regards,
Gertjan.

[Greg Wooledge]

On Tue, Oct 17, 2023 at 05:43:55PM +0200, Gertjan Klein wrote:

> This works, but the mail
> from address looks like this: "gklein <gkl...@parvos.nl>". This irks me.
> My account has my full name configured; why isn't it used? I am testing
> this as follows:
>
> $ echo "this is a test" | mail -s Subject gkl...@parvos.nl
>
> I am unclear about which of GNU mailutils or nullmailer is responsible
> for this header. I've tried various things the internet suggested; none
> made any difference at all.
>
> While this is more an annoyance than a showstopper for me, I would like
> the From address to look like "My name <my@address>". Does anyone here
> know which program to persuade, and how to persuade it?

You mean the "From:" header. This is provided by the MUA (Mail User
Agent), which in your case is whatever program "mail" links to.

On my system, mailx points to bsd-mailx:

unicorn:~$ ls -l /etc/alternatives/mailx
lrwxrwxrwx 1 root root 18 Jan 12 2018 /etc/alternatives/mailx -> /usr/bin/bsd-mailx*

The bsd-mailx(1) documentation is EXTREMELY confusing, and it took
me 8 tries to get this, so I hope you appreciate it:

unicorn:~$ cat .mailrc
set from="Someone <m...@wooledge.org>"
unicorn:~$ echo test8 | mailx -s test8 gr...@wooledge.org

This generates:

Date: 17 Oct 2023 18:30:46 -0000
From: Someone <m...@wooledge.org>
To: gr...@wooledge.org
Subject: test8

If you aren't using bsd-mailx, then you will have to figure out how to
do this using whatever implementation you're using.

[Gertjan Klein]

Op 17-10-2023 om 20:40 schreef Greg Wooledge:

> On Tue, Oct 17, 2023 at 05:43:55PM +0200, Gertjan Klein wrote:
>> While this is more an annoyance than a showstopper for me, I would like
>> the From address to look like "My name <my@address>". Does anyone here
>> know which program to persuade, and how to persuade it?
>
> You mean the "From:" header. This is provided by the MUA (Mail User
> Agent), which in your case is whatever program "mail" links to.

Yes, I've drawn this conclusion from Geert's reply as well.

> On my system, mailx points to bsd-mailx:

On Geert's as well. I wonder why. I installed nullmailer, and it
automatically pulled in mailutils. I stayed with it because I don't know
any better. You and Geert apparently decided on something different. May
I ask why?

> The bsd-mailx(1) documentation is EXTREMELY confusing, and it took
> me 8 tries to get this, so I hope you appreciate it:

I do appreciate it. If I switch to bsd-mailx I have something that
should work. Although I'm concerned by the statement (on the Debian
package page) that it doesn't speak SMTP. This is how I send the mail on
to my internet provider with mailutils. I wouldn't know how else to do it.

I've found the mailutils documentation impenetrable as well. More
searching and trying ahead, I guess.

Kind regards,
Gertjan.

[Greg Wooledge]

On Tue, Oct 17, 2023 at 09:50:13PM +0200, Gertjan Klein wrote:
> Op 17-10-2023 om 20:40 schreef Greg Wooledge:

> > On my system, mailx points to bsd-mailx:
>
> On Geert's as well. I wonder why. I installed nullmailer, and it
> automatically pulled in mailutils. I stayed with it because I don't know any
> better. You and Geert apparently decided on something different. May I ask
> why?

Hard to say. This system has been upgraded many times, and the package
names involved have perhaps changed over time. There's also this:

unicorn:~$ aptitude why bsd-mailx
i abcde Recommends bsd-mailx

There are *lots* of packages like that, including exim4-base. Chances
are pretty good that something will bring in bsd-mailx without a user
explicitly choosing it.

I can't say, at this point, whether I explicitly installed bsd-mailx or
one of its predecessor packages, or simply got it as a recommendation.

> I do appreciate it. If I switch to bsd-mailx I have something that should
> work. Although I'm concerned by the statement (on the Debian package page)
> that it doesn't speak SMTP. This is how I send the mail on to my internet
> provider with mailutils. I wouldn't know how else to do it.

The way Unix mail works is by dividing the workload between an MUA and
an MTA.

The MUA (Mail User Agent) provides the user interface. It's the program
you use to send and/or read mail. Examples include elm, pine, mutt, mailx,
Thunderbird, and so on.

The MTA (Mail Transport Agent) (or Transfer) is the back end. It's the
part that does the actual deliveries, including sending and receiving
messages over the Internet.

A command-line MUA like mailx sends mail to the MTA by invoking the
/usr/sbin/sendmail (traditionally /usr/lib/sendmail) program, which is
a hook provided by the MTA. The MUA composes the message, which is a
stream of lines of text, and sends it to /usr/sbin/sendmail on standard
input.

So, it's not *expected* that your MUA will speak SMTP directly. That's
not its job.

In your setup, nullmailer is your MTA. Here's its package description:

Nullmailer is a replacement MTA for hosts, which relay to a fixed set of
smart relays. It is designed to be simple to configure and especially
useful on slave machines and in chroots.

There are some key words in there that you might not understand at first
glance. "Relay" is the term used when an MTA sends a message along to
another MTA on another computer, usually using SMTP. This is different
from when an MTA accepts a message directly from a MUA. That's usually
called a "submission", and can use either SMTP or /usr/sbin/sendmail.

"Smart relay" or "smart host" is a term used to describe a machine
that acts as an SMTP relay for a group of computers, typically on a
local area network, but nowadays that's not as common as it used to be.
The implication is that the group of computers only know how to send
mail to the "smart relay" and nothing else. The smart relay handles
the rest of the job, doing DNS lookups, retrying upon failures, and so on.

So, when you send a message using "mail" or "mailx",

1) Mailx (MUA) composes the message for you (including the From: header).
2) Mailx sends the message to nullmailer (MTA).
3) Nullmailer relays it to its "smart host" (some other MTA).
4) The smart host sends it along, either to the final destination MTA
or to the next hop MTA.
5) Eventually, the final destination MTA receives the message and
delivers it to a local inbox.
6) The recipient reads the inbox using their MUA.

> I've found the mailutils documentation impenetrable as well. More searching
> and trying ahead, I guess.

I skimmed parts of it. It appears to read ~/.mailrc if it exists.
You could try putting commands in there and seeing what happens. Start
with something like

set from="My Name <m...@address.net>"

and test with deliveries that don't go outside of your controlled email
environment, if possible.

[David Wright]

On Tue 17 Oct 2023 at 19:41:43 (+0200), Gertjan Klein wrote:
> Op 17-10-2023 om 19:10 schreef Geert Stappers:
> > On Tue, Oct 17, 2023 at 05:43:55PM +0200, Gertjan Klein wrote:
> > > I am configuring a new VPS, and decided to try nullmailer to send mail.
> > > I don't want to receive mail on the VPS, I just want the mail the system
> > > generates to end up in my mailbox elsewhere. This works, but the mail
> > > from address looks like this: "gklein <gkl...@parvos.nl>". This irks me.
> > > My account has my full name configured;
> >
> > Where exactly is the full name configured?
>
> I meant the user account on that machine:
>
> gklein@parvos:~$ cat /etc/passwd | grep gklein
> gklein:x:1000:1000:Gertjan Klein,,,:/home/gklein:/bin/bash

On my (bullseye) system, that field is what is used to get my full
name. I tested it by inserting my middle initial in place of
the space, and then sending an email with:

$ mail -s Test3 myuse...@thishost.corp
Cc:
[^D typed here as I had nothing to say]

Null message body; hope that's ok
$

I did try to strace mail, but it would produce a return code of 1 and
not send the email. However, this did show that as well as opening
/etc/passwd (several times, at different stages), it looked for the
following files in different ways:

access("/etc/mailutils.rc", F_OK) = -1 ENOENT (No such file or directory)
stat("/etc/mailutils.conf", 0x7fffdbdbf9d0) = -1 ENOENT (No such file or directory)
stat("/home/myusername/.mail", 0x7fffdbdbf9d0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/mail.rc", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/myusername/.mailrc", O_RDONLY) = -1 ENOENT (No such file or directory)

(Note that I don't personally send mail this way—I use mutt, of course.)

> > |$ ls -l /usr/bin/bsd-mailx
> > |-rwxr-xr-x 1 root root 112968 Apr 14 2022 /usr/bin/bsd-mailx

> > [ … ]

>
> That shows that your mail executable is actually bsd-mailx. If you
> also use nullmailer, that means mailutils is the culprit here:

No, my system uses mailutils:

$ aptitude why mailutils
i emacs-gtk Depends emacs-bin-common (= 1:27.1+1-3.1+deb11u2)
i A emacs-bin-common Recommends mailutils
$

but emacs wasn't responsible for installing it. That was exim4, which
was installed about four minutes earlier. But exim4 Recommends mailx,
and bsd-mailx also Provides mailx, so I don't know why mailutils was
favoured over bsd-utils. Ironically, bsd-mailx is in the netinst
installer's pool, whereas mailutils isn't. So it may just depend on
chance—the order in which you happen to install other packages.
(Aside: does anyone know what "heirloom-mailx" is?)

> gklein@parvos:~$ command -v mail
> /usr/bin/mail
> gklein@parvos:~$ ls -l /usr/bin/mail
> lrwxrwxrwx 1 root root 22 Mar 3 2023 /usr/bin/mail ->
> /etc/alternatives/mail
> gklein@parvos:~$ ls -l /etc/alternatives/mail
> lrwxrwxrwx 1 root root 23 Mar 3 2023 /etc/alternatives/mail ->
> /usr/bin/mail.mailutils
>
> My mail is from package mailutils, as described, and it doesn't (by
> default) appear to use the full name from /etc/passwd. I had expected
> this to 'just work', seeing as "Mailutils is a swiss army knife of
> electronic mail handling". I can, of course, switch to a different
> MTA. But it's difficult for me to believe this can't be done with this
> 'Swiss army knife'.

I don't know much/anything about nullmailer, or why you chose it.
(But then, the only reason I use exim4 is because I know how to
# dpkg-reconfigure exim4-config
it.) However, I do see the following in nullmailer's man pages:

"SYNOPSIS
sendmail [ flags ] [ recipients ] < message

"DESCRIPTION
This program is a front end program for nullmailer-inject and null‐
mailer-smtpd. It is used by programs that expect a sendmail interface
for sending email. After parsing the command-line arguments, this pro‐
gram executes either nullmailer-inject or nullmailer-smtpd depending on
the presence of the -bs option on the command-line. See the documenta‐
tion for nullmailer-inject for details on how messages are reformatted
and queued."

and:

"SYNOPSIS
nullmailer-inject [-a] [-b] [-e] [-f sender] [-h] [recipient [recipient
...]]

"DESCRIPTION
This program reads a email message from standard input, reformats its
header to comply with RFC822, and sends the resulting message to the
queue.

"HEADER FIELDS
The following lines are parsed for recipient addresses: To, Cc, Bcc,
Apparently-To, Resent-To, Resent-Cc, and Resent-Bcc.

The following sender address lines are parsed and rewritten: Sender,
From, [ … ]"

and:

"This file lists all the major user-visible changes to nullmailer.
-------------------------------------------------------------------------------
Changes in version 2.2

[ … ]

"- nullmailer-inject now sets the full name of the sender to the user
name as a fallback. This helps distinguish system sent messages when
the MTA rewrites the address (as does GMail, for example)."

I can't judge the significance of any of this for you—just observations.

Cheers,
David.

[Tixy]

On Wed, 2023-10-18 at 11:28 -0500, David Wright wrote:
> (Aside: does anyone know what "heirloom-mailx" is?)

I remember it being the mailx compatible program that got installed by
default many releases ago, and it was replaced by s-nail in
2016. (Possibly as a fork of that orphaned project?)

I still use s-nail on all my machines.

--
Tixy

[Greg Wooledge]

Before I go any further, please note that "mailx" is the POSIX approved
program name for sending scripted emails. "mail" is a legacy synonym,
not standardized, because way too many systems have differing programs
named "mail" with differing behaviors. There's also capital-M "Mail"
which thankfully has fallen out of favor.

Now, <https://wiki.debian.org/NewInStretch> says:

Users of the heirloom-mailx package should beware: in stretch,
heirloom-mailx has become a transitional package bringing in
s-nail. However, the s-nail package does not provide mailx. If you
have scripts which call mailx -a attachfile ... you will need to
rewrite those scripts, or manually change the /etc/alternatives/mailx
symlink. (update-alternatives will not work.) See also bugs 858080
and 846062.

* In addition, s-nail does not provide a /usr/bin/mail program. If
you run mail in a script, be sure to install either bsd-mailx
or mailutils. Note: bsd-mailx and heirloom-mailx (s-nail) do not
conflict with each other any more.

What a mess, eh? (That addendum about /usr/bin/mail wasn't mine, I don't
think. Maybe? Dunno, it's been many years.)

Of particular note is the "-a" option. bsd-mailx's -a adds additional
headers. heirloom-mailx's -a attaches files. Any system that relied
on mailx -a somefile ... for sending email with attachments would have
broken after a stretch upgrade, unless explicit steps were taken to fix
the situation.

Before stretch, if you had heirloom-mailx installed, both "mail -a file"
and "mailx -a file" worked. After upgrading to stretch, you'd have an
s-nail program, but *not* a mail or mailx program, so all your scripts
would break. And if you did the obvious thing and installed bsd-mailx
as a replacement, "mailx -a file" would try to add a header named "file"
instead of attaching a file named "file".

So, after the stretch upgrade, you would either have to change all of
your scripts to use s-nail instead of mailx, or you'd have to change the
/etc/alternatives/mailx symlink manually, pointing it to s-nail
despite the package manager's refusal to consider s-nail a valid
alternative.

I don't know whether mailutils works better in this respect. I've never
used it.

Since I've got bsd-mailx installed on my home system, and
/etc/alternatives/mailx pointing to it, I can conclude that I didn't
suffer from this particular problem at home. Which makes sense -- I
don't send any automated emails from this computer.

Servers might be a dramatically different story.

[Tixy]

On Wed, 2023-10-18 at 13:56 -0400, Greg Wooledge wrote:
[...]

> Before stretch, if you had heirloom-mailx installed, both "mail -a file"
> and "mailx -a file" worked.  After upgrading to stretch, you'd have an
> s-nail program, but *not* a mail or mailx program, so all your scripts
> would break.  And if you did the obvious thing and installed bsd-mailx
> as a replacement, "mailx -a file" would try to add a header named "file"
> instead of attaching a file named "file".
>
> So, after the stretch upgrade, you would either have to change all of
> your scripts to use s-nail instead of mailx, or

[...]

That brings back memories. I did that and switched to using s-nail by
name for scripts that send me status and backup report emails with
filtered logs as attachments.

--
Tixy

[Gertjan Klein]

Op 17-10-2023 om 23:20 schreef Greg Wooledge:

> On Tue, Oct 17, 2023 at 09:50:13PM +0200, Gertjan Klein wrote:
>> I do appreciate it. If I switch to bsd-mailx I have something that should
>> work. Although I'm concerned by the statement (on the Debian package page)
>> that it doesn't speak SMTP. This is how I send the mail on to my internet
>> provider with mailutils. I wouldn't know how else to do it.
>
> The way Unix mail works is by dividing the workload between an MUA and
> an MTA.
>
> The MUA (Mail User Agent) provides the user interface. It's the program
> you use to send and/or read mail. Examples include elm, pine, mutt, mailx,
> Thunderbird, and so on.
>
> The MTA (Mail Transport Agent) (or Transfer) is the back end. It's the
> part that does the actual deliveries, including sending and receiving
> messages over the Internet.

[...]

Thanks for this explanation. I should have realized this, but got
confused by that above statement. If a MUA uses a MTA for transport, the
MUA doesn't need to speak SMTP, I'd say, so why emphasize that it indeed
doesn't?

> A command-line MUA like mailx sends mail to the MTA by invoking the
> /usr/sbin/sendmail (traditionally /usr/lib/sendmail) program, which is
> a hook provided by the MTA.

I'm configuring bookworm; nullmailer on it provides both.

> So, it's not *expected* that your MUA will speak SMTP directly. That's
> not its job.

Indeed. But:

> [...] This is different

> from when an MTA accepts a message directly from a MUA. That's usually
> called a "submission", and can use either SMTP or /usr/sbin/sendmail.

So a submission from a MUA can use SMTP as well... The MTA could then be
bypassed. I suppose e.g. Thunderbird would do this. So it does perhaps
make sense that a MUA explains whether it needs a MTA.

(More extensive explanation deleted -- thanks!)

> I skimmed parts of it. It appears to read ~/.mailrc if it exists.
> You could try putting commands in there and seeing what happens.

The Debian package comes with an example configuration file, which I
copied to .mailrc. I added the line you suggested; it made no
difference. The configuration documentation (*), BTW, uses a completely
different syntax, resembling JSON.

I got fed up. I installed bsd-mailx and purged mailutils. That didn't
work out of the box either. I don't understand why not, but there's a
lot I don't understand. But the setting from your earlier message did
the trick. Now the From header looks like I believe it should look by
default, and I can move on to the next problem.

Thanks again, kind regards,
Gertjan.

*) https://mailutils.org/manual/html_section/mail.html

[Gertjan Klein]

Op 18-10-2023 om 18:30 schreef David Wright:

> On Tue 17 Oct 2023 at 19:41:43 (+0200), Gertjan Klein wrote:
>> gklein@parvos:~$ cat /etc/passwd | grep gklein
>> gklein:x:1000:1000:Gertjan Klein,,,:/home/gklein:/bin/bash
>
> On my (bullseye) system, that field is what is used to get my full
> name.

It's what I'd expect to be used, unless configured otherwise.

> (Note that I don't personally send mail this way—I use mutt, of course.)

I don't intend to send mail from this machine myself, I want mail from
the system (e.g. unattended-upgrades) delivered to my personal mailbox.

> No, my system uses mailutils:

As you may have seen in my response to Greg, replacing mailutils with
bsd-mailx (and adding a configuration line) did solve the issue. I think
this at least proves that nullmailer did not mess with the From header.

I don't know why your mailutils works when mine doesn't. Perhaps a
difference between the bookworm and bullseye versions, perhaps a
configuration setting somewhere. I gave up; I've spent way more time on
this than I should already. The mail will all be sent to me; it's
ultimately irrelevant what name is in the header. It was just an itch.
(I did learn a few things along the way, though, so it wasn't a waste of
time!)

> I don't know much/anything about nullmailer, or why you chose it.
> (But then, the only reason I use exim4 is because I know how to
> # dpkg-reconfigure exim4-config
> it.)

I used to use exim4, and I have instructions written down for
configuring it. But exim4 is a full-fledged mail server, and I don't
need that (or feel competent to maintain it), so I went looking for
something more lightweight. Nullmailer seemed adequate for my needs.

Thanks, kind regards,
Gertjan.

[Greg Wooledge]

On Thu, Oct 19, 2023 at 01:10:22PM +0200, Gertjan Klein wrote:
> Op 17-10-2023 om 23:20 schreef Greg Wooledge:

> > [...] This is different
> > from when an MTA accepts a message directly from a MUA. That's usually
> > called a "submission", and can use either SMTP or /usr/sbin/sendmail.
>
> So a submission from a MUA can use SMTP as well... The MTA could then be
> bypassed. I suppose e.g. Thunderbird would do this. So it does perhaps make
> sense that a MUA explains whether it needs a MTA.

Traditional Unix terminal- or command-line-based MUAs inject new messages
into the outgoing mail queue by piping them to /usr/sbin/sendmail.

Fancy GUI MUAs like Thunderbird are often written to work on either
Windows or Unix, so they don't always offer the ability to inject through
/usr/sbin/sendmail (because obviously Windows doesn't have that).
In some of these cases -- maybe Thunderbird is one of them, maybe not,
I don't know -- they require a submission MTA to be available. New
messages are injected using SMTP, typically with password-based SMTP AUTH.
That submission MTA might be localhost, or it might be provided by one's
ISP, or it might be something on the public Internet, such as Gmail.

The difference between a submission MTA and a receiving MTA is pretty
subtle. And yes, a single MTA might serve both roles, but not always.
A submission MTA (which does only this job) doesn't accept messages from
outside senders. It only accepts messages from its authorized clients,
which are a known set of people, identified by IP address or by password
or by some other means.

SMTP is a fairly simplistic protocol, and if we ignore encryption and
authentication and various other extensions, it boils down to these steps:

1) The client (MUA or MTA) connects to the server (MTA). The server
identifies itself by sending a banner message.

2) The client identifies itself by sending a HELO message, which is
acknowledged by the server.

3) The client states the next email's sender address by sending a MAIL FROM
message, which is ack'ed by the server.

4) The client states the next email's recipient(s) by sending RCPT TO
messages, each of which is ack'ed by the server.

5) The client sends the next email's body in a DATA message, which is ack'ed.

6) The client either disconnects or goes to step 3 if there's more email.

The difference between a submission MTA and a receiving MTA is how
the MTA responds in steps 3 and 4. A submission MTA might require that
the sender address have a specific @domain part, for example. It will
generally allow *any* recipient addresses, since its job is to accept
mail on behalf of a client, and whisk it away to wherever.

A receiving MTA, on the other hand, probably allows any sender address,
but only allows recipient addresses with locally recognized @domain parts.
This is because the receiving MTA's job is to accept email *from* anywhere
in the universe, but only if it's *for* one of its known users. Anything
else should be rejected.

Of course there are in-between cases, where an MTA is configured to
accept some local deliveries, or to relay email to some specific
external destinations, but not to any other destinations. And then
there's all kinds of anti-spam technology, anti-virus, etc. Things
get ridiculously complex in the real world.

[Stefan Monnier]

> Traditional Unix terminal- or command-line-based MUAs inject new messages
> into the outgoing mail queue by piping them to /usr/sbin/sendmail.
>
> Fancy GUI MUAs like Thunderbird are often written to work on either
> Windows or Unix, so they don't always offer the ability to inject through
> /usr/sbin/sendmail (because obviously Windows doesn't have that).

While I think this explains the history behind it, nowadays most MUAs
need to speak SMTP. This is not just because it has become commonplace
for an MTA to be missing even on Unix machines, but because
spam-checking basically enforces that *machines* aren't trusted to send
mail any more, only users (as an approximation of "human being") are, so
unless you're willing to store your personal password for use by the
system's MTA, the system's MTA will usually not be able to send your
mail to your recipients :-(


Stefan

[Gertjan Klein]

Op 19-10-2023 om 15:00 schreef Greg Wooledge:

> Traditional Unix terminal- or command-line-based MUAs inject new messages
> into the outgoing mail queue by piping them to /usr/sbin/sendmail.

That means, as I believe Stefan pointed out, that they leave
authorization to this single program, for all users of the system. For
me personally that doesn't matter. But I wonder if this would nowadays
be considered unsafe.

I have, for now, configured nullmailer with my username and password at
my provider (over TLS, of course). My provider's MTA (a receiving MTA
then) therefore accepts mail from my VPS.

As an alternative, I can use the mailing system of my VPS supplier. This
requires me to add stuff to the DNS of my domain, stating that mailer
can send mail on behalf of my domain. I believe nullmailer could then
use that server without (other) authorization.

I suppose that latter is more secure, as it doesn't require a configured
password. But setting this up is considerably more difficult.

> Things get ridiculously complex in the real world.

Yes. Wasn't IT supposed to make life simpler? ;)

Kind regards,
Gertjan.

[Greg Wooledge]

On Thu, Oct 19, 2023 at 09:10:50PM +0200, Gertjan Klein wrote:
> Op 19-10-2023 om 15:00 schreef Greg Wooledge:
> > Traditional Unix terminal- or command-line-based MUAs inject new messages
> > into the outgoing mail queue by piping them to /usr/sbin/sendmail.
>
> That means, as I believe Stefan pointed out, that they leave authorization
> to this single program, for all users of the system. For me personally that
> doesn't matter. But I wonder if this would nowadays be considered unsafe.

If one of your users turns out to be a spammer, you eventually discover
this, and lock their account.

Once the spam stops coming from your system, then you can try to get your
IP address removed from all of the DNS spammer lists. Usually there's a
web form you can fill out. Or the listing may expire after a while.

[David Wright]

On Thu 19 Oct 2023 at 13:30:53 (+0200), Gertjan Klein wrote:
> Op 18-10-2023 om 18:30 schreef David Wright:
> > On Tue 17 Oct 2023 at 19:41:43 (+0200), Gertjan Klein wrote:
> > > gklein@parvos:~$ cat /etc/passwd | grep gklein
> > > gklein:x:1000:1000:Gertjan Klein,,,:/home/gklein:/bin/bash
> >
> > On my (bullseye) system, that field is what is used to get my full
> > name.
>
> It's what I'd expect to be used, unless configured otherwise.

Yes, in emails that you and I send. (Both our tests were carried out
from our bash prompts.)

> > (Note that I don't personally send mail this way—I use mutt, of course.)

And I should add that my From: address is a function of several
factors, like who I'm replying to, which mailbox I'm reading,
and so on.

> I don't intend to send mail from this machine myself, I want mail from
> the system (e.g. unattended-upgrades) delivered to my personal
> mailbox.

I wouldn't expect /then/ to see my name, or likely anything at all
from /etc/passwd. Rather, I would expect system programs to set their
own From: address, either read from configuration file or hard-wired
into the program itself, as in for example (bullseye):

$ strings /usr/sbin/cron | grep -A17 'Cron Daemon'
$ strings /usr/sbin/anacron | grep -B5 -A4 'Anacron <'

which displays the origin of their From: lines as I receive them.

It would be interesting to know whether the correct addresses were
being generated by /system/ emails when you were still using
mailutils; and whether they are, now that you're using bsd-mailx.
I've never used unattended upgrades. I have unattended updates and
downloads, but they're my own creations, initiated by root's crontab,
so those emails come from Cron Daemon.

Cheers,
David.

[Gertjan Klein]

Op 20-10-2023 om 05:10 schreef David Wright:

> On Thu 19 Oct 2023 at 13:30:53 (+0200), Gertjan Klein wrote:
>> I don't intend to send mail from this machine myself, I want mail from
>> the system (e.g. unattended-upgrades) delivered to my personal
>> mailbox.
>
> I wouldn't expect /then/ to see my name, or likely anything at all
> from /etc/passwd. Rather, I would expect system programs to set their
> own From: address, either read from configuration file or hard-wired
> into the program itself

I wouldn't expect my gklein changes to be used by system mails either.
Just for fun, I did try to configure .mailrc for root, but that made no
difference (at least for unattended-upgrades). But the default root@
address, with or without a name, is fine with me. I did have to add a
root@ mail alias in my provider settings though, otherwise it rejects
the system mail.

> It would be interesting to know whether the correct addresses were
> being generated by /system/ emails when you were still using
> mailutils; and whether they are, now that you're using bsd-mailx.

II did not receive system emails when I was still using mailutils. I
reconfigured unattended-upgrades to always send mail, and forced a run:

$ sudo unattended-upgrade -d

This gets me a mail with From being just "ro...@parvos.nl". I don't
really mind.

Kind regards,
Gertjan.

[Gertjan Klein]

Op 19-10-2023 om 21:50 schreef Greg Wooledge:

> Once the spam stops coming from your system, then you can try to get your
> IP address removed from all of the DNS spammer lists. Usually there's a
> web form you can fill out. Or the listing may expire after a while.

I've seen many postings over the years from people trying to get their
IP off of a blacklist. Their success was discouraging. (I suppose I
don't know about people for which this was a seamless experience, as
they would have no reason to post.)

Kind regards,
Gertjan.

[David Wright]

On Fri 20 Oct 2023 at 11:51:35 (+0200), Gertjan Klein wrote:
> Op 20-10-2023 om 05:10 schreef David Wright:
> > On Thu 19 Oct 2023 at 13:30:53 (+0200), Gertjan Klein wrote:
> > > I don't intend to send mail from this machine myself, I want mail from
> > > the system (e.g. unattended-upgrades) delivered to my personal
> > > mailbox.
> >
> > I wouldn't expect /then/ to see my name, or likely anything at all
> > from /etc/passwd. Rather, I would expect system programs to set their
> > own From: address, either read from configuration file or hard-wired
> > into the program itself
>
> I wouldn't expect my gklein changes to be used by system mails either.
> Just for fun, I did try to configure .mailrc for root, but that made
> no difference (at least for unattended-upgrades). But the default
> root@ address, with or without a name, is fine with me. I did have to
> add a root@ mail alias in my provider settings though, otherwise it
> rejects the system mail.

That may depend on what the Envelope-from address is, and the policy
of your provider. As you might be able to observe, I use my ISP
account for Envelope-from and authentication, and that account has
no relation to accounts on my machine or my email hosting service.
Such considerations may be the target of the header-rewriting
facilities in nullmailer, exim, etc.

> > It would be interesting to know whether the correct addresses were
> > being generated by /system/ emails when you were still using
> > mailutils; and whether they are, now that you're using bsd-mailx.
>
> II did not receive system emails when I was still using mailutils. I
> reconfigured unattended-upgrades to always send mail, and forced a
> run:
>
> $ sudo unattended-upgrade -d
>
> This gets me a mail with From being just "ro...@parvos.nl". I don't
> really mind.

It's possible that the From: address could be different for each of
the three or four cases: a manual run (as above), by cron, anacron,
or by a systemd service.

Cheers,
David.