Bug#1020328: Native systemd units

[Trent W. Buck]

Package: logcheck
Version: 1.3.23
Severity: wishlist

Please find attached a logcheck.timer and logcheck.service.
I just wrote them; they Work For Me™ so far.

If you just ship these, systemd-cron will automatically skip /etc/cron.d/logcheck.
Vixie cron might need something like this to manually skip /etc/cron.d/logcheck:

if ! [ -e /run/systemd ]; then ... ; fi

The "Security hardening" section is too aggressive to work for postfix, exim, &c.
If you ship this, you'll want to skip some/all of the hardening for portability reasons.

logcheck calls /usr/bin/mime-construct, which ONLY supports
/usr/sbin/sendmail. Replacing /usr/sbin/sendmail with SMTP
(e.g. python3's smtplib) is one way to keep maximum systemd-level
hardening, without needing a bunch of MTA-specific workarounds.

-- System Information:
Debian Release: 11.5
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'proposed-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 5.18.0-0.bpo.1-amd64 (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

[Trent W. Buck]

UPDATE: a debian/logcheck.tmpfiles (/etc/tmpfiles.d/logcheck.conf) is also needed.
The security hardening I added prevents logcheck from creating it.
See attached.

[Richard Lewis]

Hi trent - i am interested in this approach:

i see you are binding msmtp over /usr/sbin/sendmail - i dont
understand how this would lead to a different outcome: how else does
msmtp know where to send the mail? is there some implicit assumption
about local delivery here? i tried testing but msmtp does not work at
all out of the box for me - complains that it has no configuration
file)

As far as i can tell, the issue isn't with the "send mail" part, but
the part where the mta (exim/postfix) tries to deliver it


(returning to logcheck - id agree with losing the 'reboot' mail -
it's pretty clear from the log message that the system has been
rebooted, so not sure what value this is even under cron. I'd suggest
adding Persistent=true in the .timer's [Timer] so that it runs after a
resuming from suspend. Perhaps set it to skip if on battery too )

[Trent W. Buck]

On Wed 09 Nov 2022 19:29:56 +1100, Trent W. Buck wrote:
> In short, what I'm saying is:
>
> 1. you can't harden a script/daemon that uses the "fork+exec /usr/sbin/sendmail" API, because
> different /usr/sbin/sendmail implementations (e.g. postfix) require different privileges.
>
> In particular, "requires setgid" prevents ALL of the following hardening options:
>
> DynamicUser LockPersonality MemoryDenyWriteExecute
> NoNewPrivileges PrivateDevices ProtectClock
> ProtectHostname ProtectKernelLogs ProtectKernelModules
> ProtectKernelTunables RestrictAddressFamilies RestrictNamespaces
> RestrictRealtime RestrictSUIDSGID SystemCallArchitectures
> SystemCallFilter SystemCallLog
>
> 2. the smtp://localhost:25 API is usually available.
>
> It prevents fewer hardening options:
>
> PrivateNetwork=yes
> IPAddressDeny=any
> RestrictAddressFamilies=~AF_TCP
>
> Basically you have to leave TCP/IP unblocked, but that's all.

I made a minor braino here, it should be AF_INET AF_INET6 not AF_TCP.

My old (Debian 9) notes about different techniques are here:

https://github.com/cyberitsolutions/prisonpc-systemd-lockdown/tree/main/systemd/system/0-EXAMPLES

30-allow-mail-msmtp.conf: # → Overall exposure level: 1.0 OK 🙂

30-allow-mail-postfix-via-msmtp.conf: # → Overall exposure level: 0.9 SAFE 😀

30-allow-mail-postfix-root-dac-override.conf: # → Overall exposure level: 1.1 OK 🙂
30-allow-mail-postfix-root-sys-admin.conf: # → Overall exposure level: 1.4 OK 🙂

30-allow-mail-postfix-non-root-addgroup.conf: # → Overall exposure level: 0.5 SAFE 😀
30-allow-mail-postfix-non-root-dac-override.conf: # → Overall exposure level: 0.9 SAFE 😀
30-allow-mail-postfix-non-root-setgid.conf: # → Overall exposure level: 2.4 OK 🙂

[Trent W. Buck]

On Fri 04 Nov 2022 00:45:52 +0000, Richard Lewis wrote:
> Hi trent - i am interested in this approach:
>
> i see you are binding msmtp over /usr/sbin/sendmail - i dont
> understand how this would lead to a different outcome: how else does
> msmtp know where to send the mail? is there some implicit assumption
> about local delivery here? i tried testing but msmtp does not work at
> all out of the box for me - complains that it has no configuration
> file)

If you install msmtp you need to configure it.
Just as if you installed postfix, you would need to configure it.

"dpkg-reconfigure msmtp" ought to prompt you, but
here are some basic examples:

1. $ sudo apt install msmtp postfix

$ cat >/etc/msmtprc <<'EOF'
# Send everything to postfix (smtp://localhost:25)
account default
syslog on
auto_from on
host localhost
EOF

2. $ sudo apt install msmtp-mta

$ cat >/etc/msmtprc <<'EOF'
# Send everything to gmail (no "real" MTA on localhost)
account default
syslog on
auto_from on
host smtp.gmail.com
port 587
tls on
auth on
user alice
passwordeval /usr/bin/cat /etc/secret.gmail.password
EOF

$ printf swordfish >/etc/secret.gmail.password

$ chmod 640 /etc/secret.gmail.password
$ chown -h root:logcheck /etc/secret.gmail.password

> As far as i can tell, the issue isn't with the "send mail" part, but
> the part where the mta (exim/postfix) tries to deliver it

I don't know about exim.

When /usr/sbin/sendmail is implemented by postfix (i.e. "apt install postfix),

1. sendmail calls postdrop
2. postdrop is sgid postdrop, so now you run with elevated privileges
3. postdrop writes to /var/spool/postfix/maildrop, which normal users can't write to

Note the sgid bit:

-rwxr-xr-x 1 root root /usr/sbin/sendmail
-r-xr-sr-x 1 root postdrop /usr/sbin/postdrop
drwx-wx--T 2 postfix postdrop /var/spool/postfix/maildrop

If you use systemd hardening NoNewPrivileges=yes, that DISABLES SETGID -- by design.

So logcheck.service run /usr/bin/logcheck
which runs /usr/bin/mail
which runs /usr/sbin/sendmail
which runs /usr/sbin/postdrop
which DOESN'T get escalated privileges (group postdrop)
which FAILS to write to /var/spool/postfix/maildrop/<random file name>.

By telling logcheck.service "actually just use msmtp", the path instead becomes

logcheck.service runs /usr/bin/logcheck
which runs /usr/bin/mail
which runs /usr/sbin/sendmail (actually /usr/bin/msmtp)
which connects to (say) smtp://localhost:25 or smtp://smtp.gmail.com:587

Thereafter, the rest of the flow (whatever is listening on
localhost:25) is not running inside the logcheck.service hardened
namespace/cgroup. So it can do whatever it wants.

In short, what I'm saying is:

1. you can't harden a script/daemon that uses the "fork+exec /usr/sbin/sendmail" API, because
different /usr/sbin/sendmail implementations (e.g. postfix) require different privileges.

In particular, "requires setgid" prevents ALL of the following hardening options:

DynamicUser LockPersonality MemoryDenyWriteExecute
NoNewPrivileges PrivateDevices ProtectClock
ProtectHostname ProtectKernelLogs ProtectKernelModules
ProtectKernelTunables RestrictAddressFamilies RestrictNamespaces
RestrictRealtime RestrictSUIDSGID SystemCallArchitectures
SystemCallFilter SystemCallLog

2. the smtp://localhost:25 API is usually available.

It prevents fewer hardening options:

PrivateNetwork=yes
IPAddressDeny=any
RestrictAddressFamilies=~AF_TCP

Basically you have to leave TCP/IP unblocked, but that's all.

3. msmtp is a quick and easy way to convert (1) to (2).

4. "apt install msmtp-mta" does (3) easily, but
won't work if a "real" MTA is already installed.

5. BindReadOnlyPaths=/usr/bin/msmtp:/usr/sbin/sendmail does (3), and
works even if a "real" MTA is installed.

[Richard Lewis]

On Wed, 9 Nov 2022, 08:37 Trent W. Buck, <tren...@gmail.com> wrote:

My old (Debian 9) notes about different techniques are here:
    https://github.com/cyberitsolutions/prisonpc-systemd-lockdown/tree/main/systemd/system/0-EXAMPLES  

This is an amazing resource! (did you consider trying to introduce it into a debian package somehow?)

I have been studying and experimenting - and learning a lot.  

For exim4, i found that it depends on

- whether the unit is 'oneshot' - if so the unit needs to ensure exim has delivered the mail before the script exits - adding a small 'sleep' is enough - i think otherwise systemd gets confused about what process to monitor if the script causes exim to launch.

- whether or not the unit runs as root or a different user. With User=root you can get away with more hardening directives, but i think better to continue running as a non-root user

This all makes me want to abandon exim for postfix.... but exim is still debian's default.

I think this is consistent with what you found for other mtas, but for future reference and to help people searching for how to use exim in a systemd unit, ive been using the following with exim:

------------------------------------------------

ExecStart=/usr/sbin/logcheck

ExecStart=sleep 0.1s

User=logcheck

UMask=0066

ProtectSystem=strict
ReadWritePaths=/var/lib/logcheck

# for exim - probably debian should allow all of /var/spool and /var/log here
ReadWritePaths=-/var/spool/exim4 -/var/mail -/var/log/exim4

# ProtectHome=true is possible, but the message will be 
# frozen as exim wants to cd to $HOME (for .forward)    
ProtectHome=read-only
PrivateTmp=true
PrivateMounts=true
                                                                                                                                   
DevicePolicy=strict                                                                                                                
# *cannot set: PrivateDevices=true
DeviceAllow=/dev/stdout w
DeviceAllow=/dev/stdin r   
DeviceAllow=/dev/stderr w
DeviceAllow=/dev/null rw  
ProtectProc=invisible
ProcSubset=pid

RemoveIPC=true

ProtectControlGroups=true

AmbientCapabilities=

# exim needs to change ownership of mail - both when it   
# receives the mail and when it delivers it to the local user
# see capabilities(7)  
CapabilityBoundingSet=CAP_SETGID      
CapabilityBoundingSet=CAP_SETUID    
CapabilityBoundingSet=CAP_FSETID 
CapabilityBoundingSet=CAP_CHOWN 
CapabilityBoundingSet=CAP_DAC_OVERRIDE 
CapabilityBoundingSet=CAP_FOWNER

# Anything that implies NoNewPrivileges cannot be set

# cannot set: NoNewPrivileges=yes

# cannot set: DynamicUser=yes
# cannot set: PrivateUsers=true

# *cannot set: RestrictNamespaces=true

# *cannot set: LockPersonality=true 

# *cannot set: ProtectKernelModules=true        
# *cannot set: ProtectKernelLogs=true                
# *cannot set: ProtectHostname=true      
# *cannot set: ProtectClock=true           
# *cannot set: RestrictRealtime=true 
# *cannot set: MemoryDenyWriteExecute=true  
# *cannot set - and would break remote delivery: RestrictAddressFamilies=AF_INET AF_INET6

# *cannot set: RestrictSUIDSGID=true

# *cannot set:  SystemCallArchitectures=native

# *cannot set: SystemCallFilter=@system-service

# cannot set (due to chown): SystemCallFilter=~@privileged

# directives with a * can be set if we use User=root instead of User=logcheck

-------------------------------------------------------------

This still needs the tmpfiles.d dropin from your earlier email. However, i wonder if it is better to make logcheck use a single dir for both the lockfile (currently /run/lock/logcheck) and the scratch dir ( currently created in /tmp each time) and then we could use a single RuntimeDirectory=logcheck with RuntimeDirectoryPreserve=yes (to ensure it is not deleted and no clashes if logcheck dies it would retain the current code in logcheck to use a random subdir and delete it on success)

[Trent W. Buck]

On Wed 07 Dec 2022 23:46:43 +0000, Richard Lewis wrote:
> I have been studying and experimenting - and learning a lot.

> For exim4, i found ⋯

I slurped your exim notes into my repo.
I probably won't do any actual testing with exim myself :-)

https://github.com/cyberitsolutions/prisonpc-systemd-lockdown/commits/main/systemd/system/0-EXAMPLES/30-allow-mail-exim.conf

I was indeed originally plannign to push it into Debian as a kind of "apt install increased-hardening" package.
But I ran into enough nitpicking and static, my current approach is to instead try to work on individual packages, and
try to push upstreams to be more hardened by default.

e.g. SyscallFilters=@foo isn't backwards-compatible with old systemd, so
you can't use it if you don't know a minimum systemd version.

e.g. Architecture=native works fine until someone does something like "apt install curl:armhf".

e.g. the whole "if you call /usr/sbin/sendmail, everything becomes messy"

> - whether the unit is 'oneshot' - if so the unit needs to ensure exim has
> delivered the mail before the script exits - adding a small 'sleep' is
> enough - i think otherwise systemd gets confused about what process to
> monitor if the script causes exim to launch.

That doesn't sound right, but I suppose it's possible.
KillMode=process might trigger that.

> - whether or not the unit runs as root or a different user. With User=root
> you can get away with more hardening directives, but i think better to
> continue running as a non-root user

I think you'll find this is just because User=root implicitly disables a bunch of the settings.
If you set User=root, then "systemctl daemon-reload" then "systemd-analyze security foo.service",
you will see a bunch of stuff like:

Service runs as root, option does not matter
Service runs as root, option does not apply

PS: "systemd-analyze syscall-filter" is a good thing to look at when dealing with chown/seteuid.