Bug#1059649: NTP does not keep accurate time on bookworm

[Rob Janssen]

Package: snmptrapd
Version: 5.9.3+dfsg-2

I upgraded a system from bullseye to bookworm.
It had snmptrapd installed.
Before the upgrade, all was OK.

After the upgrade, the snmptrapd service no longer starts.
The error message is: couldn't open udp:162 -- errno 13 ("Permission denied")

It appears that this is caused by the service being started as user Debian-snmp
and thus not being able to open privileged ports.
I could work around it by editing the systemd service file to run snmptrapd as
User=root but this is likely not what was intended.

To cover possible upgrade issues I removed and purged the snmptrapd package
and re-installed it, but the new install suffered the same issue.

[Rob Janssen]

Sorry, title of this bug report is wrong!  (due to re-using an existing mail)

Should be: snmptrapd does not start on bookworm

[Craig Small]

On Sat, 30 Dec 2023 at 06:15, Rob Janssen <deb...@pe1chl.nl> wrote:

After the upgrade, the snmptrapd service no longer starts.
The error message is: couldn't open udp:162 -- errno 13 ("Permission denied")

Could you tell me how you start snmptrapd?

There are two ways:

The default systemd way. The socket is created with a snmptrap.socket and then passed onto the snmpdtrap server. I have tested this just now and it works fine with 5.9.4

The init way. In this case snmptrapd starts as root and changes to Debian-snmp after binding to the sockets.

It appears that this is caused by the service being started as user Debian-snmp

and thus not being able to open privileged ports.

As you can see, it doesn't need to normally.

 

To cover possible upgrade issues I removed and purged the snmptrapd package
and re-installed it, but the new install suffered the same issue.

What, specifically, are you doing to see this message? Running what you see in the service file on the command line won't work and will give this result.

 - Craig

 

[Rob Janssen]

On 2024-02-01 22:09, Craig Small wrote:

On Sat, 30 Dec 2023 at 06:15, Rob Janssen <deb...@pe1chl.nl> wrote:

After the upgrade, the snmptrapd service no longer starts.
The error message is: couldn't open udp:162 -- errno 13 ("Permission denied")

Could you tell me how you start snmptrapd?

There are two ways:

The default systemd way. The socket is created with a snmptrap.socket and then passed onto the snmpdtrap server. I have tested this just now and it works fine with 5.9.4

The init way. In this case snmptrapd starts as root and changes to Debian-snmp after binding to the sockets.

I am using systemd.

 

To cover possible upgrade issues I removed and purged the snmptrapd package
and re-installed it, but the new install suffered the same issue.

What, specifically, are you doing to see this message? Running what you see in the service file on the command line won't work and will give this result.

At first I just upgraded a bullseye system where everything was working, to bookworm, and it failed to start snmptrapd.
Then I did the uninstall/re-install of snmptrapd and it still failed.

Maybe the issue is that this system does not have IPv6 enabled?  In sysctl.conf it has:
net.ipv6.conf.all.disable_ipv6 = 1

That has effect on the default snmpd service as well, but I work around that by modifying the listen
port in the config file.
The snmptrapd.service file has "udp:162" and "udp6:162" args in the ExecStart but maybe these aren't used at all
in this config?  In bullseye I think it worked out of the box.

It would be more friendly when snmpd and snmptrapd degrade gracefully to IPv4-only when there is no IPv6 on the system.

Rob

[Craig Small]

On Fri, 2 Feb 2024 at 08:54, Rob Janssen <deb...@pe1chl.nl> wrote:

I am using systemd.

Where are you seeing this error? The systemd socket is the thing that opens up the socket, so shouldn't matter what the snmptrapd process is running as.

When I reboot, I get this:

$ sudo ss -unlp | grep 162
UNCONN 0      0             0.0.0.0:162        0.0.0.0:*    users:(("systemd",pid=1,fd=83))        
UNCONN 0      0                [::]:162           [::]:*    users:(("systemd",pid=1,fd=84))        
$ pgrep snmptrapd

So systemd is listening to the socket and snmptrapd is not even running.

After I send a trap:

$ sudo ss -unlp | grep 162
UNCONN 0      0             0.0.0.0:162        0.0.0.0:*    users:(("snmptrapd",pid=7296,fd=3),("systemd",pid=1,fd=83))
UNCONN 0      0                [::]:162           [::]:*    users:(("snmptrapd",pid=7296,fd=4),("systemd",pid=1,fd=84))
$ pgrep snmptrapd
7296

Maybe the issue is that this system does not have IPv6 enabled?  In sysctl.conf it has:
net.ipv6.conf.all.disable_ipv6 = 1

I added that and rebooted, it worked fine.

 

That has effect on the default snmpd service as well, but I work around that by modifying the listen
port in the config file.
The snmptrapd.service file has "udp:162" and "udp6:162" args in the ExecStart but maybe these aren't used at all
in this config?  In bullseye I think it worked out of the box.

In a way, they're used.

In a proper systemd socket activation, the service doesn't need to know or define the ports it's listening to, they just get a bunch of file descriptors and use those.

So we could have a command line in the service file with no ports and it looks like it works; but that is only because snmptrapd is sort-of looking for UDP port 162.

If you change the socket definition to another port, say 164 you get this:

Feb 02 15:57:37 elmo snmptrapd[9123]: couldn't open udp:162 -- errno 13 ("Permission denied")
Feb 02 15:57:37 elmo snmptrapd[9123]: couldn't open udp:162 -- errno 13 ("Permission denied")

system opens port 164, a trap to port 164 starts snmptrapd which not only listens to the systemd FD, but opens its own ports too running as a non-priv user.

 OK, so change the service definiton to listen to some port above 1024, it should be ok because it can open that port and use the systemd FD. For some reason

it doesn't (as in it starts but there are no messages).

What that means is the ports that are defined in the socket file need to match the ports in the service file. snmptrapd is shipped with that:

$ grep 162 debian/snmptrapd.s*
debian/snmptrapd.service:ExecStart=/usr/sbin/snmptrapd -LOw -f udp:162 udp6:162
debian/snmptrapd.socket:ListenDatagram=0.0.0.0:162
debian/snmptrapd.socket:ListenDatagram=[::]:162
debian/snmptrapd.socket:# ListenStream=0.0.0.0:162
debian/snmptrapd.socket:# ListenStream=[::]:162

It would be more friendly when snmpd and snmptrapd degrade gracefully to IPv4-only when there is no IPv6 on the system.

I don't think its that. 

 - Craig

[Rob Janssen]

Craig,

I'm sorry but I know absolutely nothing about that "systemd socket" mechanism.  I am an old hat
that grew up with init and hates systemd.
Maybe I have inadvertently broken something, I only know what I have written before and that it does
not work for me without forcing the service to run as root instead of Debian-snmp.
What exactly is going on now is unclear to me.

Rob