Ubuntu 22.04 tacspooflog.pl syslog-ng

[Jade Rampulla]

Hey Marc,

tac_plus-ng version 9a4766920c9bb912c99e69645bc3d256dce50480

syslog-ng version 3.35.1

Ubuntu 22.04.3

I'm having trouble getting tacspooflog.pl to send accounting logs to syslog-ng.

tac_plus-ng.conf

id = spawnd {
        listen = { port = 49 }
}
id = tac_plus-ng {
        device any {
                key = "PSK"
                address = 0.0.0.0/0
        }
        log acctlog {
                destination = "|exec sudo /etc/tac_plus-ng/tacspooflog.pl 127.0.0.1"
                destination = /var/log/syslog-ng/tac_plus-ng.log
        }
        accounting log = acctlog
}

I'm getting logs in /var/log/syslog-ng/tac_plus-ng.log.

syslog-ng.conf

source s_net_switches { udp(ip("10.10.10.10") port(50000)); };
source s_net_catchall { udp(ip("10.10.10.10") port(514)); };
source s_net_tacacs { udp(ip("127.0.0.1") port(514)); };

destination d_syslog_file_switches { file("/var/log/syslog-ng/syslog-switches.log"); };
destination d_syslog_file_catchall { file("/var/log/syslog-ng/syslog-catchall.log"); };

log { source(s_net_tacacs); destination(d_syslog_file_catchall); };
log { source(s_net_catchall); filter(f_catchall); destination(d_syslog_file_catchall); };

I'm getting logs from switches. I'm also able to match the catchall logging if I send to 10.10.10.10:514. Nothing ever shows up from tacacs though.

Not getting errors when I manually run the script

sudo perl /etc/tac_plus-ng/tacspooflog.pl
Usage:   /etc/tac_plus-ng/tacspooflog.pl [-f <facility>] [-l <level>] [-i <ident>] <dst-ip>
Example: /etc/tac_plus-ng/tacspooflog.pl -f LOG_AUTH -l LOG_INFO -i tac_plus 127.0.0.1

Any ideas what I can try?

Thanks in advance.

[Jade Rampulla]

Oops I forgot to put this line from syslog-ng.conf in my previous message:

log { source(s_net_switches); destination(d_syslog_file_switches); };

[Marc Huber]

Hi Jade,

the first thing I'd check is whether tacspooflog.pl is actually sending syslog packets (via strace) and that these packets arrive at the loopback interface (via tcpdump).

Cheers,

Marc

--
You received this message because you are subscribed to the Google Groups "Event-Driven Servers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to event-driven-ser...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/event-driven-servers/ba7df28a-ceb6-4fc9-92bf-1d5dbc655ae0n%40googlegroups.com.

[Jade Rampulla]

Thanks for getting back to me so quickly.

I haven't used strace before and not sure if the output has anything meaningful.

"ps aux | grep tac" gave me this:

root       22401  0.0  0.0   3412  1848 ?        Ss   Aug11   0:14 tac_plus-ng: 0 connections, accepting up to 480 more
root       24461  0.0  0.1   3588  2744 ?        Ss   Aug11   0:04 tac_plus-ng: 0 connections
root       24462  0.0  0.1   3588  2728 ?        Ss   Aug11   0:21 tac_plus-ng: 0 connections

Started strace on PID 22401:

sudo strace -f -p 22401

Ran the command "int eth1/1" on a switch. Got this output in "/var/log/syslog-ng/tac_plus-ng.log":

2023-08-14 12:15:04 -0400       192.168.10.10 jade        0       10.10.100.100      stop            configure terminal ; interface Ethernet1/1 (REDIRECT)
2023-08-14 12:15:04 -0400       192.168.10.10 jade        0       10.10.100.100      stop            configure terminal ; interface Ethernet1/1 (SUCCESS)

Got this output from strace:

[{events=EPOLLIN, data={u32=5, u64=30064771077}}], 128, 10001) = 1
accept(5, {sa_family=AF_INET6, sin6_port=htons(58482), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::ffff:192.168.10.10", &sin6_addr), sin6_scope_id=0}, [112 => 28]) = 6
fcntl(6, F_GETFD)                       = 0
fcntl(6, F_SETFD, FD_CLOEXEC)           = 0
setsockopt(6, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
sendmsg(12, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\6\0\0\0\0\0\0\0\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", iov_len=32}], msg_iovlen=1, msg_control=[{cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, cmsg_data=[6]}], msg_controllen=20, msg_flags=0}, MSG_NOSIGNAL) = 32
close(6)                                = 0
epoll_wait(4, [{events=EPOLLIN, data={u32=5, u64=30064771077}}], 128, 6720) = 1
accept(5, {sa_family=AF_INET6, sin6_port=htons(58560), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::ffff:192.168.10.10", &sin6_addr), sin6_scope_id=0}, [112 => 28]) = 6
fcntl(6, F_GETFD)                       = 0
fcntl(6, F_SETFD, FD_CLOEXEC)           = 0
setsockopt(6, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
sendmsg(11, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\6\0\0\0\0\0\0\0\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", iov_len=32}], msg_iovlen=1, msg_control=[{cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, cmsg_data=[6]}], msg_controllen=20, msg_flags=0}, MSG_NOSIGNAL) = 32
close(6)                                = 0
epoll_wait(4, [{events=EPOLLIN, data={u32=12, u64=21474836492}}], 128, 6691) = 1
recvmsg(12, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\0\0\0\0", iov_len=32}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 4
epoll_wait(4, [{events=EPOLLIN, data={u32=11, u64=21474836491}}], 128, 6691) = 1
recvmsg(11, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\0\0\0\0", iov_len=32}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 4
epoll_wait(4, ^Cstrace: Process 22401 detached
 <detached ...>

[Jade Rampulla]

Should this work?

root@ubuntu-vm:~# echo "TEST" | exec sudo /etc/tac_plus-ng/tacspooflog.pl 127.0.0.1
Use of uninitialized value $saddr in pattern match (m//) at /usr/lib/x86_64-linux-gnu/perl5/5.34/Net/RawIP.pm line 480, <STDIN> line 1.
root@ubuntu-vm:~#

[Marc Huber]

Hi Jade,

could you please attach strace to one of the running tacspooflog.pl Perl processes? That should indicate that a packet was sent, see below.

Also, "tcpdump -ni lo port 514" should show that packet.

Thanks,

Marc

$ ps ax | grep tac
 147481 pts/0    S+     0:00 tac_plus-ng: 0 connections, accepting up to 1920 more
 147515 pts/0    S+     0:00 sudo /home/ubuntu/DEVEL/PROJECTS/tac_plus-ng/extra/tacspooflog.pl 1
 147518 pts/2    Ss+    0:00 sudo /home/ubuntu/DEVEL/PROJECTS/tac_plus-ng/extra/tacspooflog.pl 1
 147519 pts/2    S      0:00 perl /home/ubuntu/DEVEL/PROJECTS/tac_plus-ng/extra/tacspooflog.pl 1
 147534 pts/1    S+     0:00 grep --color=auto tac
$ sudo strace -p 147519
strace: Process 147519 attached
read(0, "2023-08-14 18:31:49 +0200\t172.16"..., 8192) = 74
newfstatat(AT_FDCWD, "/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=83, ...}, 0) = 0
newfstatat(AT_FDCWD, "/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=83, ...}, 0) = 0
socket(AF_INET, SOCK_RAW, IPPROTO_RAW)  = 3
setsockopt(3, SOL_IP, IP_HDRINCL, [1], 4) = 0
sendto(3, "E\20\0L\0\0@\0@\21\16\222\254\20\0\356\177\0\0\1\270 \2\2\08\340 <38>"..., 76, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("127.0.0.1")}, 16) = 76
close(3)                                = 0
read(0, ^Cstrace: Process 147519 detached
 <detached ...>

To view this discussion on the web visit https://groups.google.com/d/msgid/event-driven-servers/da9e0742-eecc-4ea8-940c-21afe85e0aa0n%40googlegroups.com.

[Marc Huber]

Hi,

no, the assumption for input is that

        $in =~ /^[^\t]+\t([^\t]+)\t(.*)$/ or $in =~ /\s+(\S+):\s+(.*)$/;

matches, and that the network device IP address can be found after the first tab or within a space-ip-colon sequence.

printf "whatever\t1.2.3.4\twhatever"|exec ...

likely works (untested).

Cheers,

Marc

To view this discussion on the web visit https://groups.google.com/d/msgid/event-driven-servers/5dd24227-0215-4041-8790-cdeb81f7fa1en%40googlegroups.com.

[Jade Rampulla]

I don't see any tacspooflog.pl processes running in ps, just the three tac_plus-ng processes. I used this to start the process:

sudo tac_plus-ng -b /etc/tac_plus-ng/tac_plus-ng.conf

Should I start it a different way?

Is there another way to find the tacspooflog.pl PID for an strace?

Also tried tcpdump with your test command:

printf "TEST\t10.10.10.10\tDATA"|exec sudo /etc/tac_plus-ng/tacspooflog.pl 127.0.0.1

which gave tcpdump output

tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on lo, link-type EN10MB (Ethernet), snapshot length 262144 bytes
12:59:58.883128 IP 10.10.10.10.59788 > 127.0.0.1.514: SYSLOG auth.info, length: 18

Unfortunately nothing showed up in tcpdump when I tried "int eth1/1" from a switch again.

[Jade Rampulla]

Forgot to mention the test command that generated tcpdump data, didn't generate a log entry in /var/log/syslog-ng/syslog-catchall.log

[Marc Huber]

Hi Jade,

tacspooflog.pl is a Perl process, searching for "perl" should give a result. Or, just "ps ax | grep tac" as I showed in my example.

I've just run a quick test, and

$ printf "TEST\t10.10.10.10\tDATA"| sudo strace -s 500 -e sendto -f ./tac_plus-ng/extra/tacspooflog.pl 127.0.0.1
sendto(3, "E\20\0.\0\0@\0@\21\247\232\n\n\n\n\177\0\0\1\220;\2\2\0\32\325\262<38>tac_plus: DATA", 46, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("127.0.0.1")}, 16) = 46

seems to work quite well (UDP packet, sent to port 514, with priority <38> prepended. Plus, as you've seen, tcpdump shows the packet, so everything looks fine. That <38> is LOG_AUTH (4 << 3) + LOG_INFO (6), so that looks quite well, too.

I think I've tested this years ago, ten years ago with rsyslogd. I just tried again, but, sadly, it doesn't seem to work any more.

I suspected some Linux reverse-path filtering to drop the packets, but disabling RPF doesn't seem to help.

I may or may not find the time to analyze this any further.

Cheers,

Marc

To view this discussion on the web visit https://groups.google.com/d/msgid/event-driven-servers/d0975005-e0ad-4607-a3b6-bbe994a52d54n%40googlegroups.com.

[Jade Rampulla]

Thanks Marc.

Last time I used the script was on CentOS maybe 7 years ago and it worked great. Also used syslog-ng with that setup.

Not sure if this helps, but logger is able to send messages locally and they generate a message in /var/log/syslog-ng/syslog-catchall.log:

logger -n 127.0.0.1 "HELLO"

echo "HELLO" | logger -n 127.0.0.1

which generated these messages:

Aug 14 14:20:15 127.0.0.1 1 2023-08-14T14:20:15.204230-04:00 ubuntu-vm jade - - [timeQuality tzKnown="1" isSynced="1" syncAccuracy="6000"] HELLO
Aug 14 14:20:46 127.0.0.1 1 2023-08-14T14:20:46.236419-04:00 ubuntu-vm jade - - [timeQuality tzKnown="1" isSynced="1" syncAccuracy="21500"] HELLO

I tried a couple flavors of that in config, but unfortunately didn't help:

tac_plus-ng.conf

id = spawnd {
        listen = { port = 49 }
}
id = tac_plus-ng {
        device any {
                key = "PSK"
                address = 0.0.0.0/0
        }
        log acctlog {

                destination = "logger -n 127.0.0.1"
                destination = "|logger -n 127.0.0.1"
                destination = "exec sudo logger -n 127.0.0.1"
                destination = "|exec sudo logger -n 127.0.0.1"

                destination = /var/log/syslog-ng/tac_plus-ng.log
        }
        accounting log = acctlog
}

[Marc Huber]

Hi Jade,

I think the packets sent by tacspooflog.pl never make it to the input queue of the local Linux system.

To work around this you can setup a network namespace with a veth pair and reverse-path filtering disabled:

ip netns add spooflog
ip link add name spooflog0 type veth peer name spooflog1
ip link set spooflog1 netns spooflog
ip addr add 100.64.0.1/30 dev spooflog0
ip netns exec spooflog ip addr add 100.64.0.2/30 dev spooflog1
ip link set spooflog0 up
ip netns exec spooflog ip link set spooflog1 up
echo 0 > /proc/sys/net/ipv4/conf/spooflog0/rp_filter

Then run the script in that namespace:

   destination = "|exec /usr/bin/ip netns exec spooflog /path/to/tacspooflog.pl 100.64.0.1"

Cheers,

Marc

To view this discussion on the web visit https://groups.google.com/d/msgid/event-driven-servers/cb78138b-a573-4169-a47d-e6e1cf4f5f04n%40googlegroups.com.