Bug#839146: suricata failures with systemd

[Arturo Borrero Gonzalez]

Hi Peter,

thanks for your detailed report. It's really appreciated from the
maintainer point of view.

UNIX socket
===========

Regarding the UNIX socket path, I would like to note that the default
in debian is (should be):
* /var/run/suricata-command.socket

Therefore, your issues with suricata looking for
/var/run/suricata/suricata-command.socket are perhaps
related to a previous version of suricata?

I just tested with suricata 3.1.2-2~bpo8+1 in a jessie system and with
3.1.2-2 in a sid system
and suricatasc works out of the box.

Are you sure the reason of your failures are the wrong socket path?

I don't know why your suricata looks for the socket in another place.

oinkmaster
==========

Yes, the updater script looks for the socket in the default path, which is:
* /var/run/suricata-command.socket

I've not tested to run suricata with a different user apart of the
default, which is root.

ExecReload suricatasc
=====================

Again, it seems is the same issue with the socket path.

/etc/default/suricata
=====================

The /etc/default/suricata file is for running suricata with sysvinit.

If you use systemd then this file is ignored with the debian default
configuration for suricata.

$PID instead of $MAINPID
========================

I just tested this here and I see no issues. The systemd.service(5)
manpage refers to $MAINPID

Could you please give more info?

Here is an example of my debian jessie system:

$ sudo systemctl reload suricata
$ sudo systemctl status suricata
* suricata.service - Suricata IDS/IDP daemon
Loaded: loaded (/lib/systemd/system/suricata.service; disabled)
Active: active (running) since Thu 2016-09-29 16:06:05 CEST; 12min ago
Docs: man:suricata(8)
man:suricatasc(8)
https://redmine.openinfosecfoundation.org/projects/suricata/wiki
Process: 26052 ExecReload=/bin/kill -HUP $MAINPID (code=exited,
status=0/SUCCESS)
Process: 26050 ExecReload=/usr/bin/suricatasc -c reload-rules
(code=exited, status=0/SUCCESS)
Main PID: 25443 (Suricata-Main)
CGroup: /system.slice/suricata.service
`-25443 /usr/bin/suricata -D --af-packet -c
/etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid

Sep 29 16:06:05 debsolid suricata[25442]: 29/9/2016 -- 16:06:05 -
<Notice> - This is Suricata version 3.1.2 RELEASE
Sep 29 16:18:39 debsolid systemd[1]: Reloading Suricata IDS/IDP daemon.
Sep 29 16:18:39 debsolid suricatasc[25946]: {"message": "done", "return": "OK"}
Sep 29 16:18:39 debsolid systemd[1]: Reloaded Suricata IDS/IDP daemon.
[...]

As you can see, both kill and suricatasc works. This is a debian
jessie box with the suricata from backports fresh-installed.
--
Arturo Borrero González

[Peter Viskup]

Hello Arturo,

thank you for quick response.

UNIX socket
===========

Regarding the UNIX socket path, I would like to note that the default
in debian is (should be):
 * /var/run/suricata-command.socket

Therefore, your issues with suricata looking for
/var/run/suricata/suricata-command.socket are perhaps
related to a previous version of suricata?

No - it is related to run suricata under 'suri' user.

I had to create /var/run/suricata directory to let suricata user create suricata-command.socket socket. The regular user does not have write permissions to /var/run directory.

Other possibility is to create <default> socket in ExecStartPre and change it's permissions to grant suri user read-write access (not tested).

oinkmaster
==========

Yes, the updater script looks for the socket in the default path, which is:
 * /var/run/suricata-command.socket

I've not tested to run suricata with a different user apart of the
default, which is root.

And that's the point - maybe described not in enough details by me.

ExecReload suricatasc
=====================

Again, it seems is the same issue with the socket path.

Yes it is.

 

$PID instead of $MAINPID
========================

I just tested this here and I see no issues. The systemd.service(5)
manpage refers to $MAINPID

Could you please give more info?

Arturo Borrero González

These are my tries to reload:

~# /etc/init.d/suricata reload
Reloading suricata configuration (via systemctl): suricata.serviceJob for suricata.service failed. See 'systemctl status suricata.service' and 'journalctl -xn' for details.
 failed!
~# systemctl status suricata.service
● suricata.service - Suricata IDS/IDP daemon
   Loaded: loaded (/etc/systemd/system/suricata.service; disabled)
   Active: active (running) (Result: exit-code) since Thu 2016-09-29 16:01:01 CEST; 6 days ago

     Docs: man:suricata(8)
           man:suricatasc(8)
           https://redmine.openinfosecfoundation.org/projects/suricata/wiki

  Process: 19693 ExecReload=/usr/bin/suricatasc -c reload-rules (code=exited, status=1/FAILURE)
 Main PID: 2897 (Suricata-Main)
   CGroup: /system.slice/suricata.service
           └─2897 /usr/bin/suricata -D --af-packet -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid

Sep 29 16:01:01 ba-suricata-s.hq.eset.com suricata[2895]: 29/9/2016 -- 16:01:01 - <Notice> - This is Suricata version 3.1.2 RELEASE
Sep 29 16:01:01 ba-suricata-s.hq.eset.com suricata[2895]: 29/9/2016 -- 16:01:01 - <Info> - CPUs/cores online: 4
Sep 29 16:01:01 ba-suricata-s.hq.eset.com suricata[2895]: 29/9/2016 -- 16:01:01 - <Info> - HTTP memcap: 3221225472
Sep 29 16:01:01 ba-suricata-s.hq.eset.com suricata[2895]: 29/9/2016 -- 16:01:01 - <Info> - Found an MTU of 1500 for 'eth1'
Sep 29 16:01:01 ba-suricata-s.hq.eset.com suricata[2895]: 29/9/2016 -- 16:01:01 - <Info> - Found an MTU of 1500 for 'eth2'
Sep 29 16:01:01 ba-suricata-s.hq.eset.com suricata[2895]: 29/9/2016 -- 16:01:01 - <Info> - Found an MTU of 1500 for 'eth3'
Sep 29 16:01:01 ba-suricata-s.hq.eset.com systemd[1]: Started Suricata IDS/IDP daemon.
Oct 06 08:56:03 ba-suricata-s.hq.eset.com systemd[1]: Reloading Suricata IDS/IDP daemon.
Oct 06 08:56:03 ba-suricata-s.hq.eset.com suricatasc[19693]: Unable to connect to socket /var/run//suricata-command.socket: [Er...ctory
Oct 06 08:56:03 ba-suricata-s.hq.eset.com systemd[1]: suricata.service: control process exited, code=exited status=1
Oct 06 08:56:03 ba-suricata-s.hq.eset.com systemd[1]: Reload failed for Suricata IDS/IDP daemon.
Hint: Some lines were ellipsized, use -l to show in full.

And of course stop does not work either:
~# /etc/init.d/suricata stop
Stopping suricata (via systemctl): suricata.service.
~# systemctl status suricata.service
● suricata.service - Suricata IDS/IDP daemon
   Loaded: loaded (/etc/systemd/system/suricata.service; disabled)
   Active: failed (Result: signal) since Thu 2016-10-06 09:05:12 CEST; 10s ago

     Docs: man:suricata(8)
           man:suricatasc(8)
           https://redmine.openinfosecfoundation.org/projects/suricata/wiki

  Process: 21276 ExecStop=/usr/bin/suricatasc -c shutdown (code=exited, status=1/FAILURE)
  Process: 19693 ExecReload=/usr/bin/suricatasc -c reload-rules (code=exited, status=1/FAILURE)
 Main PID: 2897 (code=killed, signal=KILL)

Oct 06 08:56:03 ba-suricata-s.hq.eset.com suricatasc[19693]: Unable to connect to socket /var/run//suricata-command.socket: [Er...ctory
Oct 06 08:56:03 ba-suricata-s.hq.eset.com systemd[1]: suricata.service: control process exited, code=exited status=1
Oct 06 08:56:03 ba-suricata-s.hq.eset.com systemd[1]: Reload failed for Suricata IDS/IDP daemon.
Oct 06 09:03:42 ba-suricata-s.hq.eset.com systemd[1]: Stopping Suricata IDS/IDP daemon...
Oct 06 09:03:42 ba-suricata-s.hq.eset.com suricatasc[21276]: Unable to connect to socket /var/run//suricata-command.socket: [Er...ctory
Oct 06 09:03:42 ba-suricata-s.hq.eset.com systemd[1]: suricata.service: control process exited, code=exited status=1
Oct 06 09:05:12 ba-suricata-s.hq.eset.com systemd[1]: suricata.service stop-sigterm timed out. Killing.
Oct 06 09:05:12 ba-suricata-s.hq.eset.com systemd[1]: suricata.service: main process exited, code=killed, status=9/KILL
Oct 06 09:05:12 ba-suricata-s.hq.eset.com systemd[1]: Stopped Suricata IDS/IDP daemon.
Oct 06 09:05:12 ba-suricata-s.hq.eset.com systemd[1]: Unit suricata.service entered failed state.
Hint: Some lines were ellipsized, use -l to show in full.

With following service file the suricata (under suri user) service management is working:

~# cat /etc/systemd/system/suricata.service
[Unit]
Description=Suricata IDS/IDP daemon
After=network.target network-online.target
Requires=network-online.target
Documentation=man:suricata(8) man:suricatasc(8)
Documentation=https://redmine.openinfosecfoundation.org/projects/suricata/wiki

[Service]
Type=forking
Environment=LD_PREDLOAD=/usr/lib/libtcmalloc_minimal.so.4 UNIXCMD_SOCKET="/var/run/suricata/suricata-command.socket"
PIDFile=/var/run/suricata/suricata.pid
ExecStartPre=-/bin/mkdir /var/run/suricata
ExecStartPre=/bin/chown -R suri:suri /var/run/suricata
ExecStart=/usr/bin/suricata -D --af-packet -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata/suricata.pid
ExecReload=/bin/dash -c "/usr/bin/suricatasc -c reload-rules ${UNIXCMD_SOCKET}"; /bin/kill -HUP $MAINPID
ExecStop=/bin/dash -c "/usr/bin/suricatasc -c shutdown ${UNIXCMD_SOCKET}"
ExecStopPost=/bin/rm -rf /var/run/suricata
Restart=on-failure
ProtectSystem=full
ProtectHome=true

[Install]
WantedBy=multi-user.target

Just another minor remark - sysvinit script does not report reload action as available:
root@ba-suricata-s:~# /etc/init.d/suricata
OK
Usage: /etc/init.d/suricata {start|stop|restart|status}

--

Peter Viskup

[Arturo Borrero Gonzalez]

On 6 October 2016 at 10:05, Peter Viskup <skup...@gmail.com> wrote:
>>
>> I've not tested to run suricata with a different user apart of the
>> default, which is root.
>
>
> And that's the point - maybe described not in enough details by me.
>

So it seems there is no particular issue regarding the systemd integration.

The issue of running suricata with a different user than root is being discussed
in bug #836929 [0].

>
> Just another minor remark - sysvinit script does not report reload action as
> available:
> root@ba-suricata-s:~# /etc/init.d/suricata
> OK
> Usage: /etc/init.d/suricata {start|stop|restart|status}
>

Could you please open a separate bug for this?

I feel we have been discussing several mixed things in this bugreport.
Perhaps it makes sense closing this and moving to other dedicated bugs.
What do you think?

thanks, best regards

[0] https://bugs.debian.org/836929
--
Arturo Borrero González

[Peter Viskup]

I feel we have been discussing several mixed things in this bugreport.
Perhaps it makes sense closing this and moving to other dedicated bugs.
What do you think?

Agree.