Hi,
After perform "sudo apt-get update && sudo apt-get upgrade", it shows blank front page and displays "1 node(s) with 0.0 logs indexed and 0.0 archived" when I logon ELSA.
I checked node.log via "sudo tail -f /nsm/elsa/data/elsa/log/node.log", it seems working normal except I can't use web UI.
* TRACE [2015/06/25 06:45:23] /opt/elsa/node/
elsa.pl (349) main::_process_batch 24120 [undef]
inserted filename /nsm/elsa/data/elsa/tmp/buffers//1435214663.08698 with batch_counter 1178 and start Thu Jun 25 06:44:23 2015 and end Thu Jun 25 06:45:23 2015
* DEBUG [2015/06/25 06:45:23] /opt/elsa/node/
elsa.pl (188) main:: 24120 [undef]
Processed 1178 records
* DEBUG [2015/06/25 06:45:23] /opt/elsa/node/
elsa.pl (184) main:: 24120 [undef]
Starting process_batch
* DEBUG [2015/06/25 06:45:23] /opt/elsa/node/
elsa.pl (271) main::_process_batch 24120 [undef]
Offline processing: and using tempfile /nsm/elsa/data/elsa/tmp/buffers//1435214723.09492
Attached sostat-redacted.log, any advise is welcome, thanks.
=========================================================================
Service Status
=========================================================================
Status: securityonion
* SO-user server[ FAIL ]
=========================================================================
Interface Status
=========================================================================
eth0 Link encap:Ethernet HWaddr MM:MM:MM:MM:MM:MM
inet addr:X.X.X.X Bcast:X.X.X.X Mask:X.X.X.X
inet6 addr: X.X.X.X/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:205880 errors:0 dropped:0 overruns:0 frame:0
TX packets:370258 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:43424096 (43.4 MB) TX bytes:85094005 (85.0 MB)
lo Link encap:Local Loopback
inet addr:X.X.X.X Mask:X.X.X.X
inet6 addr: X.X.X.X/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:40317 errors:0 dropped:0 overruns:0 frame:0
TX packets:40317 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:12961300 (12.9 MB) TX bytes:12961300 (12.9 MB)
=========================================================================
Link Statistics
=========================================================================
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback MM:MM:MM:MM:MM:MM brd MM:MM:MM:MM:MM:MM
RX: bytes packets errors dropped overrun mcast
12961300 40317 0 0 0 0
RX errors: length crc frame fifo missed
0 0 0 0 0
TX: bytes packets errors dropped carrier collsns
12961300 40317 0 0 0 0
TX errors: aborted fifo window heartbeat
0 0 0 0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether MM:MM:MM:MM:MM:MM brd MM:MM:MM:MM:MM:MM
RX: bytes packets errors dropped overrun mcast
43424096 205880 0 0 0 0
RX errors: length crc frame fifo missed
0 0 0 0 0
TX: bytes packets errors dropped carrier collsns
85094005 370258 0 0 0 0
TX errors: aborted fifo window heartbeat
0 0 0 0
=========================================================================
Disk Usage
=========================================================================
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 485G 60G 401G 13% /
udev 3.9G 4.0K 3.9G 1% /dev
tmpfs 799M 724K 798M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 3.9G 0 3.9G 0% /run/shm
=========================================================================
Network Sockets
=========================================================================
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
cupsd 1064 root 8u IPv6 27112 0t0 TCP [X.X.X.X]:631 (LISTEN)
cupsd 1064 root 9u IPv4 27113 0t0 TCP X.X.X.X:631 (LISTEN)
avahi-dae 1200 avahi 12u IPv4 8839 0t0 UDP *:5353
avahi-dae 1200 avahi 13u IPv6 8840 0t0 UDP *:5353
avahi-dae 1200 avahi 14u IPv4 8841 0t0 UDP *:33012
avahi-dae 1200 avahi 15u IPv6 8842 0t0 UDP *:43966
sshd 1373 root 3u IPv4 10427 0t0 TCP *:ssh_port (LISTEN)
sshd 1373 root 4u IPv6 10429 0t0 TCP *:ssh_port (LISTEN)
syslog-ng 1525 root 8u IPv4 10514 0t0 UDP X.X.X.X:35596->X.X.X.X:514
syslog-ng 1525 root 9u IPv4 10515 0t0 UDP X.X.X.X:59811->X.X.X.X:1514
syslog-ng 1525 root 11u IPv4 9019 0t0 TCP *:514 (LISTEN)
syslog-ng 1525 root 12u IPv4 9020 0t0 UDP *:514
mysqld 1602 mysql 10u IPv4 11716 0t0 TCP X.X.X.X:3306 (LISTEN)
salt-mini 1725 root 14u IPv4 13934 0t0 TCP X.X.X.X:34416->X.X.X.X:4505 (ESTABLISHED)
searchd 1726 sphinxsearch 7u IPv4 11586 0t0 TCP *:9306 (LISTEN)
searchd 1726 sphinxsearch 8u IPv4 11587 0t0 TCP *:9312 (LISTEN)
salt-mast 1728 root 19u IPv4 10923 0t0 TCP *:4506 (LISTEN)
ossec-csy 1730 ossecm 5u IPv4 10588 0t0 UDP X.X.X.X:44728->X.X.X.X:514
salt-mast 1834 root 27u IPv4 11712 0t0 TCP *:4505 (LISTEN)
salt-mast 1834 root 29u IPv4 12005 0t0 TCP X.X.X.X:4505->X.X.X.X:34416 (ESTABLISHED)
salt-mast 1840 root 19u IPv4 10923 0t0 TCP *:4506 (LISTEN)
salt-mast 1841 root 19u IPv4 10923 0t0 TCP *:4506 (LISTEN)
salt-mast 1844 root 19u IPv4 10923 0t0 TCP *:4506 (LISTEN)
salt-mast 1845 root 19u IPv4 10923 0t0 TCP *:4506 (LISTEN)
salt-mast 1848 root 19u IPv4 10923 0t0 TCP *:4506 (LISTEN)
master 2252 root 12u IPv4 13785 0t0 TCP X.X.X.X:25 (LISTEN)
master 2252 root 13u IPv6 13786 0t0 TCP [X.X.X.X]:25 (LISTEN)
snmpd 2324 snmp 8u IPv4 14557 0t0 UDP *:161
snmpd 2324 snmp 9u IPv4 14554 0t0 UDP *:54021
ntpd 2403 ntp 16u IPv4 11890 0t0 UDP *:123
ntpd 2403 ntp 17u IPv6 11891 0t0 UDP *:123
ntpd 2403 ntp 18u IPv4 11897 0t0 UDP X.X.X.X:123
ntpd 2403 ntp 19u IPv4 11898 0t0 UDP X.X.X.X:123
ntpd 2403 ntp 20u IPv6 11899 0t0 UDP [X.X.X.X]:123
ntpd 2403 ntp 21u IPv6 11900 0t0 UDP [X.X.X.X]:123
sshd 5380 root 3u IPv4 25452 0t0 TCP X.X.X.X:ssh_port->X.X.X.X:3425 (ESTABLISHED)
sshd 5530 SO-user 3u IPv4 25452 0t0 TCP X.X.X.X:ssh_port->X.X.X.X:3425 (ESTABLISHED)
/usr/sbin 16589 root 4u IPv4 211035 0t0 TCP *:443 (LISTEN)
/usr/sbin 16589 root 5u IPv4 211038 0t0 TCP *:9876 (LISTEN)
/usr/sbin 16589 root 6u IPv4 211040 0t0 TCP *:3154 (LISTEN)
/usr/sbin 19012 www-data 4u IPv4 211035 0t0 TCP *:443 (LISTEN)
/usr/sbin 19012 www-data 5u IPv4 211038 0t0 TCP *:9876 (LISTEN)
/usr/sbin 19012 www-data 6u IPv4 211040 0t0 TCP *:3154 (LISTEN)
/usr/sbin 19023 www-data 4u IPv4 211035 0t0 TCP *:443 (LISTEN)
/usr/sbin 19023 www-data 5u IPv4 211038 0t0 TCP *:9876 (LISTEN)
/usr/sbin 19023 www-data 6u IPv4 211040 0t0 TCP *:3154 (LISTEN)
/usr/sbin 19178 www-data 4u IPv4 211035 0t0 TCP *:443 (LISTEN)
/usr/sbin 19178 www-data 5u IPv4 211038 0t0 TCP *:9876 (LISTEN)
/usr/sbin 19178 www-data 6u IPv4 211040 0t0 TCP *:3154 (LISTEN)
/usr/sbin 19179 www-data 4u IPv4 211035 0t0 TCP *:443 (LISTEN)
/usr/sbin 19179 www-data 5u IPv4 211038 0t0 TCP *:9876 (LISTEN)
/usr/sbin 19179 www-data 6u IPv4 211040 0t0 TCP *:3154 (LISTEN)
/usr/sbin 19180 www-data 4u IPv4 211035 0t0 TCP *:443 (LISTEN)
/usr/sbin 19180 www-data 5u IPv4 211038 0t0 TCP *:9876 (LISTEN)
/usr/sbin 19180 www-data 6u IPv4 211040 0t0 TCP *:3154 (LISTEN)
/usr/sbin 19181 www-data 4u IPv4 211035 0t0 TCP *:443 (LISTEN)
/usr/sbin 19181 www-data 5u IPv4 211038 0t0 TCP *:9876 (LISTEN)
/usr/sbin 19181 www-data 6u IPv4 211040 0t0 TCP *:3154 (LISTEN)
/usr/sbin 19182 www-data 4u IPv4 211035 0t0 TCP *:443 (LISTEN)
/usr/sbin 19182 www-data 5u IPv4 211038 0t0 TCP *:9876 (LISTEN)
/usr/sbin 19182 www-data 6u IPv4 211040 0t0 TCP *:3154 (LISTEN)
/usr/sbin 19183 www-data 4u IPv4 211035 0t0 TCP *:443 (LISTEN)
/usr/sbin 19183 www-data 5u IPv4 211038 0t0 TCP *:9876 (LISTEN)
/usr/sbin 19183 www-data 6u IPv4 211040 0t0 TCP *:3154 (LISTEN)
/usr/sbin 19184 www-data 4u IPv4 211035 0t0 TCP *:443 (LISTEN)
/usr/sbin 19184 www-data 5u IPv4 211038 0t0 TCP *:9876 (LISTEN)
/usr/sbin 19184 www-data 6u IPv4 211040 0t0 TCP *:3154 (LISTEN)
/usr/sbin 19185 www-data 4u IPv4 211035 0t0 TCP *:443 (LISTEN)
/usr/sbin 19185 www-data 5u IPv4 211038 0t0 TCP *:9876 (LISTEN)
/usr/sbin 19185 www-data 6u IPv4 211040 0t0 TCP *:3154 (LISTEN)
=========================================================================
IDS Rules Update
=========================================================================
=========================================================================
CPU Usage
=========================================================================
Load average for the last 1, 5, and 15 minutes:
0.00 0.17 0.32
Processing units: 4
If load average is higher than processing units,
then tune until load average is lower than processing units.
top - 13:54:58 up 2:27, 1 user, load average: 0.00, 0.17, 0.32
Tasks: 155 total, 1 running, 154 sleeping, 0 stopped, 0 zombie
Cpu(s): 10.2%us, 0.8%sy, 0.1%ni, 88.1%id, 0.7%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 8178304k total, 7295096k used, 883208k free, 91944k buffers
Swap: 12475704k total, 204k used, 12475500k free, 5739664k cached
%CPU %MEM COMMAND
24.2 2.9 /usr/sbin/mysqld
0.6 8.8 /usr/bin/searchd --nodetach
0.3 0.5 perl /opt/elsa/node/
elsa.pl -c /etc/elsa_node.conf
0.3 0.1 /usr/sbin/syslog-ng -p /var/run/syslog-ng.pid
0.2 0.0 /var/ossec/bin/ossec-syscheckd
0.2 0.0 [flush-8:0]
0.1 0.1 /usr/sbin/lightdm-gtk-greeter
0.1 1.3 /usr/sbin/apache2 -k start
0.1 0.0 [jbd2/sda1-8]
0.0 0.5 /usr/bin/python /usr/bin/salt-minion
0.0 0.2 /usr/bin/X :0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch -background none
0.0 0.0 /var/ossec/bin/ossec-analysisd
0.0 0.0 [kworker/2:1]
0.0 0.0 [kswapd0]
0.0 0.4 /usr/bin/python /usr/bin/salt-master
0.0 0.4 /usr/bin/python /usr/bin/salt-master
0.0 0.0 [kworker/1:1]
0.0 0.4 /usr/bin/python /usr/bin/salt-master
0.0 0.4 /usr/bin/python /usr/bin/salt-master
0.0 0.0 /sbin/init
0.0 0.4 /usr/bin/python /usr/bin/salt-master
0.0 0.1 /usr/bin/python /usr/bin/fail2ban-server -b -s /var/run/fail2ban/fail2ban.sock
0.0 0.0 [kworker/0:0]
0.0 0.1 -bash
0.0 0.0 sshd: SO-user@pts/2
0.0 0.0 [kworker/3:1]
0.0 0.0 /usr/sbin/snmpd -Lsd -Lf /dev/null -u snmp -g snmp -I -smux -p /var/run/snmpd.pid
0.0 0.4 /usr/bin/python /usr/bin/salt-master
0.0 0.0 /usr/sbin/irqbalance
0.0 0.0 [kworker/3:0]
0.0 0.0 [kworker/u:1]
0.0 0.0 [kworker/2:2]
0.0 0.0 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 118:126
0.0 0.2 /usr/bin/python /usr/bin/salt-master
0.0 1.3 /usr/sbin/apache2 -k start
0.0 1.3 /usr/sbin/apache2 -k start
0.0 1.3 /usr/sbin/apache2 -k start
0.0 0.0 [migration/1]
0.0 0.0 [watchdog/2]
0.0 0.0 [ksoftirqd/0]
0.0 0.0 [migration/0]
0.0 0.0 /var/ossec/bin/ossec-logcollector
0.0 0.0 [kworker/u:0]
0.0 1.3 /usr/sbin/apache2 -k start
0.0 1.3 /usr/sbin/apache2 -k start
0.0 1.3 /usr/sbin/apache2 -k start
0.0 1.3 /usr/sbin/apache2 -k start
0.0 1.3 /usr/sbin/apache2 -k start
0.0 0.0 [migration/2]
0.0 0.0 [watchdog/0]
0.0 1.3 /usr/sbin/apache2 -k start
0.0 0.1 Passenger spawn server
0.0 0.0 [kworker/0:2]
0.0 0.0 [ksoftirqd/2]
0.0 0.0 [ksoftirqd/1]
0.0 0.0 [migration/3]
0.0 0.0 /usr/lib/gamin/gam_server
0.0 0.0 /usr/lib/accountsservice/accounts-daemon
0.0 0.0 [ksoftirqd/3]
0.0 0.0 /sbin/udevd --daemon
0.0 0.0 dbus-daemon --system --fork --activation=upstart
0.0 0.0 cron
0.0 0.0 /var/ossec/bin/ossec-csyslogd
0.0 0.0 sshd: SO-user [priv]
0.0 0.0 /usr/sbin/console-kit-daemon --no-daemon
0.0 0.0 PassengerHelperAgent
0.0 0.0 upstart-udev-bridge --daemon
0.0 0.0 /usr/lib/policykit-1/polkitd --no-debug
0.0 0.0 [watchdog/1]
0.0 0.0 [watchdog/3]
0.0 0.0 /usr/sbin/cupsd -F
0.0 0.0 lightdm
0.0 0.0 /usr/lib/upower/upowerd
0.0 0.0 [scsi_eh_0]
0.0 0.0 /usr/lib/postfix/master
0.0 0.0 [kworker/1:0]
0.0 0.0 [sync_supers]
0.0 0.0 [scsi_eh_1]
0.0 0.0 avahi-daemon: running [SO-server.local]
0.0 0.0 lightdm --session-child 16 19
0.0 0.0 su -s /bin/sh -c exec "$0" "$@" sphinxsearch -- /usr/bin/searchd --nodetach
0.0 0.2 /usr/bin/python /usr/bin/salt-master
0.0 0.0 [kthreadd]
0.0 0.0 [cpuset]
0.0 0.0 [khelper]
0.0 0.0 [kdevtmpfs]
0.0 0.0 [netns]
0.0 0.0 [bdi-default]
0.0 0.0 [kintegrityd]
0.0 0.0 [kblockd]
0.0 0.0 [ata_sff]
0.0 0.0 [khubd]
0.0 0.0 [md]
0.0 0.0 [khungtaskd]
0.0 0.0 [ksmd]
0.0 0.0 [khugepaged]
0.0 0.0 [fsnotify_mark]
0.0 0.0 [ecryptfs-kthrea]
0.0 0.0 [crypto]
0.0 0.0 [kthrotld]
0.0 0.0 [devfreq_wq]
0.0 0.0 [mpt_poll_0]
0.0 0.0 [mpt/0]
0.0 0.0 [scsi_eh_2]
0.0 0.0 [ttm_swap]
0.0 0.0 [ext4-dio-unwrit]
0.0 0.0 /sbin/udevd --daemon
0.0 0.0 /sbin/udevd --daemon
0.0 0.0 [kmpathd]
0.0 0.0 [kmpath_handlerd]
0.0 0.0 [kpsmoused]
0.0 0.0 /usr/sbin/bluetoothd
0.0 0.0 [krfcommd]
0.0 0.0 avahi-daemon: chroot helper
0.0 0.0 upstart-socket-bridge --daemon
0.0 0.0 /usr/sbin/sshd -D
0.0 0.0 /sbin/getty -8 38400 tty4
0.0 0.0 /sbin/getty -8 38400 tty5
0.0 0.0 su -c salt-minion
0.0 0.0 /sbin/getty -8 38400 tty2
0.0 0.0 /sbin/getty -8 38400 tty3
0.0 0.0 su -c salt-master
0.0 0.0 /sbin/getty -8 38400 tty6
0.0 0.0 supervising syslog-ng
0.0 0.0 acpid -c /etc/acpi/events -s /var/run/acpid.socket
0.0 0.0 atd
0.0 0.0 /var/ossec/bin/ossec-execd
0.0 0.0 /var/ossec/bin/ossec-monitord
0.0 0.2 /usr/bin/python /usr/bin/salt-master
0.0 0.0 /bin/sh /usr/lib/lightdm/lightdm-greeter-session /usr/sbin/lightdm-gtk-greeter
0.0 0.0 //bin/dbus-daemon --fork --print-pid 5 --print-address 7 --session
0.0 0.0 qmgr -l -t fifo -u
0.0 0.0 /usr/lib/gvfs/gvfsd
0.0 0.0 /usr/lib/gvfs//gvfs-fuse-daemon -f /var/lib/lightdm/.gvfs
0.0 0.0 lightdm --session-child 12 19
0.0 0.0 /sbin/getty -8 38400 tty1
0.0 0.0 /bin/sh -c perl /opt/elsa/node/
elsa.pl -c /etc/elsa_node.conf
0.0 0.0 pickup -l -t fifo -u -c
0.0 0.0 PassengerWatchdog
0.0 0.0 PassengerLoggingAgent
0.0 1.2 /usr/sbin/apache2 -k start
0.0 0.0 [kworker/0:1]
0.0 0.0 sudo sostat-redacted
0.0 0.0 /bin/bash /usr/bin/sostat-redacted
0.0 0.0 /bin/bash /usr/bin/sostat
0.0 0.0 sed -r s/(\b[0-9]{1,3}\.){3}[0-9]{1,3}\b/X.X.X.X/g
0.0 0.0 sed -r s/([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}/MM:MM:MM:MM:MM:MM/g
0.0 0.0 sed -r s/(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]).){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]).){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))\b/X.X.X.X/g
0.0 0.0 sed -r s/X:ssh_port/X:ssh_port/g
0.0 0.0 sed -r s/\*:ssh_port/*:ssh_port/g
0.0 0.0 sed -r s/SO-server/SO-server/g
0.0 0.0 sed -r s/SO-node/SO-node/g
0.0 0.0 sed -r s/SO-user|SO-user/SO-user/g
0.0 0.0 ps -eo pcpu,pmem,args --sort -pcpu
=========================================================================
Sguil Uncategorized Events
=========================================================================
=========================================================================
Sguil events summary for yesterday
=========================================================================
=========================================================================
Top 50 All time Sguil Events
=========================================================================
=========================================================================
Snorby Events Summary for yesterday
=========================================================================
=========================================================================
Top 50 All Time Snorby Events
=========================================================================
=========================================================================
ELSA
=========================================================================
Syslog-ng
Checking for process:
1524 supervising syslog-ng
1525 /usr/sbin/syslog-ng -p /var/run/syslog-ng.pid
Checking for connection:
Connection to localhost 514 port [tcp/shell] succeeded!
MySQL
Checking for process:
1602 /usr/sbin/mysqld
Checking for connection:
Connection to localhost 3306 port [tcp/mysql] succeeded!
Sphinx
Checking for process:
1578 su -s /bin/sh -c exec "$0" "$@" sphinxsearch -- /usr/bin/searchd --nodetach
Checking for connection:
Connection to localhost 9306 port [tcp/*] succeeded!
ELSA Buffers in Queue:
3
If this number is consistently higher than 20, please see:
https://github.com/Security-Onion-Solutions/security-onion/wiki/FAQ#why-does-sostat-show-a-high-number-of-elsa-buffers-in-queue
ELSA Directory Sizes:
51G /nsm/elsa/data
1.5G /var/lib/mysql/syslog
227M /var/lib/mysql/syslog_data
ELSA Index Date Range: