[gentoo-user] syslog-ng segfaults

[Randy Barlow]

I saw that there were a couple of syslog-ng threads recently, but
neither of them mentioned any segfaulting which I am currently
experiencing. My log file (dmesg and /var/log/messages) is full of lines
like this:

syslog-ng[32015]: segfault at 44d8 ip 00007f4f3fa23c83 sp
00007fffb233b940 error 4 in libsyslog-ng.so.0.0.0 (deleted)[7f4f3f9eb000
+67000]

Has anybody else seen anything like that?

--
R

[Randy Barlow]

On Thu, 2013-07-18 at 18:41 -0400, Randy Barlow wrote:
> I saw that there were a couple of syslog-ng threads recently, but
> neither of them mentioned any segfaulting which I am currently
> experiencing. My log file (dmesg and /var/log/messages) is full of lines
> like this:
>
> syslog-ng[32015]: segfault at 44d8 ip 00007f4f3fa23c83 sp
> 00007fffb233b940 error 4 in libsyslog-ng.so.0.0.0 (deleted)[7f4f3f9eb000
> +67000]

I forgot to mention my version:

$ equery list syslog-ng
* Searching for syslog-ng ...
[IP-] [ ] app-admin/syslog-ng-3.4.2:0

--
R

[Randy Barlow]

On Thu, 2013-07-18 at 18:41 -0400, Randy Barlow wrote:

> Has anybody else seen anything like that?

I'll add in that this seems to be happening to all three of my x86_64
hosts. It does not appear to happen on my x86 host.

--
R

[Bruce Hill]

If it wasn't clear before:

router ~ # diff /etc/syslog-ng/syslog-ng.conf /etc/syslog-ng/syslog-ng.conf~
1c1
< @version: 3.4
---
> @version: 3.3

Cheers,
Bruce
--
Happy Penguin Computers >')
126 Fenco Drive ( \
Tupelo, MS 38801 ^^
sup...@happypenguincomputers.com
662-269-2706 662-205-6424
http://happypenguincomputers.com/

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

Don't top-post: http://en.wikipedia.org/wiki/Top_post#Top-posting

[Bruce Hill]

On Thu, Jul 18, 2013 at 06:41:14PM -0400, Randy Barlow wrote:

Try changing the version at the top of the config file to the present version.

[Randy Barlow]

On Thu, 2013-07-18 at 17:49 -0500, Bruce Hill wrote:
> Try changing the version at the top of the config file to the present version.

It's at 3.4, and I have the gentoo default config. Thanks for the
suggestion!

--
R

[Randy Barlow]

Randy Barlow wrote:
> I'll add in that this seems to be happening to all three of my x86_64
> hosts. It does not appear to happen on my x86 host.

I rebooted one of the hosts that was experiencing this issue, and it did
not return afterwards. This surprises me, as restarting the service did
not solve the problem. I don't really know what was the cause, but it
does appear to have gone away. I know that I have not rebooted since I
upgraded to that particular version, so perhaps there is something odd
that requires more than just a restart of syslog-ng.

Thanks to Bruce for help!

--
R

[Алексей Мишустин]

2013/7/19 Randy Barlow <ra...@electronsweatshop.com>:

Yes, I had the same issue last Sunday.

At x86_64 XEN-based VPS. It's gone after the reboot too. What is
CFLAGS there, I don't remember. I'll check it.

--
Regards,
Alex

[Adam Carter]

> syslog-ng[32015]: segfault at 44d8 ip 00007f4f3fa23c83 sp
> 00007fffb233b940 error 4 in libsyslog-ng.so.0.0.0 (deleted)[7f4f3f9eb000
> +67000]
>
> Has anybody else seen anything like that?

Perhaps you forgot to restart syslog-ng after the update?

# ldd /usr/sbin/syslog-ng | grep libsyslog-ng
    libsyslog-ng-3.4.2.so => /usr/lib64/libsyslog-ng-3.4.2.so (0x00007fe7dfe5b000)

If the library was called libsyslog-ng.so.0.0.0 in the previous version, and if syslog-ng tried to reload it, it would fail after the update which could cause the segfault.

So, restarting syslog-ng should be all that's required to fix it - reboot is overkill.

Also, any errors in the config file should not cause segfault. If it does, that indicates a bug in the input validation/config parsing code. Errors in the config should result in a graceful termination.

[Alexey Mishustin]

2013/7/19 Adam Carter <adamc...@gmail.com>:

>> > syslog-ng[32015]: segfault at 44d8 ip 00007f4f3fa23c83 sp
>> > 00007fffb233b940 error 4 in libsyslog-ng.so.0.0.0 (deleted)[7f4f3f9eb000
>> > +67000]
>> >
>> > Has anybody else seen anything like that?
>
>
> Perhaps you forgot to restart syslog-ng after the update?
>
> # ldd /usr/sbin/syslog-ng | grep libsyslog-ng
> libsyslog-ng-3.4.2.so => /usr/lib64/libsyslog-ng-3.4.2.so
> (0x00007fe7dfe5b000)
>
> If the library was called libsyslog-ng.so.0.0.0 in the previous version, and
> if syslog-ng tried to reload it, it would fail after the update which could
> cause the segfault.
>
> So, restarting syslog-ng should be all that's required to fix it - reboot is
> overkill.

As for me, first I updated syslog-ng, then I issued
'/etc/init.d/syslog-ng reload' (by mistake, instead of 'restart'), and
then 'restart' as I should. Then, just when syslog-ng was restarting,
the segfault happened.

> Also, any errors in the config file should not cause segfault. If it does,
> that indicates a bug in the input validation/config parsing code. Errors in
> the config should result in a graceful termination.

--
Regards,
Alex

[Randy Barlow]

Dan Johansson wrote:
> Question: Is this a "physical" host or is it a virtual host running
> under qemu?
> Reason for my question ist that recently the CPU-Id presented from quemu
> in the guest has changed and if you have CFLAGS="-march=native" then
> some newly compiles SW could fail. One way to solve this is to change
> the CFLAGS to "-mtune=native" and recompile the affected SW (or world).

Hi Dan!

Out of the three machines that were experiencing this issue, one was
physical and two were virtual. The one I rebooted that solved the
problem was virtual. I haven't yet rebooted the other two.

Thanks for the tip, I believe I am using -march=native, so perhaps I
should address that issue as well. Do you know which version of qemu
this change was introduced in?

--
R

[Alexey Mishustin]

2013/7/19 Dan Johansson <Dan.Jo...@dmj.nu>:

> Question: Is this a "physical" host or is it a virtual host running
> under qemu?
> Reason for my question ist that recently the CPU-Id presented from quemu
> in the guest has changed and if you have CFLAGS="-march=native" then
> some newly compiles SW could fail. One way to solve this is to change
> the CFLAGS to "-mtune=native" and recompile the affected SW (or world).

My virtual guest is compiled with "-mtune=generic". It's an universal
solution for all changes in virtual hosts, isn't it?

--
Regards,
Alex

[Randy Barlow]

Alexey Mishustin wrote:
>> So, restarting syslog-ng should be all that's required to fix it - reboot is
>> >overkill.

> As for me, first I updated syslog-ng, then I issued
> '/etc/init.d/syslog-ng reload' (by mistake, instead of 'restart'), and
> then 'restart' as I should. Then, just when syslog-ng was restarting,
> the segfault happened.

I also noted that restarting syslog didn't seem to solve the problem. I
do think Adam's reasoning makes sense, but there must be something else
that needed to be restarted as well.

--
R

[kwk...@hkbn.net]

Same behaviour here. In my case with an "lsof | grep libsyslog-ng" I
see in the physical host hp-systray from hplip was still
using the old libsyslog-ng.so, so killing that and a restart of
syslog-ng service stops the segfault lines. YMMV,

Kerwin.

[Neil Bothwick]

On Sat, 20 Jul 2013 20:27:51 +0800, kwk...@hkbn.net wrote:

> Same behaviour here. In my case with an "lsof | grep libsyslog-ng" I
> see in the physical host hp-systray from hplip was still
> using the old libsyslog-ng.so, so killing that and a restart of
> syslog-ng service stops the segfault lines. YMMV,

Try app-admin/checkrestart, I generally run this after updating any
daemons or libraries.


--
Neil Bothwick

"RAM DISK is NOT an installation procedure!"

[Randy Barlow]

Neil Bothwick wrote:
> Try app-admin/checkrestart, I generally run this after updating any
> daemons or libraries.

This sounds very helpful, thanks for the suggestion Neil!

--
R