Ubuntu

[James Edwards]

Hi All,

I'm trying to compile OSSEC on Ubuntu 14.04 with libmagic support and I keep running into the following error when compiling syscheck (same error running Makeall as well):

[root@hostname]/tmp/ossec-hids-2.8.2/src/syscheckd# make
cc -g -Wall -I../ -I../headers    -DUSEINOTIFY    -DUSE_MAGIC  -DARGV0=\"ossec-syscheckd\" -DOSSECHIDS -lmagic  syscheck.c config.c seechanges.c run_realtime.c create_db.c run_check.c ../config/lib_config.a ../rootcheck/rootcheck_lib.a ../shared/lib_shared.a ../os_xml/os_xml.a ../os_regex/os_regex.a ../os_net/os_net.a ../os_crypto/os_crypto.a -o ossec-syscheckd
/tmp/cc9nExX5.o: In function `init_magic':
/tmp/ossec-hids-2.8.2/src/syscheckd/syscheck.c:43: undefined reference to `magic_open'
/tmp/ossec-hids-2.8.2/src/syscheckd/syscheck.c:47: undefined reference to `magic_error'
/tmp/ossec-hids-2.8.2/src/syscheckd/syscheck.c:50: undefined reference to `magic_load'
/tmp/ossec-hids-2.8.2/src/syscheckd/syscheck.c:52: undefined reference to `magic_error'
/tmp/ossec-hids-2.8.2/src/syscheckd/syscheck.c:54: undefined reference to `magic_close'
/tmp/ccLsn7RT.o: In function `is_text':
/tmp/ossec-hids-2.8.2/src/syscheckd/seechanges.c:24: undefined reference to `magic_buffer'
/tmp/ossec-hids-2.8.2/src/syscheckd/seechanges.c:28: undefined reference to `magic_error'
collect2: error: ld returned 1 exit status
make: *** [syscheck] Error 1

libmagic-dev 5.14-2ubuntu3.3 is installed and I see the following magic.h header files:

/usr/include/linux/magic.h
/usr/include/magic.h

Any advice on how to resolve this?

Thanks,
James

[theresa mic-snare]

Hi James,

I'm not the expert here, but I just had a quick look in the docs... I'm not sure if this is possible or even supported.
I couldn't find any reference to libmagic

Have you checked?
http://ossec-docs.readthedocs.org/en/latest/development/build/makefile.html

Out of curiosity, what whould OSSEC be capable of doing with ligmagic support other than recognizing file formats (which it usually does) ?!

best,
theresa

[Ryan Schulze]

I can verify the problem with Ubuntu 14.04.

According to the syscheck docs libmagic is optionally used with report_changes (if found on the system). I haven't checked the source code yet to see what exactly the ramifications are, but according to the docs:

http://ossec-docs.readthedocs.org/en/latest/manual/syscheck/
"Report Changes"
    If OSSEC has not been compiled with libmagic support, report_changes will copy any file designated, e.g. mp3,
    iso, executable, /chroot/dev/urandom (which would fill your hard drive). So unless libmagic is used, be very carefull
    on which directory you enable report_changes.

--

---
You received this message because you are subscribed to the Google Groups "ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ossec-list+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[James Edwards]

Due to the scope of the directories that we are monitoring, and a lot of NFS shares, space and performance are concerns with OSSEC.  By leveraging libmagic, it helps resolve the space issues.  In my case (on a working RHEL compilation) without libmagic, /var/ossec/queue was ~770MB, while with libmagic, it is ~62MB.

On another note, I've previously recompiled the software in our RHEL environment (6.6) using the same source tarball with libmagic, but Ubuntu 14.04 has been problematic for me.  Thanks for confirming, Ryan.

Thanks,
James

[theresa mic-snare]

Ok, sorry for the stupid questions.
But what does libmagic exactly do?
How does it enhance OSSEC?
How can I check if my OSSEC installation has libmagic support enable?
Is this only necessary for the Master or for the Agent as well? (Agents do syschecks too)

thanks,
theresa

[dan (ddp)]

On Jul 20, 2015 3:27 PM, "James Edwards" <x86bs...@gmail.com> wrote:
>
> Hi All,
>
> I'm trying to compile OSSEC on Ubuntu 14.04 with libmagic support and I keep running into the following error when compiling syscheck (same error running Makeall as well):
>
> [root@hostname]/tmp/ossec-hids-2.8.2/src/syscheckd# make
> cc -g -Wall -I../ -I../headers    -DUSEINOTIFY    -DUSE_MAGIC  -DARGV0=\"ossec-syscheckd\" -DOSSECHIDS -lmagic  syscheck.c config.c seechanges.c run_realtime.c create_db.c run_check.c ../config/lib_config.a ../rootcheck/rootcheck_lib.a ../shared/lib_shared.a ../os_xml/os_xml.a ../os_regex/os_regex.a ../os_net/os_net.a ../os_crypto/os_crypto.a -o ossec-syscheckd
> /tmp/cc9nExX5.o: In function `init_magic':
> /tmp/ossec-hids-2.8.2/src/syscheckd/syscheck.c:43: undefined reference to `magic_open'

Which file provides magic_open?

> /tmp/ossec-hids-2.8.2/src/syscheckd/syscheck.c:47: undefined reference to `magic_error'
> /tmp/ossec-hids-2.8.2/src/syscheckd/syscheck.c:50: undefined reference to `magic_load'
> /tmp/ossec-hids-2.8.2/src/syscheckd/syscheck.c:52: undefined reference to `magic_error'
> /tmp/ossec-hids-2.8.2/src/syscheckd/syscheck.c:54: undefined reference to `magic_close'
> /tmp/ccLsn7RT.o: In function `is_text':
> /tmp/ossec-hids-2.8.2/src/syscheckd/seechanges.c:24: undefined reference to `magic_buffer'
> /tmp/ossec-hids-2.8.2/src/syscheckd/seechanges.c:28: undefined reference to `magic_error'
> collect2: error: ld returned 1 exit status
> make: *** [syscheck] Error 1
>
> libmagic-dev 5.14-2ubuntu3.3 is installed and I see the following magic.h header files:
>
> /usr/include/linux/magic.h
> /usr/include/magic.h
>
> Any advice on how to resolve this?
>
> Thanks,
> James
>

[James Edwards]

libmagic support allows you to not have to copy files in order to monitor changes
libmagic is the magic number recognition library, which allows "File type determination library using "magic" numbers"
This enhances OSSEC by not copying files in directories you are monitoring (such as ISO's, mpegs, large archives, etc)
OSSEC does not come with this by default, it must be recompiled with support
I *think* this is required for both, but I'm not completely positive if the master/agent use the same syscheckd code.

[James Edwards]

I think this is a compiler issue... I checked my RHEL compilation and it used gcc-4.4.7, so I downgraded from gcc-4.8 on Ubuntu to gcc-4.4 and am able to get this to successfully compile with libmagic.

On another note, it is worth noting that I was able to successfully compile OSSEC from git using gcc-4.8 with libmagic support.

Thanks,
James

[theresa mic-snare]

James, thanks for the information on what libmagic does!!
and Congratulations for solving the compiler mystery! :)

Would you mind sharing the RHEL binaries with me? I would love to try the OSSEC enabled libmagic version on my CentOS test server.

out of curiosity: is there any disadvantage that comes with libmagic? why is it not enabled by default?

cheers,
theresa

[theresa mic-snare]

Hi James,

i'm now really interested in this...

Would you mind sharing the RHEL binaries with me? I would love to try the OSSEC enabled libmagic version on my CentOS test server.

out of curiosity: is there any disadvantage that comes with libmagic? why is it not enabled by default?

Am Mittwoch, 22. Juli 2015 17:00:02 UTC+2 schrieb James Edwards:

[Ryan Schulze]

I had a look at the Makeall file, and if the header (dev) files for magic are found, it is compiled with libmagic, if they aren't found it isn't. So by default it does try to compile with libmagic, but if it can't find the required files to do so, it falls back to not using libmagic.

[James Edwards]

@Theresa, here is a link to the RPM's I've built with libmagic. YMMV.  http://www.bsdftw.org/ossec/

@Ryan:

I can easily reproduce this problem:

I setup a new, "clean", RHEL environment - 6.6 x86_64, grabbed the srpm from http://www5.atomicorp.com/channels/source/ossec/ossec-hids-2.8.2-49.art.src.rpm and installed the following utilities: inotify-tools-devel.x86_64 file-devel.x86_64 httpd-devel glibc-devel openssl-devel sqlite-devel GeoIP-devel zlib-devel mysql-devel gcc kernel-headers.

kernel-headers provides /usr/include/linux/magic.h, and file-devel provides /usr/include/magic.h.  Both files are present:

[root@ossec-build SPECS]# ls -l /usr/include/linux/magic.h
-rw-r--r--. 1 root root 2292 Oct 15  2014 /usr/include/linux/magic.h
[root@ossec-build SPECS]# ls -l /usr/include/magic.h
-rw-r--r--. 1 root root 3890 Oct 15  2014 /usr/include/magic.h

Using the rpmbuild -ba ossec-hids-art.spec:
... build starts ...

 *** Making syscheckd ***

make[1]: Entering directory `/root/rpmbuild/BUILD/ossec-hids-2.8.2/src/syscheckd'
cc -g -Wall -I../ -I../headers   -DUSE_OPENSSL -DUSEINOTIFY     -DGEOIP -DARGV0=\"ossec-syscheckd\" -DOSSECHIDS -D_FORTIFY_SOURCE=2 -O2 -fexceptions -fstack-protector -fPIE -pie -Wl,-z,relro,-z,now   syscheck.c config.c seechanges.c run_realtime.c create_db.c run_check.c ../config/lib_config.a ../rootcheck/rootcheck_lib.a ../shared/lib_shared.a ../os_xml/os_xml.a ../os_regex/os_regex.a ../os_net/os_net.a ../os_crypto/os_crypto.a -o ossec-syscheckd
seechanges.c: In function 'seechanges_dupfile':
seechanges.c:150: warning: ignoring return value of 'fwrite', declared with attribute warn_unused_result
make[1]: Leaving directory `/root/rpmbuild/BUILD/ossec-hids-2.8.2/src/syscheckd'

... build completes ...

This is the ossec-syscheckd that is compiled:
[root@ossec-build syscheckd]# ldd ossec-syscheckd
        linux-vdso.so.1 =>  (0x00007fff1b5fd000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f5df2f17000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f5df3500000)

I modified Makeall to explicitly enable MAGIC by removing the if statements that check for those files, and rebuilt:
... starting build ...

 *** Making syscheckd ***

make[1]: Entering directory `/root/rpmbuild/BUILD/ossec-hids-2.8.2/src/syscheckd'
cc -g -Wall -I../ -I../headers   -DUSE_OPENSSL -DUSEINOTIFY    -DUSE_MAGIC -DGEOIP -DARGV0=\"ossec-syscheckd\" -DOSSECHIDS -D_FORTIFY_SOURCE=2 -O2 -fexceptions -fstack-protector -fPIE -pie -Wl,-z,relro,-z,now -lmagic  syscheck.c config.c seechanges.c run_realtime.c create_db.c run_check.c ../config/lib_config.a ../rootcheck/rootcheck_lib.a ../shared/lib_shared.a ../os_xml/os_xml.a ../os_regex/os_regex.a ../os_net/os_net.a ../os_crypto/os_crypto.a -o ossec-syscheckd
seechanges.c: In function 'seechanges_dupfile':
seechanges.c:150: warning: ignoring return value of 'fwrite', declared with attribute warn_unused_result
make[1]: Leaving directory `/root/rpmbuild/BUILD/ossec-hids-2.8.2/src/syscheckd'

... build completes ...

[root@ossec-build syscheckd]# ldd ossec-syscheckd
        linux-vdso.so.1 =>  (0x00007fff2dbff000)
        libmagic.so.1 => /usr/lib64/libmagic.so.1 (0x00007f655bf5b000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f655bbc7000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f655b9b0000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f655c3cf000)

I would think that modifying the .spec file with the following lines would be adequate to compile with libmagic support, but these get ignored:
echo "MEXTRA=-DUSE_MAGIC" >> ./Config.OS
echo "MAGICCMD=-lmagic" >> ./Config.OS

Those are put immediately after the 'echo "HEXTRA=-DMAX_AGENTS=16384" >> ./Config.OS' line, but to no avail using the rpmbuild command.

I ran into the same problem on Ubuntu when running Makeall, and had to explicitly enable it in the Makeall file.

Regards,
James