Suricata filemagic ?s

[Brian Kellogg]

Hoping someone more experienced with Suricata can teach me how to enable filemagic matching. I don't want to necessarily extract every file.

Also I can't find all of the filemagic string options for matching against all of the different file types. Any pointers in the right direction very much appreciated.

thanks

[Peter Manev]

Hi,

A basic example usage would be -
alert http any any -> any any (msg:"FILEMAGIC pdf"; filemagic:"PDF document"; sid:123456789; rev:1;)

The standard libmagic/file utility is being used by Suricata to determine the magic ( file magic numbers - https://en.wikipedia.org/wiki/List_of_file_signatures)
Note: The version of that utility differs by OS/OS version itself.

Ex:
root@LTS-64-1:~ # file /usr/bin/suricata
/usr/bin/suricata: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=6d2465f965b8c188d51d5c27866269de0059683a, stripped
root@LTS-64-1:~ #

Usually the compiled database is located in /usr/share/file/magic.mgc
Also according to "man libmagic":
"/usr/share/misc/magic The non-compiled default magic database.
/usr/share/misc/magic.mgc The compiled default magic database."


Where from and in what order that magic.mgc is loaded on your system you can check with - strace file - and see the order of loading (example using the root user):
root@LTS-64-1:~ # strace file
...
...
stat("/root/.magic.mgc", 0x7ffee3aa7560) = -1 ENOENT (No such file or directory)
stat("/root/.magic", 0x7ffee3aa7560) = -1 ENOENT (No such file or directory)
open("/etc/magic.mgc", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/etc/magic", {st_mode=S_IFREG|0644, st_size=111, ...}) = 0
open("/etc/magic", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=111, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbce1a45000
read(3, "# Magic local data for file(1) c"..., 4096) = 111
read(3, "", 4096) = 0
close(3) = 0
munmap(0x7fbce1a45000, 4096) = 0
open("/usr/share/misc/magic.mgc", O_RDONLY) = 3
...
...

As to a complete list of the different file types/magic - I am not sure where to find a full "readable" list. If I need to create a rule based on the file magic info of a particular file type - I would first look it up with the file utility and then based on that info create the rule - ex:

root@LTS-64-1:~ # file /opt/NetworkMiner_1-6-1/NetworkMiner.exe
/opt/NetworkMiner_1-6-1/NetworkMiner.exe: PE32 executable (GUI) Intel 80386 Mono/.Net assembly, for MS Windows
root@LTS-64-1:~ #

alert http any any -> any any (msg:"FILEMAGIC exe - PE32 executable (GUI) Intel for MS Windows "; filemagic:"PE32 executable"; sid:123456789; rev:2;)

Thanks

[Brian Kellogg]

Thanks for the detailed explanation, really appreciate it.

I have some RAR files that don't trigger the rule, in fact all RAR files I've created haven't triggered the rule but they aren't identified as RAR by Bro either. I'll have to look at the file in a hex editor and see whats going on with it.

thanks again

[Seth Hall]

> On Oct 2, 2015, at 1:28 PM, Brian Kellogg <thef...@gmail.com> wrote:
>
> I have some RAR files that don't trigger the rule, in fact all RAR files I've created haven't triggered the rule but they aren't identified as RAR by Bro either. I'll have to look at the file in a hex editor and see whats going on with it.

Could you send one along to me? I suspect these might be RARs encrypted in a certain way which I’ve heard is a bit harder to detect (one of the other Bro core developers did some work with RARs in her past life).

.Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

[Brian Kellogg]

The problem with file detection ended up having to do with my VirtualBox test VMs. I ran the same simulations on a physical sensor and everything worked as it should; both Suri and Bro detected the files without issue.

VirtualBox VMs re-enable Tx offloading on reboot it seems. So disabling Tx offloading. Putting the below in your /etc/rc.local will fix this feature:

ethtool --offload eth? tx off

[Doug Burks]

sosetup should automatically configure /etc/network/interfaces to
disable tx offloading on your sniffing interface. Did you run sosetup
and allow it to configure your management and sniffing interfaces?
What does /etc/network/interfaces look like?


--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com

[Brian Kellogg]

Yep I did.  In fact I setup two VMs and tested this on both and saw the same behavior each time.  I do install the VB client utilities but I do that before running SO setup. 

Trying it again and not blowing through the setup prompts, setup states that since I have one interface it will be configured as the management interface.  Therein is the issue as I assumed that since there was only one NIC it would configure it for both MGMT and for monitoring.  Since there is only one NIC would it be advisable to also add the disabling of offloading to the lone NIC as well?

/etc/network/interfaces:

--
You received this message because you are subscribed to a topic in the Google Groups "security-onion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/security-onion/-m6cP6sAUFM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.
Visit this group at http://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

[Doug Burks]

On Sat, Oct 3, 2015 at 6:20 AM, Brian Kellogg <thef...@gmail.com> wrote:
>
> Yep I did.  In fact I setup two VMs and tested this on both and saw the same behavior each time.  I do install the VB client utilities but I do that before running SO setup.
>
> Trying it again and not blowing through the setup prompts, setup states that since I have one interface it will be configured as the management interface.  Therein is the issue as I assumed that since there was only one NIC it would configure it for both MGMT and for monitoring.  Since there is only one NIC would it be advisable to also add the disabling of offloading to the lone NIC as well?

No, instead I would recommend going into your VirtualBox configuration and adding a second NIC dedicated to sniffing:

https://github.com/Security-Onion-Solutions/security-onion/wiki/Hardware#nic