I know this is not supported by SO, but maybe you had a similar problem already and you could give me a hint about how to solve it.
I am willing to test a dynamic preprocessor called "pehunter", that should be able to extract PE files from the network as they pass by. The source code is here: http://src.carnivore.it/pehunter/tree/
Given that pehunter hasn't been deveoped in a while, I wanted to give it a try nevertheless, so I downloaded Snort's sources with the following command:
$sudo apt-get source securityonion-snort
Now, the strange thing is that, when I run autoreconf my system crashes because of what seems to be a memory leak:
root@myhost:~/securityonion-snort-2.9.4.6# autoreconf -fvi -I ./m4
autoreconf: Entering directory `.'
autoreconf: configure.in: not using Gettext
autoreconf: running: aclocal -I ./m4 --force
Killed
autom4te: /usr/bin/m4 failed with exit status: 137
aclocal: /usr/bin/autom4te failed with exit status: 137
autoreconf: aclocal failed with exit status: 137
root@myhost:~/securityonion-snort-2.9.4.6#
This also happens with a fresh source without even trying to install pehunter:
root@myhost:~# apt-get source securityonion-snort
Reading package lists... Done
Building dependency tree
Reading state information... Done
[...]
root@myhost:~# ls
securityonion-snort-2.9.4.6 securityonion-snort_2.9.4.6-0ubuntu0securityonion1.debian.tar.gz securityonion-snort_2.9.4.6-0ubuntu0securityonion1.dsc securityonion-snort_2.9.4.6.orig.tar.gz
root@myhost:~# cd securityonion-snort-2.9.4.6/
root@myhost:~/securityonion-snort-2.9.4.6# ls
aclocal.m4 config.guess config.sub configure.in COPYING depcomp etc LICENSE m4 Makefile.in mkinstalldirs RELEASE.NOTES snort.8 src tools ylwrap
ChangeLog config.h.in configure contrib debian doc install-sh ltmain.sh Makefile.am missing preproc_rules rpm snort.pc.in templates verstuff.pl
root@myhost:~/securityonion-snort-2.9.4.6# autoreconf -fvi -I ./m4
autoreconf: Entering directory `.'
autoreconf: configure.in: not using Gettext
autoreconf: running: aclocal -I ./m4 --force
Killed
autom4te: /usr/bin/m4 failed with exit status: 137
aclocal: /usr/bin/autom4te failed with exit status: 137
autoreconf: aclocal failed with exit status: 137
root@myhost:~/securityonion-snort-2.9.4.6#
I tried this on two different hosts, one running Ubuntu 12.04.2 from SO's iso and the other running Ubuntu 13.04. The latter has 8GB of RAM so this should not be a problem.
Any idea of what may be going on?
Thanks,
Pietro
Thanks for your recipe, I was looking at something similar yesterday ( http://liamrandall.com/bro-ids-2-1-file-extraction-how-to/ ) and I think I'll go with it. However this approach still has one problem: it only extracts files that are declared to be "application/*" in their HTTP Content-Type header. Unfortunately, a lot of malware is served with a different MIME type (e.g. "text/plain" or "text/html" or whatever), or even without any MIME type at all.
Do you think that it would be too computationally expensive to write a script that intercepts all files (including html, text, whatever) and feeds them to an equivalent of the Unix "file" command (or the python-magic module)? This would also allow me to filter based on other parameters: for example, I am not interested in storing files that are bigger than 10Mb, as they are less likely to be malware and analyzing them would be too costly).
I definitely need to learn Bro scripting, and I'm going to start right now! :-)
Thanks for your help, very appreciated!
Pietro
On Tuesday, July 23, 2013 4:05:44 PM UTC+2, Seth Hall wrote: