afl rookie, fuzzing openssl1.1.0f; path/to/fuzzed/app question

[Filmon Belay]

Hi all any help would be greatly appreciated.


I am conducting a software testing emprical study and attempting to fuzz openssl1.1.0f with afl.

the files I want to use as input are the ones that come  /testcases/ folder with AFL.

After extracting the openssl tar into a folder I ran the follwing 2 commands to instrument.



CC=afl-clang-fast ./config enable-fuzz-afl no-shared -DPEDANTIC   
enable-weak-ssl-ciphers enable-rc5 enable-md2    
enable-ssl3 enable-ssl3-method enable-nextprotoneg   
 enable-ec_nistp_64_gcc_128 -fno-sanitize=alignment     --debug

make

I believe that worked correctly
however I can't run the tool!
I'm attempting to do so by issuing

afl-fuzz -i /home/filmon/afl/testcases/ -o ~/afl/openssl/openssl-1.1.0f/out -- /can't/figure/out/what/goes/here/@@

I understand its the /path/to/fuzzed/app I'm not properly speficiying.
the current path to openssl I want to test is ~/afl/openssl/openssl-1.1.0f/ however when I use that I get the following

filmon@Momento-Mori:~/afl$ afl-fuzz -i /home/filmon/afl/testcases/ -o ~/afl/openssl/openssl-1.1.0f/out ~afl/testcases/@@
afl-fuzz 2.52b by <lca...@google.com>
[+] You have 8 CPU cores and 1 runnable tasks (utilization: 12%).
[+] Try parallel jobs - see /usr/local/share/doc/afl/parallel_fuzzing.txt.
[*] Checking CPU core loadout...
[+] Found a free CPU core, binding to #0.
[*] Checking core_pattern...
[*] Checking CPU scaling governor...
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Scanning '/home/filmon/afl/testcases/'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Validating target binary...

[-] PROGRAM ABORT : Program '~afl/testcases/@@' not found or not executable
         Location : check_binary(), afl-fuzz.c:6821




That is where all the files where extracted to. I used wget and tar to get these files. I am running Ubuntu 16.04

any help would be MUCH appreciated.

[Vegard Nossum]

On 5 April 2018 at 07:59, Filmon Belay <filbe...@gmail.com> wrote:
> I am conducting a software testing emprical study and attempting to fuzz
> openssl1.1.0f with afl.

[...]

> afl-fuzz -i /home/filmon/afl/testcases/ -o ~/afl/openssl/openssl-1.1.0f/out
> -- /can't/figure/out/what/goes/here/@@

It's the path to the binary you want to fuzz. The OpenSSL package on
my system has just a few binaries:

$ dpkg --listfiles openssl | grep bin
/usr/bin
/usr/bin/openssl
/usr/bin/c_rehash

> I understand its the /path/to/fuzzed/app I'm not properly speficiying.

Try something like

find -type f -executable

in your openssl-1.1.0f build or install directory to list all the
executable binaries.

The "openssl" binary itself is a swiss knife type tool so it can do a
lot of stuff depending on what arguments you pass it. Maybe something
like "openssl verify" or "openssl x509" if you want to fuzz
certificate files?


Vegard

[Filmon Belay]

 Thank you for your response! So I've listed out all the executable and I have more than just a few.

They all come from the cypto, test, fuzz folders and a couple form ms folders but I don't see the openssl binary its self.

I also don't have a usr/bin/openssl folder (there is a green openssl script though), does this mean I have compiled build and instrumented

the binary for openssl incorrectly? the pathe to that green openssl script is in~/afl/openssl/openssl-1.1.0f is that the path I want??

I do see the ./Confiure executable and a x509 executable..

I feel like I'm making such a stupid mistake but I just can't figure out what I'm doing wrong.

-Best
FIimon Belay

[Filmon Belay]

Hi so, I found a openssl executable in my usr/bin folder however when I try to run it wiht the following


it tells me that the binary isn't instredted. Here is the output that afl provides while instrumenting. I noticed that in the output it says 'l

afl-fuzz -i /home/filmon/afl/testcases/ -o ~/afl/openssl/openssl-1.1.0f/out  /usr/bin/openssl fuzz/@@

[-] Looks like the target binary is not instrumented! The fuzzer depends on
    compile-time instrumentation to isolate interesting test cases while
    mutating the input data. For more information, and for tips on how to
    instrument binaries, please see /usr/local/share/doc/afl/README.

    When source code is not available, you may be able to leverage QEMU
    mode support. Consult the README for tips on how to enable this.
    (It is also possible to use afl-fuzz as a traditional, "dumb" fuzzer.
    For that, you can use the -n option - but expect much worse results.)

[-] PROGRAM ABORT : No instrumentation detected
         Location : check_binary(), afl-fuzz.c:6920



here is my outtput showing its being instrumented: Again thank you for your time!


afl-clang-fast 1.96b by <lsze...@google.com>
afl-llvm-pass 1.96b by <lsze...@google.com>
[+] Instrumented 54 locations (non-hardened mode, ratio 100%).
rm -f test/x509aux
make -f ./Makefile.shared -e \
    PERL="/usr/bin/perl" SRCDIR=. \
    APPNAME=test/x509aux OBJECTS="test/x509aux.o" \
    LIBDEPS=' '" -L. -lcrypto"' -ldl ' \
    CC='afl-clang-fast' CFLAGS='-DDSO_DLFCN -DHAVE_DLFCN_H -DOPENSSL_THREADS -DOPENSSL_NO_DYNAMIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DPEDANTIC -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\"" -Wall -O0 -g -pthread -m64 -DL_ENDIAN -fno-omit-frame-pointer -g  -fno-sanitize=alignment ' \
    LDFLAGS='' \
    link_app.
make[2]: Entering directory '/home/filmon/afl/openssl/openssl-1.1.0f'
( :; LIBDEPS="${LIBDEPS:--L. -lcrypto -ldl }"; LDCMD="${LDCMD:-afl-clang-fast}"; LDFLAGS="${LDFLAGS:--DDSO_DLFCN -DHAVE_DLFCN_H -DOPENSSL_THREADS -DOPENSSL_NO_DYNAMIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DPEDANTIC -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\"" -Wall -O0 -g -pthread -m64 -DL_ENDIAN -fno-omit-frame-pointer -g  -fno-sanitize=alignment  }"; LIBPATH=`for x in $LIBDEPS; do echo $x; done | sed -e 's/^ *-L//;t' -e d | uniq`; LIBPATH=`echo $LIBPATH | sed -e 's/ /:/g'`; echo LD_LIBRARY_PATH=$LIBPATH:$LD_LIBRARY_PATH ${LDCMD} ${LDFLAGS} -o ${APPNAME:=test/x509aux} test/x509aux.o ${LIBDEPS}; LD_LIBRARY_PATH=$LIBPATH:$LD_LIBRARY_PATH ${LDCMD} ${LDFLAGS} -o ${APPNAME:=test/x509aux} test/x509aux.o ${LIBDEPS} )
LD_LIBRARY_PATH=.: afl-clang-fast -DDSO_DLFCN -DHAVE_DLFCN_H -DOPENSSL_THREADS -DOPENSSL_NO_DYNAMIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DPEDANTIC -DOPENSSLDIR="/usr/local/ssl" -DENGINESDIR="/usr/local/lib/engines-1.1" -Wall -O0 -g -pthread -m64 -DL_ENDIAN -fno-omit-frame-pointer -g -fno-sanitize=alignment -o test/x509aux test/x509aux.o -L. -lcrypto -ldl
afl-clang-fast 1.96b by <lsze...@google.com>
make[2]: Leaving directory '/home/filmon/afl/openssl/openssl-1.1.0f'
make[1]: Leaving directory '/home/filmon/afl/openssl/openssl-1.1.0f'

On Thursday, April 5, 2018 at 12:59:22 AM UTC-5, Filmon Belay wrote:

[Filmon Belay]

 ITS RUNNING!!!

 the command I issued was

 afl-fuzz -i /home/filmon/afl/testcases/ -o ~/afl/openssl/openssl-1.1.0f/ fuzz/x509

for some reason though trying to use fuzz/@@ causes the fuzzer to not recongnize or find any executables in that folder..

any ideas why??

Thanks so much.

-FB

On Thursday, April 5, 2018 at 12:59:22 AM UTC-5, Filmon Belay wrote:

[Jakub Wilk]

* Filmon Belay <filbe...@gmail.com>, 2018-04-05, 10:41:

>for some reason though trying to use fuzz/@@ causes the fuzzer to not
>recongnize or find any executables in that folder..

"fuzz/@@" is a strange name for an executable. Perhaps you misunderstood
what @@ does?

>>filmon@Momento-Mori:~/afl$ afl-fuzz -i /home/filmon/afl/testcases/ -o ~/afl/openssl/openssl-1.1.0f/out ~afl/testcases/@@

(Also note there's apparently missing slash between "~" and "afl" near
the end of this line.)

--
Jakub Wilk

[Filmon Belay]

>> "fuzz/@@" is a strange name for an executable. Perhaps you misunderstood
what @@ does?
 

  I think your right, I might be misunderstanding the read me. It says
 "

For programs that take input from a file, use '@@' to mark the location in
the target's command line where the input file name should be placed. The
fuzzer will substitute this for you:" 

So since I'm using the test files that come with afl to fuzz, I specified that 
thinking AFL would use each executable there to fuzz the program. Is that wrong?


If I also may ask one more question. Once I have AFL running how do I get it
to terminate without using ^c.. I didn't specifiy a run time length, I have me 
message saying it would default to 20ms.

Thanks SO much for your responses. (Missing backslash noted.)

Best
Filmon Belay

On Thursday, April 5, 2018 at 12:59:22 AM UTC-5, Filmon Belay wrote:

[Jakub Wilk]

* Filmon Belay <filbe...@gmail.com>, 2018-04-05, 11:55:

>For programs that take input from a file, use '@@' to mark the location
>in the target's command line where the input file name should be
>placed. The fuzzer will substitute this for you:"
>
>So since I'm using the test files that come with afl to fuzz, I
>specified that thinking AFL would use each executable there to fuzz the
>program.

No, afl-fuzz runs only one executable.

>If I also may ask one more question. Once I have AFL running how do I
>get it to terminate without using ^c..

^C is the normal way to stop AFL.

If you have specialised needs, you can send the SIGINT signal
programmatically using kill(1) or timeout(1) from coreutils.

(There are also AFL_BENCH_JUST_ONE and AFL_BENCH_UNTIL_CRASH environment
variables that make AFL stop early, but that's unlikely to be what you
want.)

>I didn't specifiy a run time length, I have me message saying it would
>default to 20ms.

That's timeout for single execution of the target executable.

--
Jakub Wilk

[Kurt Roeckx]

On Thu, Apr 05, 2018 at 11:55:35AM -0700, Filmon Belay wrote:
> >> "fuzz/@@" is a strange name for an executable. Perhaps you misunderstood
> what @@ does?
>
> I think your right, I might be misunderstanding the read me. It says
> "
>
> For programs that take input from a file, use '@@' to mark the location in
> the target's command line where the input file name should be placed. The
> fuzzer will substitute this for you:"

The fuzzers in the openssl source all take their input from stdin,
not from a file. That is those like fuzz/x509, fuzz/asn1, ... I
guess you can also try to run programs like fuzz/x509-test that do
take a filename as argument. But fuzz/x509 is really what you want
to run without specifying a file to read as argument.

Also note that the git repository contains a large corpus already,
those file do not end up in the released tarballs.


Kurt

[Filmon Belay]

   Thank you all for your inputs. I have been fuzzing happily away all weekend and feel comfortable using the tool and think I have a working understanding of how to use it.

   If I may ask one more question and also for some advice I would be very grateful.

    Question: Why are the files that come with openssl1.0.1/Fuzz  which appear to be input files for AFL are always 'to larg an input file for the fuzzer ( > 1 MB)

    Advice: Is there any testing that is considered to be good avenues for testing openssl wit AFL? I ran a long test ( > 16hrs) with x509 and found a few timeouts but nothing intersting. I've ran various
    other tests that  come included with afl but not any that came included with openssl (see question)

    MANY thanks to all!

    -Best,
    Filmon

   

On Thursday, April 5, 2018 at 12:59:22 AM UTC-5, Filmon Belay wrote:

[Jakub Wilk]

* Filmon Belay <filbe...@gmail.com>, 2018-04-11, 07:26:

>Why are the files that come with openssl1.0.1/Fuzz which appear to be
>input files for AFL are always 'to larg an input file for the fuzzer
>(> 1 MB)

Which files exactly do you have in mind?
I don't see any such large file in the OpenSSL git repo.

>I ran a long test ( > 16hrs) with x509 and found a few timeouts but
>nothing intersting.

16 hours is very little for any software of reasonable quality.

--
Jakub Wilk

[Filmon Belay]

  Hey Jakub!

   Sorry for the late response.

    >16 hours is very little for any software of reasonable quality.

      oh wow okay thats something I definitely didn't know.

  >Which files exactly do you have in mind?
   > I don't see any such large file in the OpenSSL git repo.
 

      I honestly have no idea what I was referring to either as I don't see any large files as well.
      It was back before I was really comfortable with everything so it could of been due to operator error that I thought that.
       

    Thanks for the information and taking the time to respond!

   Filmon

   

On Thursday, April 5, 2018 at 12:59:22 AM UTC-5, Filmon Belay wrote:

[Filmon Belay]

    Hi!

    Me again, I actually see what I was talking about with the files being to large. I currently can only fuzz using the files in the ~/afl/testcases/ directory.like so

         

               afl-fuzz -i /home/filmon/afl/testcases/ -o ~/afl/openssl/openssl-1.1.0f/out/crl/ fuzz/x509

    If I try to use the files in the ~/openssl/fuzz/ directory the all give me the below message. I have included the command I'm running to get that output before.
    
   

           afl-fuzz -i ~/afl/openssl/openssl-1.1.0f/fuzz -o ~/afl/openssl/openssl-1.1.0f/out/cms fuzz/cms

 

          [-] PROGRAM ABORT : Test case '/home/filmon/afl/openssl/openssl-1.1.0f/fuzz/asn1' is too big (7.13 MB, limit is 1.00 MB)
         Location : read_testcases(), afl-fuzz.c:1472

            

    Many thanks,
     Filmon Belay

On Thursday, April 5, 2018 at 12:59:22 AM UTC-5, Filmon Belay wrote:

[Jakub Wilk]

* Filmon Belay <filbe...@gmail.com>, 2018-04-16, 22:49:

>afl-fuzz -i /home/filmon/afl/testcases/ -o ~/afl/openssl/openssl-1.1.0f/out/crl/ fuzz/x509

Wait, what are you trying to achieve?

The only file in the AFL's "testcases" directory is a README. This is,
of course, not a very good input for an x509 parser.

(AFL doesn't look into subdirectories when reading input files. Even if
did, it wouldn't help, because it doesn't ship any valid x509 input.)

>afl-fuzz -i ~/afl/openssl/openssl-1.1.0f/fuzz -o ~/afl/openssl/openssl-1.1.0f/out/cms fuzz/cms

The "fuzz" directory in the OpenSSL tarball contains source code for
test programs. It doesn't contain any useful input for a CMS parser.

>'/home/filmon/afl/openssl/openssl-1.1.0f/fuzz/asn1' is too big (7.13 MB, limit is 1.00 MB)

And here AFL tries to use an ELF binary as an input file. You should be
glad that it didn't work. :)

--
Jakub Wilk

[Filmon Belay]

     Hi all,

      I just wanted to thank everyone that took the time to respond to this thread.
  
     I was doing a school project to get comfortable with a testing technique of my choice.
  
       I did an empirical comparison of AFL and libfuzzer.

     Anyway thank for taking the time to respond!!


      Best regards,
       FIlmon Belay

On Thursday, April 5, 2018 at 12:59:22 AM UTC-5, Filmon Belay wrote:

[Fayozbek Rustamov]

Hi Filmon Belay.

Did you test OpenSSl1.1.0f binaries (cms,asn1,ct,x509...) successfully by LibFuzzer?

суббота, 12 мая 2018 г., 8:47:02 UTC+9 пользователь Filmon Belay написал: