Git Product home page Git Product logo

honggfuzz's Introduction

Honggfuzz

Description

A security oriented, feedback-driven, evolutionary, easy-to-use fuzzer with interesting analysis options. See the Usage document for a primer on Honggfuzz use.

Code

Installation

sudo apt-get install binutils-dev libunwind-dev libblocksruntime-dev clang
make

Features

  • It's multi-process and multi-threaded: there's no need to run multiple copies of your fuzzer, as honggfuzz can unlock potential of all your available CPU cores with a single running instance. The file corpus is automatically shared and improved between all fuzzed processes.
  • It's blazingly fast when the persistent fuzzing mode is used. A simple/empty LLVMFuzzerTestOneInput function can be tested with up to 1mo iterations per second on a relatively modern CPU (e.g. i7-6700K).
  • Has a solid track record of uncovered security bugs: the only (to the date) vulnerability in OpenSSL with the critical score mark was discovered by honggfuzz. See the Trophies paragraph for the summary of findings to the date.
  • Uses low-level interfaces to monitor processes (e.g. ptrace under Linux and NetBSD). As opposed to other fuzzers, it will discover and report hijacked/ignored signals from crashes (intercepted and potentially hidden by a fuzzed program).
  • Easy-to-use, feed it a simple corpus directory (can even be empty for the feedback-driven fuzzing), and it will work its way up, expanding it by utilizing feedback-based coverage metrics.
  • Supports several (more than any other coverage-based feedback-driven fuzzer) hardware-based (CPU: branch/instruction counting, Intel BTS, Intel PT) and software-based feedback-driven fuzzing modes. Also, see the new qemu mode for blackbox binary fuzzing.
  • Works (at least) under GNU/Linux, FreeBSD, NetBSD, Mac OS X, Windows/CygWin and Android.
  • Supports the persistent fuzzing mode (long-lived process calling a fuzzed API repeatedly). More on that can be found here.
  • It comes with the examples directory, consisting of real world fuzz setups for widely-used software (e.g. Apache HTTPS, OpenSSL, libjpeg etc.).
  • Provides a corpus minimization mode.


Requirements

  • Linux - The BFD library (libbfd-dev) and libunwind (libunwind-dev/libunwind8-dev), clang-5.0 or higher for software-based coverage modes
  • FreeBSD - gmake, clang-5.0 or newer
  • NetBSD - gmake, clang, capstone, libBlocksRuntime
  • Android - Android SDK/NDK. Also see this detailed doc on how to build and run it
  • Windows - CygWin
  • Darwin/OS X - Xcode 10.8+
  • if Clang/LLVM is used to compile honggfuzz - link it with the BlocksRuntime Library (libblocksruntime-dev)

Trophies

Honggfuzz has been used to find a few interesting security problems in major software packages; An incomplete list:

Projects utilizing or inspired-by Honggfuzz

Contact

This is NOT an official Google product

honggfuzz's People

Contributors

aaronc100 avatar anestisb avatar atechroma0j avatar cnheitman avatar devnexen avatar dobin avatar douglasbagnall avatar dyjakan avatar gpoulios avatar hansmi avatar jmccl avatar jtpereyda avatar krytarowski avatar kshithijiyer avatar liumuqing avatar liyishuai avatar neuracr avatar ngg avatar paulgrandperrin avatar plusun avatar q3k avatar riusksk avatar robertswiecki avatar stefanbucur avatar thebabush avatar tl0gic avatar trofi avatar yaroslavyaroslav avatar zachriggle avatar zyv avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

honggfuzz's Issues

[new-feature] honggfuzz extensions

The main idea is to easily glue existing C code (file parser, verifiers, struct definitions, etc.) with fuzzing engine in order to apply smarter mutations that will respect attributes of fuzzing target. Such gluing can be achieved by exporting some callback functions for the main fuzzing actions: a) payload resize / prepare, b) mangling and c) post-mangling fix-ups.

Initially I was thinking a dynamic runtime approach to introduce such callbacks to be invoked from engine in a similar way that ASAN registers and consumes them. However, I abandoned that approach as it would result (IMHO) into heavy changes.

As such I've prefered a makefile / pre-processor static approach. Extensions can be coded under the /extensions folder following a naming convention (e.g. "target1_extension.c", "target2_Makefile"). Then extensions can be activated at compile time by enabling the matching flags (e.g. EXTENSION=target1).

Extension writers only need to follow the naming conventions for the filenames, paths, flags & callbacks. After that there is plenty of room to introduce any helper code (even entire parsing libs) or makefile fu in the imported files.

A quick prototype has been coded using the the Android DEX file format as a working case. Code is available at the following branch and can be activated by providing the "EXTENSION=dex" value when compiling:
https://github.com/anestisb/honggfuzz/tree/master_dev

Reference commits:
anestisb@9736046
anestisb@d888abc

IMHO such type of built-in extensions are significantly better compared to invoking external binaries to prepare files that are treated as a single input entity.

What are your thoughts on this?

build error in osx

The latest version of honggfuzz have the fllowing build error in osx, because "timer_t" is not defined in osx.

./common.h:317:9: error: unknown type name 'timer_t'; did you mean 'time_t'?
        timer_t timerId;
        ^~~~~~~
        time_t
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/sys/_types/_time_t.h:30:26: note: 
      'time_t' declared here
typedef __darwin_time_t         time_t; 
                                ^
1 error generated.
make: *** [honggfuzz.o] Error 1

You can add the following code in common.h to fix it:

typedef long timer_t;

error: 'readdir_r' is deprecated when compiling with glibc 2.24.90

Hello,
while trying to package this project for Fedora I encountered the following compilation error:

files.c: In function 'files_readdir':
files.c:161:9: error: 'readdir_r' is deprecated [-Werror=deprecated-declarations]
         if (readdir_r(dir, &de, &res) > 0) {
         ^~
In file included from /usr/include/features.h:397:0,
                 from /usr/include/limits.h:25,
                 from /usr/lib/gcc/x86_64-redhat-linux/6.1.1/include/limits.h:168,
                 from /usr/lib/gcc/x86_64-redhat-linux/6.1.1/include/syslimits.h:7,
                 from /usr/lib/gcc/x86_64-redhat-linux/6.1.1/include/limits.h:34,
                 from common.h:27,
                 from files.c:24:
/usr/include/dirent.h:189:12: note: declared here
 extern int __REDIRECT (readdir_r,

The glibc package version used during compilation was 2.24.90.

Won't build on OS X: El Capitan 10.11.6

When I run

make
Makefile:94: *** "macosx10.11" not found in /Library/Developer/CommandLineTools.  Stop.

Machine info:

uname -a
Darwin mac.local 15.6.0 Darwin Kernel Version 15.6.0: Mon Aug 29 20:21:34 PDT 2016; root:xnu-3248.60.11~1/RELEASE_X86_64 x86_

Let me know what other info I can provide.

how to detect windows gui app crash by honggfuzz?

honggfuzz not support to check windows gui app crash, beacause it only support to check command program crash with POSIX signal interface via CygWin .

WerFault.exe is launched when windows gui app crash, and windows GUI app process is still exist, so honggfuzz can't detect whether to crash.

do u have some solution to slove this issue?

Here is a few solutions what I think:
1、Loop check if the WerFault.exe process exist
2、close werfault. Exe warning if system allow, so that windows GUI app crash which can be catched by honggfuzz.

where is the file of fuzzedlib.c

when using compile-time instrumentation with clang as the FeedbackDrivenFuzzing.md in docs, it puzzles me a lot that how to execute the command of "clang-4.0 -fsanitize-coverage=trace-pc-guard,indirect-calls,trace-cmp fuzzedlib.c -o fuzzedlib.o".
Where is the file of fuzzedlib.c ? Strange!
Thank you!

Unable to compile due to binutils

I'm unable to compile honggfuzz on ArchLinux with the following errors:

cc -c -D_GNU_SOURCE -Wall -Werror -Wframe-larger-than=51200 -std=c11 -I. -I/usr/local/include -I/usr/include -Wextra -Wno-initializer-overrides -Wno-override-init -Wno-unknown-warning-option -funroll-loops -O2 -D_FILE_OFFSET_BITS=64 -msse4.2 -D_HF_ARCH_LINUX -o linux/bfd.o linux/bfd.c
In file included from linux/bfd.c:27:0:
/usr/include/bfd.h:35:2: error: #error config.h must be included before this header
 #error config.h must be included before this header
  ^
cc1: error: unrecognized command line option ‘-Wno-unknown-warning-option’ [-Werror]
cc1: error: unrecognized command line option ‘-Wno-initializer-overrides’ [-Werror]
cc1: all warnings being treated as errors
Makefile:161: recipe for target 'linux/bfd.o' failed
make: *** [linux/bfd.o] Error 1

It works properly on Ubuntu 14.4, while I'm using:

$ uname -a
Linux zenbook 4.4.1-2-ARCH #1 SMP PREEMPT Wed Feb 3 13:12:33 UTC 2016 x86_64 GNU/Linux
$ gcc --version
gcc (GCC) 5.3.0
$ pacman -Qi binutils
Name : binutils
Version : 2.25.1-3

As far as I can tell, it seems to be a bug in binutils?

Docker

I'd be nice to have this as a built docker image.

If I submit a PR for a Dockerfile, are you willing to turn on automated builds?

how to solve it that required the fuzzed file path must use quotation marks

I test in Windows, for example:

 rundll32.exe shimgvw.dll,ImageView_Fullscreen "C:\Users\Administrator\Desktop\test.jpg"

it can't open image if you don't use quotation marks, I try to fuzz with the following cmd, but no effect:

./honggfuzz.exe -f "C:\Users\Administrator\Desktop\test.jpg" -t3 -e jpg -- rundll32.exe shimgvw.dll,ImageView_Fullscreen \"___FILE___\"

how to solve it ?

honggfuzz does not build with gcc7

apparently gcc7 dropped the -Wno-initializer-overrides -Wno-unknown-warning-option options used in Makefile
and added some new warnings that become errors because of -Werror

Benchmarks of different feedback-driven fuzzing types?

It would be useful if someone with a Skylake CPU could post some example benchmarks of the different types of feedback-driven fuzzing.

  • --linux_perf_bts_block
  • --linux_perf_bts_edge
  • --linux_perf_ipt_block
  • --linux_perf_instr
  • --linux_perf_branch

I would do it myself, but I don't have any Skylake CPUs and all the cloud instances I've seen don't allow using PT or BTS.

Too many open files

What steps will reproduce the problem?
1.  Running a honggfuzz session on OS X 10.6
2.
3.

What is the expected output? What do you see instead?

[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-79818c280a08c94bf7fd2a54b1
e2567d.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-79818c280a08c94bf7fd2a54b1
e2567d.tif' in R/O mode: Too many open files
[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-93456679a773921d30efafd08f
3ad542.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-93456679a773921d30efafd08f
3ad542.tif' in R/O mode: Too many open files
[INFO] Launched new process, pid: 86649, (8/8)
[INFO] Launched new process, pid: 86650, (8/8)
[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-96292a1bd64fec83bb6cdd2480
a755b6.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-96292a1bd64fec83bb6cdd2480
a755b6.tif' in R/O mode: Too many open files
[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-76c43508fc007bcf5902b6a28e
8055a5.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-76c43508fc007bcf5902b6a28e
8055a5.tif' in R/O mode: Too many open files
[INFO] Launched new process, pid: 86651, (8/8)
[INFO] Launched new process, pid: 86652, (8/8)
[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-84da94dc7e5469f7849b0a7efd
ff5462.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-84da94dc7e5469f7849b0a7efd
ff5462.tif' in R/O mode: Too many open files
[WARNING] Couldn't open() 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-93456679a773921d30efafd08f
3ad542.tif' file in R/O mode: Too many open files
[ERROR] Couldn't open and map 
'/Users/admin/Fuzz/test_files/imagetestsuite/tif/7/m1-93456679a773921d30efafd08f
3ad542.tif' in R/O mode: Too many open files


What version of the product are you using? On what operating system?

OS X 10.6.7
honggfuzz, version 0.1 Robert Swiecki <[email protected]>, Copyright 2010 by 
Google Inc. All Rights Reserved.

Please provide any additional information below.

An lsof of the honggfuzz process is attached.


Original issue reported on code.google.com by [email protected] on 11 Apr 2011 at 9:35

Attachments:

empty SIG* files

current git version creates empty crash files:
 wc -c SIG*
0 SIGABRT.PC.7ffff6be6198.STACK.2878eff05.CODE.-6.ADDR.(nil).INSTR.cmp____$0xfffffffffffff000,%rax.fuzz
0 SIGSEGV.PC.7ffff7a60afd.STACK.1837a41547.CODE.1.ADDR.0x7fffff7feff8.INSTR.callq__0x0000000000009cab.fuzz
0 SIGSEGV.PC.7ffff7a6321d.STACK.d7f536722.CODE.1.ADDR.0x7fffff7feff8.INSTR.callq__0x000000000000758b.fuzz
0 SIGSEGV.PC.7ffff7aa9962.STACK.1836e5ace.CODE.1.ADDR.0x7fffff7feeb8.INSTR.mov____%rdi,0x28(%rsp).fuzz
0 SIGSEGV.PC.7ffff7ab7f0c.STACK.15f222c5c.CODE.1.ADDR.0x7fffff7fef38.INSTR.mov____%rdx,0x18(%rsp).fuzz
0 SIGSEGV.PC.7ffff7abb1e0.STACK.2bd7616a5.CODE.1.ADDR.0x7fffff7fef88.INSTR.mov____%rdx,(%rsp).fuzz
0 SIGSEGV.PC.7ffff7b978fa.STACK.d2647e.CODE.1.ADDR.0x7fffff7feff8.INSTR.callq__0xffffffffffeb5fc6.fuzz
0 SIGSEGV.PC.7ffff7b9b6bf.STACK.1594f9b8c9.CODE.1.ADDR.0x7fffff7fef38.INSTR.mov____%rdx,-0x2a8(%rbp).fuzz

honggfuzz being invoked like this:
honggfuzz -q -n 10 -f crap/ -c ./external.sh -- /testbinary ___FILE___
external.sh:
#!/bin/bash
filename=`perl -e 'print rand(10);' | cut -d "." -f 2` # $RANDOM collides sometimes
radamsa -r crap/ > $filename
mv $filename $1

0.6 doesn't create the empty files, nor does 0.8.

[Question] Run-time sync/reseed support?

Hi,

does honggfuzz come with some kind of sync/reseed feature? Let's say I've got some new samples obtained from an external sources (test case repo, produced by other fuzzer node, ...). Is there a way to get honggfuzz to pick these samples up during run-time, maybe by adding these files to the initial input dir or the working dir? In case that's possible, do we have to follow any file naming conventions?

Thanks
rc0r

Code coverage error

Hello,

I've tried to fuzz x64 target on ubuntu 14.04 lts with code coverage.
Target is properly configured and build with asan and code coverage.
When I run honggfuzz I get this error:

[2016-03-28T10:44:11+0200][E][11737] sancov_queryBitmap():131 bitmap overflow (67156064)
[2016-03-28T10:44:11+0200][E][11737] sancov_setBitmap():144 Out of range index (67156064 > 67108863)
Segmentation fault

about unique crash check bug

the function of unique crash check have a bug, for example, the fllowing crash files are same bug, beacause stack is dynamic addr(they are diff in each run) ,but both pc and instr are same.

SIGABRT.PC.7ffff695ecc9.STACK.187f3bf740.CODE.-6.ADDR.(nil).INSTR.cmp____$0xfffffffffffff000,%rax.pdf
SIGABRT.PC.7ffff695ecc9.STACK.18b82e134b.CODE.-6.ADDR.(nil).INSTR.cmp____$0xfffffffffffff000,%rax.pdf
SIGABRT.PC.7ffff695ecc9.STACK.29504b171.CODE.-6.ADDR.(nil).INSTR.cmp____$0xfffffffffffff000,%rax.pdf

The dynamic file corpus is empty bug with -C option in Mac

I compiled libpng with -fsanitize=address -fsanitize-coverage=bb, then run the following cmd in mac osx :
honggfuzz -C -f /samples/png -e png -- ./pngtest ___FILE___

but it have a error"The dynamic file corpus is empty".

Do I using the wrong cmd or it need hardware supported?

----------------------------[ honggfuzz v0.9alpha ]---------------------------
Iterations : 100
Phase : Dynamic Pre
Run Time : 0 hrs 0 min 10 sec (since: 2016-11-04 17:41:56)
Input Dir : '/samples/png'
Fuzzed Cmd : './pngtest FILE'
Threads : 2, CPUs: 4, CPU: nan% (nan%/CPU)
Speed : 1/sec (avg: 10)
Crashes : 0 (unique: 0, blacklist: 0, verified: 0)
Timeouts : 0 [10 sec.]
Corpus Size : 0, max size (bytes): 613321
Coverage :
*** total hit #bb: 0 (coverage 0.00%)
*** total #dso: 0 (instrumented only)
*** discovered #bb: 0 (new from input seed)
*** crashes: 0
-----------------------------------[ LOGS ]-----------------------------------

[2016-11-04T17:42:06+0800][F][1034] fuzz_prepareFileDynamically():76 The dynamic file corpus is empty. Apparently, the initial fuzzing of the provided file corpus (-f) has not produced any follow-up files with positive coverage and/or CPU counters

ls /samples/png/
008b8bb75b8a487dc5aac86c9abb06fb.png      Waldau-plan.png

Feature request: add a flag to save inputs that caused timeouts

I'm interested not only in the crashes but timeouts as well (to prevent DoS-like attacks).
It would be great to have a command line option to save those inputs as well.

I could create a small PoC but it's not too elegant:

If I call arch_ptraceSaveData from the WIFSIGNALED(event_msg) case in arch_ptraceEvent (linux/ptrace_utils.c), it's kind of working.

But it would count timeouts as crashes as well and the filename would start with SIGTRAP instead of SIGKILL (or TIMEOUT would be even better).
Also it seems that it would report slightly different PCs each time and this mean lots of output files would be generated.
I tested it only on Linux so far.

So I think this would need more work to make it really usable, that's why I didn't create a PR so far.

Error occured when compile libunwind

When i try to compile libunwind by third_party/android/scripts/compile-libunwind.sh third_party/android/libunwind arm64, then I get a error “/mnt/disk/android-ndk-r13b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/../lib/gcc/aarch64-linux-android/4.9.x/../../../../aarch64-linux-android/bin/ld: cannot find -lgcc_s”. But I have found libgcc_s.so in /usr/lib/gcc path, and it is a valid soft_link to /lib/x86_64-linux-gnu/libgcc_s.so.1.
i am so confused.

Code review request

Purpose of code changes on this branch:

Include originally fuzzed filename in output format. Helps to identify original 
and compare with crash.

After the review, I'll merge this branch into:
/trunk


Original issue reported on code.google.com by [email protected] on 17 Jun 2013 at 4:19

Attachments:

Death by infinite loop

Using honggfuzz (nice tool btw), I have faced an interesting issue.

I was fuzzing a dynamic library on Android (closed source), using persistent mode and basic hardware code coverage.
Several code paths were full of infinite loops, triggering tons of hangs.
And basically, honggfuzz was dying in a few seconds (with performances dropping from very good to very very bad).

After investigating, I've found that honggfuzz was killing the child due to a timeout, as expected, but feedback routines were still called.

Problem is, during an infinite loop, performance counter values are high.
So faulty test case was found interesting by fuzz_perfFeedback and kept, literally poisoning the fuzzer by maximizing the hangs.

I've written a quick and dirty patch, and performances came from an average of 10 tests per seconds to more than 1000.

ignore_hangs_patch.txt

[new-feature] stackhash blacklist

The idea is to skip during fuzzing already analyzed crashes without having to transfer entire crash files between workspaces (extra painful if remote device, e.g. Android) in order to hit the naming duplicates.

Additionally, same vulnerable targets might be loaded (or crashed) in different address resulting into noise duplicates (due to PC or ADDR) that can be avoided with stack hash blacklists.

Feature is already implemented (and used the past few day against Android) here: anestisb@d59af69

I think that stack hashing is good enough to practically prevent collisions. However, I'd like to hear your thoughts before PR.

defer blocks mis-behavior in dynamic semantics

While testing the recent commits with defer blocks against Android (nice work btw) noticed a few invalid frees (working with debug versions of jemalloc at my fuzzing env) that got me suspicious to dig further your implementation.

Based on your commit logs my understanding is that you introduced a go-style implementation of defer. As far as I'm aware of in golang defer is function-scoped, while the honggfuzz implementation appears to be block-based. As such calling defer() inside dynamic semantics (e.g. for loops) does not appear to always have identical behavior with replaced logic. Have a look at the following snippet for example:

368 bool files_parseBlacklist(honggfuzz_t * hfuzz)
369 {
370     FILE *fBl = fopen(hfuzz->blacklistFile, "rb");
371     if (fBl == NULL) {
372         PLOG_E("Couldn't open '%s' - R/O mode", hfuzz->blacklistFile);
373         return false;
374     }
375     DEFER(fclose(fBl));
376
377     char *lineptr = NULL;
378     size_t n = 0;
379     for (;;) {
380         if (getline(&lineptr, &n, fBl) == -1) {
381             break;
382         }
383         DEFER(free(lineptr));
384
385         if ((hfuzz->blacklist =
386              realloc(hfuzz->blacklist,
387                      (hfuzz->blacklistCnt + 1) * sizeof(hfuzz->blacklist[0]))) == NULL) {
388             PLOG_E("realloc failed (sz=%zu)",
389                    (hfuzz->blacklistCnt + 1) * sizeof(hfuzz->blacklist[0]));
390             return false;
391         }
392
393         hfuzz->blacklist[hfuzz->blacklistCnt] = strtoull(lineptr, 0, 16);
394         LOG_D("Blacklist: loaded %'" PRIu64 "'", hfuzz->blacklist[hfuzz->blacklistCnt]);
395
396         // Verify entries are sorted so we can use interpolation search
397         if (hfuzz->blacklistCnt > 1) {
398             if (hfuzz->blacklist[hfuzz->blacklistCnt - 1] > hfuzz->blacklist[hfuzz->blacklistCnt]) {
399                 LOG_F
400                     ("Blacklist file not sorted. Use 'tools/createStackBlacklist.sh' to sort records");
401                 return false;
402             }
403         }
404         hfuzz->blacklistCnt += 1;
405     }
406
407     if (hfuzz->blacklistCnt > 0) {
408         LOG_I("Loaded %zu stack hash(es) from the blacklist file", hfuzz->blacklistCnt);
409     } else {
410         LOG_F("Empty stack hashes blacklist file '%s'", hfuzz->blacklistFile);
411     }
412     return true;
413 }

line 383 DEFER() will be executed at the end of each iteration of the for loop which is not the desired behavior for lineptr as utilized from getdelim. Based on previous code lineptr is set to null outside the loop so that getdelim will malloc on 1st exec and then realloc (if required) for each following iteration. As such, lineptr needs to be free'd once before function exit (error or completed loop). DEFER() call inside the loop will result into an invalid pointer since free will be invoked after 1st loop iteration.

Hooking free (and other funcs invoked from defer block) and injecting a log message might be a good starting point to detect such cases faster than reading the entire code-base. For example a sample log message with hooked free for the previous bug is looking like (the last error message is from jemalloc system allocator):

[1970-05-30T11:04:31+0000][I][5049] files_readdir():156 372 input files have been added to the list
[1970-05-30T11:04:31+0000][D][5049] files_parseBlacklist():399 Blacklist: loaded 520788916'
[1970-05-30T11:04:31+0000][W][5049] hf_free():369 free() invoked
[1970-05-30T11:04:31+0000][D][5049] files_parseBlacklist():399 Blacklist: loaded 528159365'
[1970-05-30T11:04:31+0000][W][5049] hf_free():369 free() invoked
[1970-05-30T11:04:31+0000][D][5049] files_parseBlacklist():399 Blacklist: loaded 1411144527'
[1970-05-30T11:04:31+0000][W][5049] hf_free():369 free() invoked
[1970-05-30T11:04:31+0000][D][5049] files_parseBlacklist():399 Blacklist: loaded 1414514899'
[1970-05-30T11:04:31+0000][W][5049] hf_free():369 free() invoked
[1970-05-30T11:04:31+0000][D][5049] files_parseBlacklist():399 Blacklist: loaded 1446874223'
[1970-05-30T11:04:31+0000][W][5049] hf_free():369 free() invoked
[1970-05-30T11:04:31+0000][D][5049] files_parseBlacklist():399 Blacklist: loaded 1738392012'
[1970-05-30T11:04:31+0000][W][5049] hf_free():369 free() invoked
[1970-05-30T11:04:31+0000][D][5049] files_parseBlacklist():399 Blacklist: loaded 1742973494'
[1970-05-30T11:04:31+0000][W][5049] hf_free():369 free() invoked
Invalid address 0xf713f000 passed to free: value not allocated
Aborted

For the previous problematic case the defer statement can be moved outside the for loop as following:

diff --git a/files.c b/files.c
index 16c9143..9aa6501 100644
--- a/files.c
+++ b/files.c
@@ -374,13 +374,12 @@ bool files_parseBlacklist(honggfuzz_t * hfuzz)
     }
     DEFER(fclose(fBl));

-    char *lineptr = NULL;
+    char *lineptr = NULL; DEFER(free(lineptr));
     size_t n = 0;
     for (;;) {
         if (getline(&lineptr, &n, fBl) == -1) {
             break;
         }
-        DEFER(free(lineptr));

         if ((hfuzz->blacklist =
              realloc(hfuzz->blacklist,

Also for the sake of technical completeness just to let you know that I'm testing the latest upstream against Android M 6.0.1 compiled with gcc 4.9 from NDK r11b (clang for android is not an option yet since libBlocksRuntime is not supported from AOSP).

Unfortunately, I'm struggling to find extra time these days for more testing but I'll try to PR or report similar bugs as soon as I verify them.

Fuzzing multiple targets at once

While fuzzing network sniffers, I'd typically use something similair:

honggfuzz -c ../external.sh -p 1234 -f input/ -- sudo tcpreplay -M topspeed -i lo ___FILE___

Where pid 1234 belongs to a sniffer such as tshark/wireshark/tcpdump etc listening on the loopback interface.

With this setup I can easily start and configure multiple sniffers to listen on the loopback interface, and therefore also fuzz them with the same data generated by honggfuzz, however honggfuzz will in this setup have no knowledge of the other sniffers and therefore never save the input data once/if they crash.

What I want is the ability to honggfuzz to monitor multiple targets at once.

Please ask if I havn't made myself clear :)

Undefined symbols for architecture x86_64: "_LLVMFuzzerInitialize" in MacOS

it have a bug when compiled libpng with Compile-time instrumentation in MacOS.

libpng_read_fuzzer.cc source code:
https://github.com/google/oss-fuzz/blob/master/targets/libpng/libpng_read_fuzzer.cc

riusksk@MacBook  ~/Downloads/libpng-master ‹›$ ./configure CFLAGS='-fsanitize-coverage=trace-pc-guard,indriect-calls,trace-cmp'
riusksk@MacBook  ~/Downloads/libpng-master ‹›$ make
riusksk@MacBook  ~/Downloads/libpng-master ‹›$ clang++ -std=c++11  libpng_read_fuzzer.cc ../honggfuzz/libhfuzz/libhfuzz.a ./.libs/libpng16.dylib -lz -o png_read_fuzzer
Undefined symbols for architecture x86_64:
  "_LLVMFuzzerInitialize", referenced from:
      _main in libhfuzz.a(persistent.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Segmentation Fault when using external mutators + hardware feedback

Having the following external mutator:

root@IPHONE9:/home/javier/Fuzzing/Sessions/hongg_resolv_conf# cat ../mutatorx.sh 
#!/bin/sh
cat /some/file.txt | radamsa > $1

And running the following command, triggers a segmentation fault in my Debian 4.9.0-0.bpo.2-rt-amd64:

/home/javier/Fuzzing/Fuzzers/honggfuzz/honggfuzz -c /home/javier/Fuzzing/Sessions/mutatorx.sh --linux_perf_instr -n 1 -- ../../Victims/TargetFuzzingBinaries/afl_fuzz_resolv-conf ___FILE___

Removing --linux_perf_instr causes other weird stuff and crashes in honggfuzz.

[LINUX remote pid] Both ends monitoring

The scenario is that you have a target system daemon-server that you want to fuzz test via a client cmd tool. Normally you would expect that monitoring/tracing only the daemon/server side would be enough. Unfortunately that is not the case for quite a few components in Android.

A common case is that the metadata (and a light verification) of the input data are processed locally from the client process (aka the pid group spawned from the fuzzer) and the heavy processing is performed from the daemon/server process (aka the remote long-lived pid group fuzzer attaches to). Since the client-side part will be incorporated also in privileged processes (which is not easy to fuzz at the first time) at a production setup, bugs at both sides are interesting for these scenarios.

Currently honggfuzz is configured to monitor either the spawned fuzzer-side or the remote pid. Can't do both. It would be really useful if such a feature can be implemented as part of the Linux arch_reapChild functionality.

I've made an attempt (anestisb@e745114) a few weeks ago which seemed to work fine. Didn't bother to PR though since it was kind of ugly with lot's of duplicate code.

What are your thoughts for a more elegant implementation? Maybe this e2e pid monitoring can be splitted into two levels:

  1. The attach/ptrace level: If chosen by the user ptrace attach tid groups on both sides
  2. The feedback level: Which end will provide feedback data for the dynamic data prepare logic (both, fuzzer or remote pid)

The first is quite easy, although the second part might prove a bit tricky since it will require to offload the pid decisions from sancov & perf components and explicitly define it as part of the calling args.

At the same time to fuzzing multiple format files

Suggest to add a function that generated fuzzed files to retain the original file suffix, so that can fuzzing multiple file format at the same time.
Some app must only parse the specific file suffix name.,but the default generated file suffix name is *.fuzz, and -e arg can only specify a file suffix, so the fuzzing would be invalid when input dir have multiple file suffix name.

Release 1.0

Hi all!

According to the quite random release numbering - from https://github.com/google/honggfuzz/releases - the next version will be 1.0 :). Hoorraaay!

I'd like to release it somewhere next week, and by that time to confirm that it compiles and works under various platforms w/o bigger problems. If you'd like to help me, I'll be glad for testing the master branch and letting me know if it works (or doesn't under any reasonable platform you can test it with).

I'd like to have it tested, at a minimum, with the following configs:


Linux (various distros):

  • i386:
    • Unknown distro: Compiles, Works
  • x86-64:
    • GCC: Ubuntu 17.04: Compiles, Works (sancov, bts/ipt, finstrument-functions, persistent mode):
    • Clang 4.0/5.0 Ubuntu 17.04: Compiles, Works
    • GCC: Fedora 25: Compiles, Works
  • arm: Ubuntu 16.04
  • arm64: Ubuntu 16.04
  • ppc32:
  • ppc64: Ubuntu 16.04
  • mips:
  • mips64:

Android:

  • armeabi:
  • armeabi-v7a:
  • x86
  • x86_64
  • arm64-v8a

CygWin:

  • x86-64
    • [-] Clang - Will not compile, one of the system headers is using __blocks as a argument's name
    • GCC - Compiles, Works with basic cases (no instrumentation)

FreeBSD:

  • x86-64
    • Clang
    • GCC

MacOS:

  • Mac OS Sierra 10.12.5
  • macOS El Capitan 10.11.6

Mentioning a couple of people who worked on the code in the past in case they would like to help me with quick testing :)

@anestisb @tl0gic @felixgr @riusksk @Manouchehri @dyjakan

Honggfuzz exits unexpectedly

Having the following "mutator"

root@IPHONE9:/home/javier/Fuzzing/Sessions/hongg_resolv_conf# cat ../wot.sh 
#!/bin/sh
echo "WOT" > $1

And choosing -s switch for stdin, the following command exits unexpectedly after a while:

/home/javier/Fuzzing/Fuzzers/honggfuzz/honggfuzz -f . -c /home/javier/Fuzzing/Sessions/wot.sh -s -n 1 -- tee /tmp/test

With the following error:

[2017-04-05T20:08:09+0100][E][21756] subproc_System():315 Command '/home/javier/Fuzzing/Sessions/wot.sh' terminated with signal: 11
[2017-04-05T20:08:09+0100][E][21756] fuzz_prepareFileExternally():146 Subprocess '/home/javier/Fuzzing/Sessions/wot.sh' returned abnormally
[2017-04-05T20:08:09+0100][F][21756] fuzz_fuzzLoop():532 fuzz_prepareFileExternally() failed

Also it doesn't generate any /tmp/test file. It does but still fails after a while.

fuzz bug about open fuzzed file with app

some time use honggfuzz to fuzz, it can't open the file with fuzzed app,for example(Preview in Mac OSX):

honggfuzz -f /Volumes/Macintosh/Samples/pdf/ -n 3 -- /Applications/Preview.app/Contents/MacOS/Preview _FILE_

it only open the Preview, but it can't open the fuzzed pdf file with Preview.

Compilation error: cast to pointer from integer of different size on ppc64le

Hi,
here are some more compilation errors. This time from an attempt to compile on ppc64le arch:

linux/ptrace_utils.c: In function 'arch_getPC':
linux/ptrace_utils.c:402:63: error: unused parameter 'status_reg' [-Werror=unused-parameter]
 static size_t arch_getPC(pid_t pid, REG_TYPE * pc, REG_TYPE * status_reg)
                                                               ^~~~~~~~~~
linux/ptrace_utils.c: In function 'arch_ptraceAnalyzeData':
linux/ptrace_utils.c:696:27: error: cast to pointer from integer of different size [-Werror=int-to-pointer-cast]
             funcs[0].pc = (void *)pc;
                           ^
linux/ptrace_utils.c: In function 'arch_ptraceSaveData':
linux/ptrace_utils.c:762:27: error: cast to pointer from integer of different size [-Werror=int-to-pointer-cast]
             funcs[0].pc = (void *)pc;
                           ^

make fail in macOS 10.12.4

Undefined symbols for architecture x86_64:
"_CSSymbolicatorGetDyldAllImageInfosAddress", referenced from:
-[CrashReport _extractDyldInfoFromAllImageInfos:withMemory:] in CrashReport_Yosemite.o
ld: symbol(s) not found for architecture x86_64

because some symbols not found in third_party/mac/CrashReport_Yosemite.o , and Apple CrashWarngler has not been updated for a long time.
Do you have a solution for this?

Darwin sigaction struct has no .sa_restorer member

Darwin sigaction struct has no ".sa_restorer" member. Bug was introduced at commit:0764ad7

Member initialization can be wrapped with "_HF_ARCH_LINUX" ifdef or removed completely since it doesn't seem to affect timer logic.

Also just to let you know that display.c has a few errors under MAC due to gnu-zero-variadic-macro-arguments (dprintf macro) and integer format (different return values for __sync_add_and_fetch()).

New display timer is also not working in Android (no idea why). Will start investigating as soon feature reaches a stable state.

Cool stuff, cheers!

Error when build arm on release version 0.9

~/Downloads/honggfuzz-0.9$ third_party/android/scripts/compile-libunwind.sh third_party/android/libunwind x86_64
fatal: Not a git repository (or any of the parent directories): .git
[-] git submodules init failed

~/Downloads/honggfuzz-0.9$ third_party/android/scripts/compile-libunwind.sh third_party/android/libunwind arm
fatal: Not a git repository (or any of the parent directories): .git
[-] git submodules init failed

Wiki link?

I might just be tired, but the link to the wiki/docs on line 4 of README.md seems to link back to itself?

files_mapSharedMem() is failing on Android

Android has no "/tmp" as such files_mapSharedMem() will fail:

https://github.com/google/honggfuzz/blob/master/files.c#L494

void *files_mapSharedMem(size_t sz, int *fd)
{
    char template[] = "/tmp/hfuzz.XXXXXX";
    if ((*fd = mkstemp(template)) == -1) {
        PLOG_W("mkstemp('%s')", template);
        return MAP_FAILED;
    }
    unlink(template);

Also considering the SELinux enforcing limitations if fuzzing against Android production images, I think the best approach would be to create file under the workspace directory (hfuzz->workDir).

Something like the following:

    char template[PATH_MAX] = { 0 };
    snprintf(template, sizeof(template), "%s/%s", hfuzz->workDir, "hfuzz_SHMEM.XXXXXX");
    if ((*fd = mkstemp(template)) == -1) {

I haven't pushed a pull request since I'm not sure what would be your preferred way of dealing with the func arguments in case you want to keep the method as abstract as possible for other usages too.

Dumping a non-crash path

I'm looking for a way to 'flag' a function, and dump it as if it was a crash when it's reached. Any suggestion? I'm trying to get honggfuzz to help out with angr (which can load state data).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.