In May 2010, CERT released the Basic Fuzzing Framework, a Linux-based file fuzzer. We released BFF with the intent to increase awareness and adoption of automated, negative software testing. An often-requested feature is that BFF support the Microsoft Windows platform. To this end, we have worked to create a Windows analog to the BFF: the Failure Observation Engine (FOE). Through our internal testing, we've been able to help identify, coordinate, and fix exploitable vulnerabilities in Adobe, Microsoft, Google, Oracle, Autonomy, and Apple software, as well as many others. Our office shootout post is a good example of this testing.

Why create FOE when other Windows fuzzers are freely available? In our experience, existing fuzzing tools were too difficult for vendors to use or were easy to use but not very effective. With FOE, vendors and QA testers can collect seed files, define how to run the target application in a configuration file, and begin fuzzing quickly.

Mutation Strategies

FOE 1.0 comes with a limited set of mutation strategies, or mutators:

• bytemut - randomly change a percentage of bytes per seed file
• bitmut - randomly change a percentage of bits per seed file
• swap - swap adjacent bytes in a seed file
• wave - change every byte index from 0x00 through 0xFF

These strategies are similar to those used in other dumb fuzzers such as zzuf and fuzz. Because these mutators work at a binary level, they work best against binary file formats. Copy, an additional "strategy," doesn't actually fuzz the seed file, but can be useful for testing a new config or triaging known crashing test cases.

Crash Detection

FOE provides two methods for detecting crashes, using the Console Debugger (cdb), or hooking the user mode exception dispatcher, KiUserExceptionDispatcher. In our experience, using the hook can be slightly faster for the non-crashing test case and provides a minimally invasive environment for the target application. Please note that the hook has only been tested with 32-bit Windows XP and Windows Server 2003. Depending on your platform, the installer should automatically provide an appropriate config file. FOE can optionally save "heisenbug" crashes that occur when using the hook, but do not occur when the debugger is attached.

Post-crash

FOE uses the !exploitable debugging extension to perform crash uniqueness determination and preliminary crash triage. Optionally, the crashing test case can be minimized to the smallest difference from the original seed file, similar to BFF. Output for a campaign is collected in a single output directory tree.

Installation

Because fuzzing can fill temporary directories and put the target application in an unusable state, we recommend that FOE be used in a virtual machine. To install, simply unzip and run the Windows installer, which can also download the required dependencies. An .iso image with included dependencies is available for offline, virtual machine installation.

Quick Start

1. Place seed files in the FOE seedfiles directory
2. Modify the cmdline and runid options in configs\foe.cfg. Depending on how quickly the target application consumes the input file, you may need to increase the runner and debugger runtimeout values.
3. Run "foe.py configs\foe.cfg"

Many options can be specified in the configuration file. See the included README and configs\examples for more information. For those who would like to see FOE in action, Jared Allar has created a quick demonstration video. FOE can be downloaded here.

Feedback

If you have any comments or feedback, please email us.