Menu

What is Fuzz Testing? 

Fuzz Testing is a dynamic testing method used for finding functional bugs and security issues in software. During a fuzz test a program gets executed with invalid, unexpected, or random inputs, with the aim to crash the application. This is a particularly effective approach to detecting bugs and security vulnerabilities in software.

Running Security Tests On the Source Code

Modern, effortless fuzzing solutions can analyze the structure of the code they are supposed to test. They can generate thousands of automated test cases per second, and mark each path the inputs take through the program. This way a fuzzer gets details feedback about the code coverage, the inputs are reaching during the execution of the source code.

Each Finding Leads to More Findings

Once a fuzzing solution found an input that has caused a crash, they use mutation algorithms to generate even more inputs which can reproduce the finding with a high probability. 

Fuzzing Protects Against the Unexpected

Modern fuzzers executes a program with invalid, unexpected, or random inputs. This way you can also cover unlikely or unexpected edge cases, that you would not cover with other testing approaches. 

Reach Maximum Code Coverage Without False Positives

Because fuzzers actually execute the software under test, they always provide you inputs that you can use to reproduce the bug. That's why fuzzing enables you to reach up to 99% code coverage without any false positives. Fuzzing speeds up your development process and helps you to ship more reliable and secure software.

Fuzzing explained in 120 seconds on the example of an image parser. Click here to see all findings and the bug report.

Fuzzing Explained for Developers

Due to their high degree of automation, modern fuzzing solutions, such as CI Fuzz, enable developers to conduct advanced security tests themselves.

Read Fuzzing 101
SOURCE_Unmatched Code Coverage

Industries Where Fuzzing Is Used

Fuzzing Automotive Industry
Fuzzing Aviation Industry
Fuzzing Finance Industry
Fuzzing Healthcare Industry
Fuzzing Telecommunication Industry
Fuzzing EnergyIndustry

More and More Industry Standards Require Fuzz Testing

Due to increasing security regulations, more and more software companies have to run automated security tests before shipping their software. That's why many industry and ISO standards recommend integrating automated fuzz testing into the development process. Especially in industries, that already have advanced quality and security regulations. A good example are ISO/SAE 21434 and UNECE WP.29, which deal with the security of automotive software.

What Standards and ISO Norms Recommend Fuzzing?

  • ISO/IEC 12207
    Systems and Software Engineering – Software Life Cycle Processes
  • ISO 27001
    Information Technology – Security Techniques – Information Security Management Systems
  • ISO 22301
    Security and Resilience — Business Continuity Management Systems
  • IT-Grundschutz (Germany)
    Based on ISO 27001
  • and others

How Fuzzing Prevented a Total Ethereum Shutdown

Fuzz testing finds bugs that other testing methods cannot detect. In November 2020 a serious DoS vulnerability was fixed in the source code of the Ethereum network due to advanced fuzz tests. In the wrong hands, this vulnerability (CVE-2020-28362) could have caused the entire shout down of the Ethereum network. Although the memory-safe Golang module has already undergone extensive security testing, this vulnerability could only be found through fuzz testing.

Ethereum

Frequently Asked Questions About Fuzzing (FAQs)

1. Why is Fuzzing (Especially) Useful for Security Testing?

There are some features that make fuzzing enormously useful for security testing. Here is why: 
  • Fuzzing is considered to be an almost completely automated testing technique. 
  • Fuzzing does not only find the vulnerabilities but also tells you the reasons that caused the error messages. 
  • Fuzzing detects bugs without false positives. 

2. What Is Feedback-Based Fuzzing?

Modern fuzzing engines use smart algorithms tailoring the input to increase the amount of code that is tested with the fuzzer. The commonly used term for this is feedback-driven or feedback-based fuzzing.  It uses information about code coverage when generating new inputs. Due to measuring code coverage, the fuzzer can monitor which parts of the program were reached with a given input and reach other program parts by generating similar inputs with random but small changes.  

3. What Is a Fuzz Target?

Fuzz targets are small programs that test predefined API functions, similar to unit tests. However, the inputs are not provided by the developer but produced with the fuzz generators. The generators are responsible for creating random mutations of inputs that are sent to the software under test (SUT). The output of a fuzz generator (i.e. random inputs) is then sent to the SUT. The delivery mechanism processes inputs from fuzz generator and feeds them to SUT for execution.   

Finally, the monitoring system keeps track of how the inputs are executed within SUT and detects triggered bugs, which plays a critical part in the fuzzing process as it also influences what types of vulnerabilities can be discovered during fuzzing.  

ExampleValueProfileFuzzer

Example of a fuzz target, for Java applications. See full gist. Click here to learn how to set up fuzz targets yourself.

4. What Are the Benefits of Fuzzing Compared to Other Testing Methods?

If you are looking for a way to secure your software, there are a variety of testing approaches, such as Static Applications Security Testing (SAST), Dynamic Application Security Testing (DAST), Interactive Application Security Testing (IAST), and Feedback-based Application Security Testing (FAST). Each of these methods has its advantages and disadvantages. We have collected some of them in the table below.   

FuzzingIsTheMostScalableApproach

Fuzz Testing compared to other Application Security Testing approaches, such as SAST, DAST and IAST. 

5. What Bugs Can You Find With Fuzzing?

Since 2018, Code intelligence provides a platform for automated fuzz testing. Working closely together with industry and academia,  engineers at Code Intelligence were able to find thousands of bugs and extended the reach of modern fuzz testing to a variety of different use cases. Here, you will find an overview of some of the bug classes that the CI team found over the past years.

Memory Leaks

Injections

Sensitive Data Exposure

Insecure Deserialization

Buffer overflows

Use After Free

Data races

Software Crashes

Functional Bugs

Uncaught Exceptions

Undefined Behavior

& many more...

Some of the most common bugs that you can find with fuzzing. Click here to see the full list.

6. What Is the Difference Between Black-Box Fuzzing vs. White-Box Fuzzing?

What Is Black-Box Fuzzing?  
Black-box fuzzing generates inputs for a target program without knowledge of its internal behavior or implementation. A black-box fuzzer may generate inputs from scratch, or rely on a static corpus of valid input files to base mutations on. Unlike coverage-guided approaches and white-box fuzzing, the corpus does not grow here.

What Is White-Box Fuzzing?  
White-box fuzzing analyses the internal structure of the program, and with each new fuzzing run, they learn to track and maximize the code coverage. White-box fuzzers usually use intelligent instrumentation and adaptable algorithms, which makes them more effective and accurate in detecting vulnerabilities.  
Black_vs_white_box_testing

The difference between black-box-fuzzing and white-box-fuzzing.

7. List of Common Fuzzing Tools

Developers can benefit from a whole range of open-source fuzzing tools. There are often specialized for specific use cases (e.g. Kernel fuzzing) or programming languages. But there are also a few commercial solutions that become relevant if you're working in larger development teams or DevOps environments. Usually they come with more integrations and features, such as automated bug reporting, CI/CD and dev tool integration, Web API fuzzing, or OWASP vulnerability detection.  

Open Source Fuzzers:

Fuzzing: How to Get Started? 

Try to start with an open-source fuzzer like Atheris (for Python) or Jazzer (for Java).  If you’re feeling more comfortable with the testing approach and want to try fuzzing in a more complex environment, there are plenty of enterprise solutions, like CI Fuzz, that come with many additional features like reporting, CI/CD, dev tool integration and WebAPI fuzzing. It’s really not that hard, and it will instantly improve your code quality. 

Fuzz Your First Application Today!

Fuzzing is even simpler than running unit tests. 

Start Now

Code Intelligence

Fuzzing Tutorial

Learn how every developer can secure any Java library ins less than 3 minutes.