Skip to content

What Is Fuzz Testing?

Fuzzing is a dynamic application security testing method used for finding functional bugs
and security issues in software.

What to Expect on this Page

 

What Are the Benefits of Fuzz Testing?

Fuzz testing enables developers to ship secure software fast, by detecting security and stability issues in the early stages of software development.

1. Run Security Tests On the Source Code

During a fuzz test, a program gets executed with invalid, unexpected, or random inputs, with the aim to crash the application. Modern fuzzing solutions can analyze the structure of the code they are supposed to test. They can generate thousands of automated test cases per second, and mark each path the inputs take through the program. This way, a fuzzer gets detailed feedback about the code coverage, the inputs are reaching during the execution of the source code.

2. Each Finding Leads to More Findings

Once a fuzzer found an input that has caused a crash, it uses mutation algorithms to generate even more inputs that can reproduce the finding with a high probability. 

3. Fuzzing Protects Against Unexpected Edge Cases

Modern software fuzzers execute a program with invalid, unexpected, or random inputs. This way you can also cover unlikely or unexpected edge cases, that you would not cover with other testing approaches. 

4. Fuzzing Maximizes Code Coverage Without False Positives

Since fuzzers actually execute the software under test, they always provide inputs that you can use to reproduce the bug. This way, fuzzing enables you to reach up to 99% code coverage without any false positives. Fuzzing speeds up your development process and helps you to ship more reliable and secure software.

Fuzzing Explained in 120 Seconds

Fuzzing Explained for Developers

Due to their high degree of automation, modern fuzzing solutions, such as CI Fuzz, enable developers to conduct advanced security tests themselves.

Industries Where Fuzzing Is Used

More and More Industry Standards Require Fuzz Testing

Due to increasing security regulations, more and more software companies have to run automated security tests before shipping their software. That's why many industries and ISO standards recommend integrating automated fuzz testing into the development process. Especially in industries, that already have advanced quality and security regulations. A good example is ISO/SAE 21434 and UNECE WP.29, which deal with the security of automotive software.

What Standards and ISO Norms Recommend Fuzzing?

ISO 26262
Road vehicles – Functional Safety
UNECE WP.29
United Nations World Forum for Harmonization of Vehicle Regulations

ISA/IEC 62443-4-1
Secure Product Development Lifecycle Requirements
ISO/SAE DIS 21434
Road Vehicles — Cybersecurity Engineering
UL2900-1 and UL2900-2-1
Healthcare and Wellness Systems - Software Cybersecurity for
Network-Connectable Products
ISO/IEC/IEEE 29119
Software and Systems Engineering - Software Testing
ISO/IEC 12207
Systems and Software Engineering – Software Life Cycle Processes
ISO 27001
Information Technology – Security Techniques – Information Security Management Systems
ISO 22301
Security and Resilience — Business Continuity Management Systems
IT-Grundschutz (Germany)
Based on ISO 27001
NIST SP 800-95
Web Services — standard for software testing (USA)
and others
 

Microsoft and Google Are Driving Industry Research

Tech corporations like Microsoft and Google were early adopters of fuzzing technologies to test their own systems with it. Google released ClusterFuzz, the first cloud-based fuzzing infrastructure, in 2012 as part of a larger open source strategy. Since then, numerous open source tools (such as AFL, libFuzzer, HonggFuzz, and OSS-Fuzz) have also emerged, enabling more and more people to use fuzzing technologies. At this point, it was already becoming apparent that fuzzing was going to be a best practice in security testing. However, one major problem that was still looming at the time was the usability of fuzz testing. Open source tools such as AFL and libFuzzer were primarily developed by security experts for security experts and were intended for very specific fields of application. At first, normal developers could do little with them.

Best Practice: How Is Fuzzing Used in Practice?

Fuzzing is most effective when testing is done continuously. Therefore, it is a good idea to integrate fuzzing into CI/CD pipelines. This enables short feedback cycles and makes it possible for developers to quickly fix security vulnerabilities before they become a problem. But when implementing CI/CD fuzzing, developers should be careful not to block the pipeline for too long. This is because fuzzing can take a very long time in some circumstances. This problem can be solved by defining fixed time periods in which fuzzing should take place. For example, at night, between 10 pm and 6 am. Or, by performing regression tests on a regular basis. The fuzzer can also take over the corpus and crash inputs from past test runs, so that you don't have to start from scratch with every test.
When working together in larger development teams, it can also be useful to integrate fuzzing tools into code hosting systems like Gitlab or GitHub. This facilitates team communication and alerts developers if a problem is found during a pull request. Some commercial fuzzing solutions offer such integrations as well as helpful interfaces to issue tracking systems like Jira and Jenkins.

If vulnerabilities in dependencies of modern Java applications are exploited, they can be used to maliciously spread data throughout multiple microservices rapidly. Since many microservices fully trust inputs from internal sources, attackers can often access large parts of the entire web backend through a small entry point. This was most recently shown by the RCE found in the popular Java library log4j, which affected thousands of Java applications around the world.

While modern microservice architectures are easier to maintain and more fail-safe, their complexity leaves little room for error. Accordingly, Java applications need to be tested thoroughly and with testing tools that can deal with their complexity.

How Fuzzing Prevented a Total Ethereum Shutdown

Fuzz testing finds bugs that other testing methods cannot detect. In November 2020 a serious DoS vulnerability was fixed in the source code of the Ethereum network using advanced fuzz tests. In the wrong hands, this vulnerability (CVE-2020-28362) could have caused a shutdown of the entire Ethereum network. Although the memory-safe Golang module has already undergone extensive security testing, this vulnerability could only be found through fuzz testing.

Ethereum

Frequently Asked Questions About Fuzzing (FAQs)

New to fuzzing? To help you get acquainted, we answered some of the most frequently asked questions surrounding fuzzing below.

1. Why Is Fuzzing (Especially) Useful for Security Testing?

There are some features that make software fuzzing enormously useful for security testing. Here is why: 
Fuzzing can be almost completely automated  
Fuzzing tools do not only find vulnerabilities, they also provide the inputs that trigger them 
Fuzzing detects bugs without false positives
  

2. What Is Feedback-Based Fuzzing?

As the name implies, feedback-based fuzzers collect feedback about the code coverage of previous inputs and use it to generate new, more efficient inputs. Collecting this feedback allows feedback-based fuzzers to constantly refine and improve test inputs and to monitor which parts of the program they reach. Since code coverage is one of the most important metrics in software security testing, feedback-based fuzzing tools can provide useful reports that aid in the interpretation of test findings.

3. What Is a Fuzz Target?

Fuzz targets are small programs that test predefined API functions, similar to unit tests. However, the inputs are not provided by the developer, but produced by fuzz generators. The generators are responsible for creating random mutations of inputs that are sent to the software under test (SUT). The output of a fuzz generator (i.e., random inputs) is then sent to the SUT. The delivery mechanism processes inputs from the fuzz generator and feeds them to the SUT for execution.   

Finally, the monitoring system keeps track of how inputs are executed within the SUT and detects triggered bugs. This plays a critical part in the software fuzzing process, as it also influences what types of vulnerabilities can be discovered during fuzzing.  

Example of a Fuzz Target, for Java Applications

Click here to learn how to set up fuzz targets yourself.

public class ExampleValueProfileFuzzer {
// consume data from fuzzer and give it to function under test
public static void fuzzerTestOneInput(FuzzedDataProvider data) {
String str = data.consumeString(100); // get a string with max_len 100
long input1 = data.consumeLong();
long input2 = data.consumeLong();
// call function under test with fuzz data
ApplicationUnderTest.consume(str, input1, input2);
}

4. What Are the Benefits of Fuzzing Compared to Other Testing Methods?

If you are looking for a way to secure your software, there are a variety of testing approaches, such as Static Applications Security Testing (SAST), Dynamic Application Security Testing (DAST), Interactive Application Security Testing (IAST), and Feedback-Based Application Security Testing (FAST). Each of these methods has its advantages and disadvantages. We have collected some of them in the matrix below.   

FuzzingIsTheMostScalableApproach

Fuzz Testing compared to other Application Security Testing approaches, such as SAST, DAST and IAST. 

If vulnerabilities in dependencies of modern Java applications are exploited, they can be used to maliciously spread data throughout multiple microservices rapidly. Since many microservices fully trust inputs from internal sources, attackers can often access large parts of the entire web backend through a small entry point. This was most recently shown by the RCE found in the popular Java library log4j, which affected thousands of Java applications around the world.

While modern microservice architectures are easier to maintain and more fail-safe, their complexity leaves little room for error. Accordingly, Java applications need to be tested thoroughly and with testing tools that can deal with their complexity.

5. What Bugs Can You Find With Fuzzing?

Since 2018, Code intelligence provides a platform for automated fuzz testing. Working closely together with industry and academia,  engineers at Code Intelligence were able to find thousands of bugs and extended the reach of modern fuzz testing to a variety of different use cases. Here, you will find an overview of some of the bug classes that the CI team found over the past years. Click here to see the full list.

  • Memory Leaks
    Incorrect management of memory allocation, which is exhausting available memory

  • Injections
    Untrusted input supplied to a program

  • Sensitive Data Exposure
    When an application inadvertently exposes personal data

  • Insecure Deserialization
    Manipulated object injected into the context of a web application

  • Buffer Overflow
    Exceeding the buffer when writing data, which can lead to overwriting adjacent memory locations

  • Use After Free
    No pointing to a valid object that can lead to data corruption, segmentation faults, or general protection faults

  • Data Races
    Accessing the same memory location concurrently in multi-thread process, which can lead to security vulnerabilities

  • Software Crashes
    Failing to function properly and exiting, possible fatal system errors

  • Functional Bugs
    Ceasing to respond to inputs

  • Uncaught Exceptions
    Not properly handling exceptions, which can lead to disrupting program executing

  • Undefined Behavior
    Executing code not prescribed by language specification (C/C++), which can lead to security vulnerabilities

  • And many more

6. What Is the Difference Between Black-Box Fuzzing vs. White-Box Fuzzing?

Much has happened since the early days of fuzzing in the 80s. Among modern solutions, one has to differ between white-box and black-box fuzzers. 

What Is Black-Box Fuzzing?  

Black-box fuzzing generates inputs for a target program without knowledge of its internal behavior or implementation. A black-box fuzzer may generate inputs from scratch, or rely on a static corpus of valid input files to base mutations on. Unlike coverage-guided approaches and white-box fuzzing, the corpus does not grow here.

What Is White-Box Fuzzing?  

White-box fuzzers analyses the internal structure of a program, and with each new fuzzing run, learn to track and maximize code coverage. White-box fuzzers usually use intelligent instrumentation and adaptable algorithms, which makes them more effective and accurate in detecting vulnerabilities. 
 
Black_vs_white_box_testing

The difference between black-box-fuzzing and white-box-fuzzing.

 

7. List of Common Fuzzing Tools

Developers can benefit from a whole range of open-source software fuzzing tools. There are often specialized for specific use cases (e.g. Kernel fuzzing) or programming languages. But there are also a few commercial solutions that become relevant if you're working in larger development teams or DevOps environments. Usually they come with more integrations and features, such as automated bug reporting, CI/CD and dev tool integration, REST API fuzzing, or OWASP vulnerability detection.  

Open Source Fuzzers

Fuzzing: How to Get Started? 

Try to start with an open-source fuzzer like Atheris (for Python) or Jazzer (for Java Testing).  If you’re feeling more comfortable with the testing approach and want to try fuzzing in a more complex environment, there are plenty of enterprise solutions, like CI Fuzz, that come with many additional features like reporting, CI/CD/dev tool integration, and API fuzzing. It’s really not that hard, and it will instantly improve your code quality.

Fuzz Your First Application Today!

Fuzzing you software is even simpler than running unit tests. 

Fuzzing Tutorial

Learn how every developer can secure any Java library ins less than 3 minutes. 

Fuzzing Java code using Jazzer.