Why Should You Use Fuzzing?
Fuzzing Testing is a technology that helps you ship secure software fast, by detecting security and stability issues in the earliest stages of the development process.
Menu
Test Your Software With Each Pull Request
If you run fuzzing in your CI/CD pipeline, you can perform automated penetration tests with each commit. This helps you to fix bugs faster and to improve the quality of your code.
/250x250/SOURCE_Effortless%20Integration.png "run fuzzing in your CI/CD")
What is Fuzz Testing?
Fuzz Testing is a dynamic testing method used for finding functional bugs and security issues in software. During a fuzz test a program gets executed with invalid, unexpected, or random inputs, with the aim to crash the application. This is a particularly effective approach to detecting bugs and security vulnerabilities in software.
Running Security Tests On the Source Code
Modern, effortless fuzzing solutions can analyze the structure of the code they are supposed to test. They can generate thousands of automated test cases per second, and mark each path the inputs take through the program. This way a fuzzer gets details feedback about the code coverage, the inputs are reaching during the execution of the source code.
Each Finding Leads to More Findings
Once a fuzzing solution found an input that has caused a crash, they use mutation algorithms to generate even more inputs which can reproduce the finding with a high probability.
Fuzzing Protects Against the Unexpected
Modern fuzzers executes a program with invalid, unexpected, or random inputs. This way you can also cover unlikely or unexpected edge cases, that you would not cover with other testing approaches.
Reach Maximum Code Coverage Without False Positives
Because fuzzers actually execute the software under test, they always provide you inputs that you can use to reproduce the bug. That's why fuzzing enables you to reach up to 99% code coverage without any false positives. Fuzzing speeds up your development process and helps you to ship more reliable and secure software.
Fuzzing explained in 120 seconds on the example of an image parser. Click here to see all findings and the bug report.
Fuzzing Explained for Developers
Due to their high degree of automation, modern fuzzing solutions, such as CI Fuzz, enable developers to conduct advanced security tests themselves.
/250x250/SOURCE_Unmatched%20Code%20Coverage.png "SOURCE_Unmatched Code Coverage")
Industries Where Fuzzing Is Used
/SVGs/Auto.svg "Fuzzing Automotive Industry")
/SVGs/Aviation.svg "Fuzzing Aviation Industry")
/SVGs/Finance.svg "Fuzzing Finance Industry")
/SVGs/Health.svg "Fuzzing Healthcare Industry")
/SVGs/Tele.svg "Fuzzing Telecommunication Industry")
/SVGs/Energy_Neu_transparent.svg "Fuzzing EnergyIndustry")
More and More Industry Standards Require Fuzz Testing
Due to increasing security regulations, more and more software companies have to run automated security tests before shipping their software. That's why many industry and ISO standards recommend integrating automated fuzz testing into the development process. Especially in industries, that already have advanced quality and security regulations. A good example are ISO/SAE 21434 and UNECE WP.29, which deal with the security of automotive software.
What Standards and ISO Norms Recommend Fuzzing?
- ISO 26262
Road vehicles – Functional Safety - UNECE WP.29
United Nations World Forum for Harmonization of Vehicle Regulations - ISA/IEC 62443-4-1
Secure Product Development Lifecycle Requirements - ISO/SAE DIS 21434
Road Vehicles — Cybersecurity Engineering - UL2900-1 and UL2900-2-1
Healthcare and Wellness Systems - Software Cybersecurity for Network-Connectable Products - ISO/IEC/IEEE 29119
Software and Systems Engineering - Software Testing
- ISO/IEC 12207
Systems and Software Engineering – Software Life Cycle Processes - ISO 27001
Information Technology – Security Techniques – Information Security Management Systems - ISO 22301
Security and Resilience — Business Continuity Management Systems - IT-Grundschutz (Germany)
Based on ISO 27001 - and others
How Fuzzing Prevented a Total Ethereum Shutdown
Fuzz testing finds bugs that other testing methods cannot detect. In November 2020 a serious DoS vulnerability was fixed in the source code of the Ethereum network due to advanced fuzz tests. In the wrong hands, this vulnerability (CVE-2020-28362) could have caused the entire shout down of the Ethereum network. Although the memory-safe Golang module has already undergone extensive security testing, this vulnerability could only be found through fuzz testing.
Frequently Asked Questions About Fuzzing (FAQs)
1. Why is Fuzzing (Especially) Useful for Security Testing?
There are some features that make fuzzing enormously useful for security testing. Here is why:
-
Fuzzing is considered to be an almost completely automated testing technique.
-
Fuzzing does not only find the vulnerabilities but also tells you the reasons that caused the error messages.
-
Fuzzing detects bugs without false positives.
2. What Is Feedback-Based Fuzzing?
Modern fuzzing engines use smart algorithms tailoring the input to increase the amount of code that is tested with the fuzzer. The commonly used term for this is feedback-driven or feedback-based fuzzing. It uses information about code coverage when generating new inputs. Due to measuring code coverage, the fuzzer can monitor which parts of the program were reached with a given input and reach other program parts by generating similar inputs with random but small changes.
3. What Is a Fuzz Target?
Fuzz targets are small programs that test predefined API functions, similar to unit tests. However, the inputs are not provided by the developer but produced with the fuzz generators. The generators are responsible for creating random mutations of inputs that are sent to the software under test (SUT). The output of a fuzz generator (i.e. random inputs) is then sent to the SUT. The delivery mechanism processes inputs from fuzz generator and feeds them to SUT for execution.
Finally, the monitoring system keeps track of how the inputs are executed within SUT and detects triggered bugs, which plays a critical part in the fuzzing process as it also influences what types of vulnerabilities can be discovered during fuzzing.
Example of a fuzz target, for Java applications. See full gist. Click here to learn how to set up fuzz targets yourself.
4. What Are the Benefits of Fuzzing Compared to Other Testing Methods?
If you are looking for a way to secure your software, there are a variety of testing approaches, such as Static Applications Security Testing (SAST), Dynamic Application Security Testing (DAST), Interactive Application Security Testing (IAST), and Feedback-based Application Security Testing (FAST). Each of these methods has its advantages and disadvantages. We have collected some of them in the table below.
Fuzz Testing compared to other Application Security Testing approaches, such as SAST, DAST and IAST.
5. What Bugs Can You Find With Fuzzing?
Since 2018, Code intelligence provides a platform for automated fuzz testing. Working closely together with industry and academia, engineers at Code Intelligence were able to find thousands of bugs and extended the reach of modern fuzz testing to a variety of different use cases. Here, you will find an overview of some of the bug classes that the CI team found over the past years.
Memory Leaks
Injections
Sensitive Data Exposure
Insecure Deserialization
Buffer overflows
Use After Free
Data races
Software Crashes
Functional Bugs
Uncaught Exceptions
Undefined Behavior
& many more...
Some of the most common bugs that you can find with fuzzing. Click here to see the full list.
6. What Is the Difference Between Black-Box Fuzzing vs. White-Box Fuzzing?
What Is Black-Box Fuzzing?
Black-box fuzzing generates inputs for a target program without knowledge of its internal behavior or implementation. A black-box fuzzer may generate inputs from scratch, or rely on a static corpus of valid input files to base mutations on. Unlike coverage-guided approaches and white-box fuzzing, the corpus does not grow here.
What Is White-Box Fuzzing?
White-box fuzzing analyses the internal structure of the program, and with each new fuzzing run, they learn to track and maximize the code coverage. White-box fuzzers usually use intelligent instrumentation and adaptable algorithms, which makes them more effective and accurate in detecting vulnerabilities.
The difference between black-box-fuzzing and white-box-fuzzing.
7. List of Common Fuzzing Tools
Developers can benefit from a whole range of open-source fuzzing tools. There are often specialized for specific use cases (e.g. Kernel fuzzing) or programming languages. But there are also a few commercial solutions that become relevant if you're working in larger development teams or DevOps environments. Usually they come with more integrations and features, such as automated bug reporting, CI/CD and dev tool integration, Web API fuzzing, or OWASP vulnerability detection.
Open Source Fuzzers:
Fuzzing: How to Get Started?
Try to start with an open-source fuzzer like Atheris (for Python) or Jazzer (for Java). If you’re feeling more comfortable with the testing approach and want to try fuzzing in a more complex environment, there are plenty of enterprise solutions, like CI Fuzz, that come with many additional features like reporting, CI/CD, dev tool integration and WebAPI fuzzing. It’s really not that hard, and it will instantly improve your code quality.
Fuzz Your First Application Today!
Fuzzing is even simpler than running unit tests.
Fuzzing Tutorial
Learn how every developer can secure any Java library ins less than 3 minutes.