Menu

How to Write Fuzz Targets With Jazzer

May 5 2021 | 6 min

Fabian Meumertzheim is a Senior Software Engineer at Code Intelligence and one of the leading engineers behind Jazzer, Code Intelligence’s open-source fuzzer for JVM-based languages. In this article, he gives a brief introduction on how to fuzz with Jazzer.


What Is Jazzer?

Jazzer is an open-source fuzzing engine for the Java Virtual Machine (JVM). With Jazzer, developers can increase their test coverage to find edge cases and avoid software bugs more effectively. 

We built Jazzer based on popular and proven tools, including the libFuzzer fuzzing engine and JaCoCo for coverage instrumentation. Like most modern fuzzers, Jazzer is a coverage-guided and in-process fuzzer, which means that it runs Java targets in its own JVM instance while monitoring the reached code paths. This allows developers to test their software at high speed. Since Jazzer operates directly on the JVM bytecode, this is possible even without access to source code.

libFuzzer vs. Jazzer

Jazzer has all the advanced libFuzzer features and ports them to Java while also adding some additional Java smarts, like a “keep going” mode. This mode allows developers to quickly skip uninteresting exceptions in favor of more interesting bugs and vulnerabilities.

Tried and Tested libFuzzer Features:

  • Minimization with deduplication
  • Parallel fuzzing in fork mode
  • Value profiling

Additional Java Features:

  • Pure Java reproducers
  • Keep going mode
  • Method hooking framework

With Jazzer, we trigger a lot of bugs. But as a Java developer, you might want to debug the code without all the fuzzer "overhead". For this, we create simple java classes with a main function triggering the malicious inputs, so-called reproducers

Method hooking is an advanced feature that can be used to write bug detectors and sanitizers (for more information on Method hooking, have a look at our GitHub documentation.)

How to Build a Fuzz Target With Jazzer?

Below, you can see a small but challenging example application. The application consumes a String and two longs. First, it encodes the String using Base64 and checks whether it equals a predefined String. (SmF6emVy is the Base64 encoding of “Jazzer”). Only if the Base64 encoding matches does the program continue. Next, the application uses a simple XOR encryption with a static key on the two longs and again compares them to predefined values. If both match, the program continues and reaches the planted bug. While this is a simple example for humans to solve, fuzzers can easily struggle with this kind of code since very specific “magic” values need to be found to pass the checks. In this case, the fuzzer also needs to “break” the XOR encryption.

ApplicationUnderTest

Simple application. See full gist.

Jazzer Makes It Look Easy

Jazzer, however, can solve this problem with very little effort. Below you see the Jazzer fuzz target, which is basically just a static fuzzerTestOneInput function. The function takes a FuzzedDataProvider, a helper object for splitting the raw fuzzer input into useful primitive types, and gets the String and two longs required by the sample application. Then all it does is call the application with that data. It is as simple as that. 

ExampleValueProfileFuzzer

Simple fuzz target. See full gist.

It Only Takes 5 Seconds

If you run Jazzer, it will only take about five seconds to crash this target. After about 2.5 million runs with over 600 000 runs per second, Jazzer will have found the “magic” values that get it past the Base64 encoding and XOR encryption and trigger the bug. The reason Jazzer can get past these hurdles so quickly is that like libFuzzer, Jazzer uses value profiling to guide the fuzzer past these kinds of checks much more efficiently than simply hoping to stumble on the exact value by chance. For more on this see https://llvm.org/docs/LibFuzzer.html#value-profile and https://github.com/CodeIntelligenceTesting/jazzer/#value-profile.

Jazzer output

Jazzer output. See full gist

Why Fuzz Memory-Safe Languages Like Java?

Among developers, it is a common misconception, that fuzz testing is only suitable for detecting memory corruption bugs. And since Java is a memory safe language, there is no need to fuzz Java

However, fuzzers can find many more bugs than only memory corruptions. It’s just that languages like C and C++ have so many memory bugs with high impact that they get all the attention. Indeed, fuzzers can find any type of bug you care to specify. Simple yet common examples are uncaught exceptions and memory leaks that crash services or applications. Infinite loops that can be used in DoS attacks can also be found. But more interestingly, anything that can be expressed as an assertion can be found quite simply, e.g. multiple implementations or libraries that should produce the same output for a given input but diverge in edge cases. 

It is also possible to find SQL injections or even remote code execution bugs. So in our view, there is no question that memory-safe languages need to be fuzzed since all languages are susceptible to functional bugs as well as severe security vulnerabilities.

Common Java Vulnerabilities That Can Be Found With Fuzzing

Functional Bugs

  • Uncaught exceptions
  • Assertions
  • Inconsistent Implementations
    (differential fuzzing)

Security issues

  • Infinite Loops
  • OutOfMemoryError
  • SQL injections
  • XSS (Framework, Sanitizers, …)
  • RCE (Jakarta, Serialization, …)
  • ...

Use Case: Fuzzing a Popular Java Library

Let's have a look at a real bug we found with Jazzer. In this particular use case, we will talk about the JSON sanitizer, which is a popular Java library developed at Google and maintained by the OWASP Foundation. You can feed the JSON-sanitizer arbitrary bytes and strings while it ensures that the output is always valid JSON. The JSON Sanitizer guarantees that the output will never contain certain substrings that, due to the nature of how HTML is parsed, might mess up your scripts or even cause XSS. Thus, after running untrusted input through the sanitizer, you should be able to safely embed it into a script block in your HTML, like in the orange box below. 

JSON Sanitizer

JSON-sanitizer: example of an embedded script.

We set this up with a property-based fuzzing approach in Jazzer. We consume a string and pass it through the sanitizer. Then, we assert that the “safe” JSON that comes out of the sanitizer does not contain the specified substring. Because if it does, we would be vulnerable to cross-site scripting.

fuzzerTestOneInput

Property-based fuzz target. See full gist.

XSS in JSON Sanitizer (CVE-2021-23899)

With this fuzz target, we found a vulnerability in the JSON sanitizer, which is not hard to trigger. By “escaping” the first letter of an HTML tag in your JSON string, you can pass it through the sanitizer, and it will output the original tag. By the nature of HTML, this closing script tag, even if contained in a JavaScript string literal, will close the script block. You then start a new one, you insert an alert, and you get a cross-site scripting exploit.

XXS in JSON sanitizer

XXS-Bug in JSON Sanitizer (CVE-2021-23899).

Long story short: Fuzzing allowed us to find a high severity security vulnerability in this popular and well-tested Java project with very little effort.

Watch CI Fuzz Demo


Update: Google integrated Jazzer into OSS-Fuzz. Now open-source projects can leverage Google’s infrastructure and computational power to secure their Java libraries. Read the full release note on the Google Security Blog.


Jazzer Findings

Since Jazzer's release in mid-February 2021, we have already found over 100 bugs in more than 20 open-source projects that we fuzzed. 8 out of the 50 bugs were security issues, 5 of which were found in stable releases and thus assigned CVEs. (See full trophy list).

  • >100 bugs found
  • >20 OSS projects fuzzed
  • 8 Security Issues
  • 5 CVEs

I hope this post gave you a good first impression of how easy it is to find bugs in Java applications with Jazzer. If you want to learn more about Jazzer, I also recommend the article on “Engineering Jazzer.” Or just check out Jazzer on GitHub.

See Jazzer on GitHub

If you find a bug with it, I would be happy to hear from you (once it’s public) on Twitter (@fhenneke).

 


For more exciting talks on fuzz testing and web security, visit fuzzcon.eu.


Fabian Meumertzheim

About Fabian Meumertzheim

Fabian Meumertzheim is a Senior Software Engineer at Code Intelligence. He maintains and contributed to multiple open-source projects, such as Chromium, Jazzer, systemd, and Android Password Store, all with the aim of making security unobtrusive and ubiquitous.

Recent Posts

One Year of Fuzzing and Fixing Suricata

Autofuzz: Fuzzing Without Writing Fuzz Targets or Harnesses

Fuzzing 101 – The Basics (FAQ)

19 Bugs in Jsoup Found With Jazzer

Share Article

Subscribe to updates