Best Practices for
Java Security Testing
Testing Java applications for security vulnerabilities is crucial to prevent downtimes
and to protect the integrity and functionality of your systems.
What to Expect on this Page
- Why Java Vulnerabilities Are So Dangerous
- Is Java More Secure Than C/C++?
- Common Java Vulnerabilities
- The Most Spectacular Java Bug of the Century: Log4j
- Common Security Testing Approaches for Java
- Testing Java Applications with Feedback-Based Fuzzing
- XSS in JSON-Sanitizer Testing a Popular Java Application
- Jazzer: Powerful Java Tests for Open-Source
- Autofuzz - Testing Java Applications in Less Than a Minute
Why Java Vulnerabilities Are So Dangerous
A large part of our lives happens on the internet. As it happens, a lot of it is written in Java (how much exactly is frequently discussed). Traditionally Java applications were large and monolithic. Within modern Java applications, there is a clear trend towards microservice architectures that are connected via web APIs (mostly REST APIs). Unlike C/C++ applications, the challenge in security testing for Java apps is not to secure the memory access, but the immense complexity that arises from the connectivity between the modules and APIs that connect them.
If vulnerabilities in dependencies of Java applications are exploited, they can be used to maliciously spread data throughout multiple microservices rapidly. Since many microservices fully trust inputs from internal sources, attackers can often access large parts of the entire web backend through a small entry point. This was most recently shown by the RCE found in the popular Java library log4j, which affected thousands of Java applications around the world.
While modern microservice architectures are easier to maintain and more fail-safe, their complexity leaves little room for error. Accordingly, Java applications need to be tested thoroughly and with testing tools that can deal with their complexity.
Is Java More Secure Than C/C++?
Memory-safe languages such as Java, Go, or Python get their names from their runtime error prevention mechanisms. Compared to C/C++, this basically makes memory-safe languages immune to memory corruption. Combined with the fact that memory-safety vulnerabilities make up a large portion of all vulnerabilities (e.g., 90% of Android vulnerabilities), this can lead to the impression that Java is much more secure than C/C++. However, this statement has to be treated carefully. Although the density of vulnerabilities in Java is far less than in C/C++, Java vulnerabilities can have devastating consequences such as downtime or data theft.
Common Java Vulnerabilities
While C/C++ vulnerabilities are frequent but generally easier to find, Java vulnerabilities are often more intricate. The OWASP foundation maintains a list of the ten most dangerous web vulnerabilities. Java applications are susceptible to most of them. Here are three particularly harmful bugs that are often found in Java applications.
During an SQL injection (SQLi) or a similar injection attack, attackers insert malicious SQL statements into a system through an input field (e.g., username or password). If the application fails to sanitize such inputs, the statement will be executed, which enables attackers to access the database.
The term SQL injection can be divided into several sub-categories, including in-band, out-of-band, error-based,and union-based SQLis. One of the best ways to prevent SQL injections is by validating user inputs thoroughly, using filters, and testing for vulnerabilities that could expose the database to arbitrary commands.
Remote Code Execution
A remote code execution (RCE) is a software attack during which attackers exploit software vulnerabilities to gain control of someone’s computing device, usually by tricking victims into downloading malware. This allows attackers to steal data, monitor devices, divert funds, and other malicious activities. To prevent RCEs, it is considered best practice to implement firewalls, alert systems, and security testing measures that can detect potentially harmful vulnerabilities.
Cross-site scripting (XSS) is a widespread security vulnerability among Java applications. When exploited, It allows attackers to inject malicious code into web pages. The code is then executed in the client's web browser. By using XSS, attackers do not target a victim directly but rather a vulnerability in a website or web application that the victim visits or uses. Thus, the vulnerable website essentially serves as a channel for the attacker to send a malicious script to the victim's browser. Cross-site scripting can be prevented by escaping and validating user inputs and by testing web applications for vulnerabilities that could be exploited.
The Most Spectacular Java Bug of the Century: Log4j
An example that demonstrates the importance of security testing for Java applications is the RCE (CVE-2021-44228) that was found in the open-source library log4j in December 2021. Since log4j is a highly popular logging utility, the disclosure of the RCE affected countless applications and over 3 Billion end devices. In early January, yet another RCE vulnerability was found in log4j, although this one was not as severe as CVE-2021-44228.
After the log4shell (CVE-2021-44228) vulnerability was patched with version 2.15, another CVE was filed. Apparently, log4j was still vulnerable in some cases to a denial of service. However, it turned out that on some systems, the issue can still lead to a remote code execution. In this video, LiveOverflow uses the Java fuzzer Jazzer to find a bypass.
Try Out Jazzer
Jazzer is a coverage-guided open-source fuzzer for Java and other JVM-based programming languages.
Jazzer has found bugs in numerous Java projects including commons imaging, jsoup, and many more.
Feel free to try it out on your own code.
Common Security Testing Approaches for Java
Like all other programming languages, there are many approaches to testing Java applications, that each have their strengths and weaknesses. The following paragraphs aim to elaborate on the characteristics of some of the most common Java testing approaches. It is important to mention that many of these testing methods complement each other and can be implemented together.
Unit Testing for Java
Unit testing is done to test an individual software unit, by ideally testing every method that it offers. Most unit tests use a predefined set of inputs, to execute the functions of the system under test. The outputs and the behavior of the application are then observed to determine whether a unit test fails or passes. In many development teams, test cases for unit tests are still created manually, which is highly time-consuming and not particularly accurate. Apart from security testing, unit tests are also frequently used to drive software design.
Static Application Security Testing for Java (SAST)
During static application security testing, the software under test is scanned without executing it to analyze its underlying framework. Since DAST is a white-box testing approach, it requires access to the source code. Opponents of SAST criticize that it produces many false-positive test results that have to be sorted out manually. SAST tools are most efficient when implemented alongside more specialized tools that can analyze the structure of Java applications and discover runtime errors.
Dynamic Application Security Testing for Java (DAST)
Dynamic Application Security Testing is a testing method during which randomized or predefined test inputs are fed into a system, with the goal of triggering faulty program states. DAST is often conducted as a black-box testing method and therefore does not require access to the source code. Since DAST tools test the software under test from the outside-in, it is a great method to test programs from the attacker's perspective.
Fuzz Testing for Java
Fuzz testing is an increasingly popular solution for Java testing. As opposed to a predefined set of inputs, fuzzers generate test inputs at random (black-box fuzzing) or based on information about the software under test (white-box fuzzing). The latter approach uses the source code to create coverage-guided test inputs that can find defects that are opaque to many other testing approaches. Fuzz testing somewhat falls out of the classical testing setup of: unit test > integration test > system test > acceptance test, as it combines several of these steps. But it can be implemented alongside this structure, since fuzzing is best applied as a method for continuous software security testing, beginning in the early stages of software development. The CI/CD integrations that many modern fuzzing platforms offer allow developers to initiate bug fixes right away, which speeds up development dramatically.
Testing Java Applications With Feedback-Based Fuzzing
Among many security experts, feedback-based fuzzing is considered best practice for application security testing in Java. What makes this fuzzing approach so effective is that it can be largely automated. Feedback-based fuzzing approaches instrument Java applications with so-called Java agents. Java agents are custom code plugins that can be configured to feed information back to the fuzzer by using the instrumentation API to modify JVM bytecode.
The fuzzer can then use this information to automatically generate further test cases that increase the code coverage of the current test. This allows the fuzzer to gradually refine test inputs up to a point where they become highly accurate and efficient at detecting even complex vulnerabilities that are off-limits to most security testing software. By integrating such an approach into your CI/CD, you can configure feedback-based fuzzers to run software security tests at each code change.
XSS in JSON-Sanitizer - Testing a Popular Java Application
JSON-sanitizer is a popular Java library developed by Google and maintained by OWASP. The JSON-sanitizer’s primary purpose is to convert JSON-like content to valid JSON. Thus, its outputs should not contain substrings that might damage your scripts or even cause Cross-site scripting (XSS).
Our Java testing experts used a property-based fuzzing approach with the open-source Java fuzzer Jazzer to test this application. During this test, they would first consume a string and pass it through the JSON sanitizer. Then, the output would be analyzed for the given string. If the output would still contain it, this would mean that the application is potentially exposed to XSS.
The above fuzz target enabled our testers to generate test inputs until the result contained an invalid string which led to the XSS vulnerability. To trigger the bug, they “escaped” the first letter of an HTML tag in the JSON string. After passing it through the sanitizer, the output was the original tag, which, by nature, closes the script block. All that was left to do was to start a new script block and insert an alert to get the XSS-bug.
Jazzer: Powerful Java Tests for Open-Source
In early 2020, Code Intelligence released Jazzer, an open-source fuzzer for Java and other JVM-based languages. Shortly after its release, Jazzer was integrated into Google’s OSS-Fuzz, to enable Java fuzzing within Google’s powerful infrastructure. While fuzzing was already finding wide adoption in C/C++, the release of Jazzer was a huge advancement as it extended fuzzing to Java and provided the open-source community with a powerful testing tool.
"Code Intelligence's new Java Fuzzer enabled us to quickly find bugs and vulnerabilities in Java applications."
Principal Software Engineer at Google
Autofuzz - An Easy Way to Get Started With Fuzzing
Jazzer recently received an Autofuzz mode that allows users to run fuzz tests without writing test cases or harnesses. For developers who are just getting started with fuzzing, writing test harnesses is often the first big obstacle. With autofuzz, it only takes a simple command line to get your fuzzer started:
jazzer --cp=[name of jar].jar \ --autofuzz=[name of class to fuzz]::[name of function to fuzz]
This allows fuzzing newcomers to start fuzzing without roadblocks and gives more experienced users the tooling to speed up their testing efforts even further. Here is a quick demonstration of how autofuzz can be used to run a complete Java fuzz test in less than a minute.
Enough Talking, Let’s Fuzz Some Code
If you want to see Java fuzzing in action, you can experience it hands-on in our CI Fuzz demo app. There, we have put together fuzzing runs with real bug findings and coverage reportings completely free of charge. You can even reach out to our engineers to onboard your own code for a test run.