Modern vehicles do not just consist of one centralized computer but of many interconnected systems that communicate with each other. Infotainment systems, for example, communicate with a whole range of external sensors. The surge of these interconnected systems led to the rapid adoption of modern fuzz testing approaches by OEMs and car manufacturers.
Fuzz Testing is a highly effective way to find security issues early in the development process. With the upcoming ISO 21434, this is now gaining even more importance, particularly within the automotive industry. In this article, I want to discuss two major challenges of fuzzing embedded systems with dependencies, and I would like to make some suggestions on how to address these problems.
Why Is it So Hard to Fuzz Embedded Applications with Dependencies?
With all this increasing connectivity, coding can often get messy and complicated pretty quickly. These dependencies add an additional layer of complexity for security testing, as they usually require plenty of manual effort. Since C and C++ code is dependent on the hardware and the operating system, it cannot be cross-compiled. This leads to two significant problems:
- Problem 1: Although the Public API documentation is usually available, developers need to write plenty of test harnesses, which is incredibly time-consuming.
- Problem 2: The communication between the hardware and the Hardware-Dependent-API has to be secured. However, the security tests have to prevent the application from crashing under all circumstances. Therefore, they have to cover all possible states and behaviors, even when a sensor should send unexpected or erroneous inputs. In this case, the system should ideally put out an error message that allows us to trace the error back to the defective sensor.
The Obvious Solution
A way to solve these two problems is a testing procedure that involves providing invalid, unexpected, or random data as input to your embedded application. Generally speaking, your hardware dependencies require fuzz testing and mocking with fuzz data. By applying fuzz testing to your software, you can protect yourself against edge cases and unexpected behaviors. It will save you valuable time and allow you to find vulnerabilities much earlier in the development process.
Use the API Documentation to Generate Fuzz Tests
You will probably need some form of structured input to automatically generate your test harnesses, for example, a documentation that defines the Public APIs. But in Automotive and other industries that work on embedded hardware, some form of API documentation often exists already.
In most cases, the information is stored in a simple CSV or Excel table. This documentation can be used to create sophisticated fuzz tests without any further adaptations automatically.
While running the automatically generated fuzz test, the fuzzer will randomly select functions from your Public API and call them in a random order with random parameters. Using feedback about the covered code, the fuzzer can change the called functions and the parameters to achieve a high code coverage. This will allow you to generate test cases you might not have thought of and detect errors early onset.
Static Mocking vs. Mocking with Fuzz Data
You would usually have to compile and run the application first, to test it with dynamic inputs. But with embedded software, you often have the problem that the code only runs on certain hardware or that the application requires input from external sources. For this reason, mocking/simulating these dependencies is needed for DAST, IAST or fuzzing.
There are different ways to mock your dependencies. For example, you could use simple mocking for the hardware-dependent functions that return static values, but this way your fuzzing runs will most likely not reach a high code coverage because not all possible behavior is taken into account.
On the other hand, you can use your fuzz testing engine to dynamically generate return values for your mocked functions. This way, you can make use of the magic of feedback-based fuzzing to simulate the behavior of the external sources under realistic conditions and cover unexpected and unlikely edge cases. However, if you have unlimited resources, I would always recommend combining different DAST, IAST, and fuzzing approaches. Because, this way, your software would become even more secure.
Using just two Excel sheets, it is possible to generate a fuzz test that will achieve high code coverage while also using fuzz data to simulate the input from external sources
Integrate Fuzz Testing In Your Development
Implementing fuzzing into your continuous integration will allow you to find vulnerabilities and bugs at an earlier stage of your development process. The earlier you find a bug in an embedded application, the easier it is to fix it.
For example, if you detect a bug in a JSON parser during unit testing, you can most likely fix it in a couple of minutes. This is relatively easy compared to fixing a crash that was possibly caused by a specific user input that only affects one certain component. So do yourself a favor and fix the bugs before they pop up in production.
If you want to learn how to fuzz embedded systems with dependencies in action, join me at my next live hacking session.
Daniel Teuchert is a Customer Success Engineer and automotive security specialist at Code Intelligence. He usually works with large car manufacturers, like Bosch and Escrypt, where he supports dev teams in automating their security testing. Daniel holds a master's degree in IT Security as well as OSWE and OSCP offensive security certifications. Together with the Customer Success Team of Code Intelligence, he strives to revolutionize the way the world tests software by supporting numerous companies in implementing modern fuzz testing.