Best Practices for REST API Testing
REST API testing is a prerequisite for the security and stability of interconnected services.
REST (Representational State Transfer) is a highly popular web API type because it offers flexible, fast, and simple communication between RESTful web applications. Compared to other API formats, REST is by far the most used, as over 80% of public web APIs are RESTful. Although REST APIs are theoretically compatible with any protocol or data format, they mostly communicate through HTTP, using JSON, XLT, HTML, XML, or simple text. Out of these data formats, JSON is the most common as it is compatible with most languages.
Their adaptability makes REST APIs especially useful for services that are growing in complexity. Thanks to their ability to process commands from multiple users and different data formats, REST APIs are highly popular in various industries, such as e-commerce or IoT.
REST APIs use five HTTP methods to request a command:
Below you can see an example of a POST request:
The main difference between REST and SOAP (Simple Object Access Protocol) is that to be RESTful, an API has to simply meet a specific set of characteristics. Meanwhile, SOAP is an actual protocol, built to enable applications to communicate across languages and platforms. REST APIs are generally seen as more flexible and faster than SOAP protocols. Although SOAP protocols slightly decrease the speed of web services, they provide several features such as improved security, atomicity, consistency isolation, and durability (ACID). SOAP interfaces can process multiple protocol types (HTTP, SMTP TCP, etc.). However, SOAP return messages are always sent in XML. Thus, while REST APIs enable flexible high-speed communication, SOAP web services are slightly slower, but offer more built-in functionality.
REST API tests are not only done for security, but also for other reasons such as performance, functionality, and stability. Which testing approach is the right one for your REST APIs, strongly depends on what you are trying to achieve. However, most modern testing tools can be used for more than one form of testing. Generally, REST API testing approaches, include:
As presented below, REST APIs consist of various different parameters such as the request methods, request URI and query parameter - just to name a few. These parameters can take up countless combinations that have to be tested. Specific parameter combinations can otherwise lead to erroneous program states.
Validating REST API parameters is highly challenging. If they are not validated properly, issues such as wrong string/data types and parameter data outside the predefined value range can come up.
The data formatting schema specifies how REST APIs handle responses and requests. The challenge in maintaining data formatting is that whenever new parameters are added, they have to be included in the schema.
Testers need to ensure that REST API calls are called in the correct order to prevent errors. In REST APIs this is especially important since they are generally multithreaded.
Setting up automated testing cycles is the part of REST API testing that requires the most manual effort. Especially for large projects enterprise testing platforms will help you speed up the initial set-up dramatically.
Especially with black-box testing tools, error reporting for REST APIs is tricky, as the amount of tested parameter combinations is unknown. The best way to monitor and report REST API tests is with coverage-guided testing approaches, as they can provide meaningful coverage and error reports.
Find out more about common REST API testing challenges.
A step-by-step walkthrough of how we found a remote code execution vulnerability in an unreleased version of the German Covid-19 tracing app (CWA), using automated testing methods.
Security testing is a particularly important part of REST API testing, as the implications of an exploited security vulnerability are usually not limited to the functionality and usability of a program. Due to the complexity and connectivity of REST APIs, conducting effective security tests that detect endpoints and cover all relevant parameter combinations is a tough nut. Manual testing is often too time-consuming and tends to neglect edge cases and vulnerabilities that stem from the communication between services.
To test your APIs with all their dependencies, it is considered a best practice to automate your testing efforts as much as possible. The main reasons why test automation is so beneficial for REST APIs are:
Automating your API tests will save you time and increase the functionality, reliability, and security of your application. So, automate your testing, if you can! But also, don’t avoid manual testing completely. Your team should always be able to run manual tests, to validate if the automated tests are still working, as they are supposed to. As always, you need to find the mix that fits your use case best.
Due to their complex structure, automated testing is one of the most effective ways to ensure the security and stability of REST APIs. But not all automated testing approaches are equally effective. The fastest way to implement test automation would be with black-box testing tools, such as Burp or OWASP ZAP, potentially enhanced with some additional system tests.
Although these black-box approaches are somewhat automated, they leave plenty of room for improvement, as they still require testers to have prior knowledge about the system under test to be effective. Black-box tests are great to test a program from an attacker's perspective. They generate test inputs randomly, from static corpora, from OpenAPI imports or based on heuristics. However, such inputs often fail to reach complex vulnerabilities and edge cases, since they do not take code coverage into account. For example, a black-box testing tool would take the API request from above and try out countless different parameter settings in hopes of identifying a request that breaks something.
Automated white-box testing is far more effective at finding buggy REST API requests: Since they use information about the source code, white-box approaches can automatically exclude irrelevant parameter settings from the corpus. Through information about code coverage, they can find crashing REST API requests much faster and much more accurately. White-box automation also enables better reporting by providing code-coverage visibility. The advantages of this approach are especially useful to secure vast microservice environments that are connected through APIs, and projects that are expanding in size.
The most efficient way to implement automated white-box testing is with feedback-based fuzzing. During feedback-based fuzz testing, the instrumentation is used to measure the test progress within individual microservices and APIs. Through this instrumentation, the fuzzer collects information about test inputs, which it then uses to create further inputs that traverse even more code paths. Since this technology can be integrated into any build system, it enables developers to continuously test their microservice architectures for security vulnerabilities and stability issues.
Daniel Teuchert explains the common challenges of fuzzing REST APIs
The video above is a snippet from a recorded live coding session on automated REST API testing. If you want to see the code that Daniel used to set up an automated test for WebGoat, check out the full recording.