Peter Samarin4 min read

AI-automated Fuzzing Uncovers Two More Vulnerabilities in wolfSSL

Daniel Pouzzner from wolfSSL has challenged us to find 3 more vulnerabilities in the wolfSSL library, after we found the first one in October 2024. We weren't quite able to find three, but here are the additional two that we found:

Heap buffer overflow WRITE in wolfSSL_d2i_RSAPrivateKey_bio. Fixed in PR 8426.
Stack buffer overflow READ into heap buffer overflow WRITE in wolfSSL_SMIME_write_PKCS7. Fixed in PR 8466.

Both vulnerabilities were fixed in wolfSSL version 5.8.0, released on 24 April 2025. The fuzz tests that found these vulnerabilities were generated by our AI Test Agent.

Vulnerability #1

The first vulnerability was discovered automatically by running cifuzz spark command for a few hours. One of the auto-generated fuzz tests targeted the function wolfSSL_d2i_RSAPrivateKey_bio and triggered a heap buffer overflow write in wolfssl_read_der_bio function. If the attacker controls the input to wolfSSL_d2i_RSAPrivateKey_bio, they can write arbitrary 4 bytes of data past the array allocated on the heap in wolfssl_read_der_bio function. It does not directly lead to an exploit, however it might be used as part of a larger exploit chain in applications using wolfSSL library.

Vulnerability #2

The second vulnerability is caused by a memcpy of 4294967123 bytes from stack to heap during a call to wolfSSL_SMIME_write_PKCS7 function. Here is the corresponding stack trace:

#0 0x55e2d73917fd in __asan_memcpy /llvm-project/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:63
#1 0x55e2d7b795e7 in wc_PKCS7_WriteOut wolfssl/wolfcrypt/src/pkcs7.c:8147:9
#2 0x55e2d7b4f154 in PKCS7_EncodeSigned wolfssl/wolfcrypt/src/pkcs7.c:3271:9
#3 0x55e2d7b5292a in wc_PKCS7_EncodeSignedData wolfssl/wolfcrypt/src/pkcs7.c:3596:19
#4 0x55e2d766eb1f in wolfSSL_i2d_PKCS7 wolfssl/src/ssl_p7p12.c:388:16
#5 0x55e2d7676d7b in wolfSSL_SMIME_write_PKCS7 wolfssl/src/ssl_p7p12.c:1479:20
#6 0x55e2d73ec739 in main wolfssl/reproducer.c:176:7

Here is the code in PKCS7_EncodeSigned (wolfcrypt/src/pkcs7.c:3271) function that later causes the two buffer overflows:

esd->issuerSnSz = (word32)SetSerialNumber(pkcs7->issuerSn, pkcs7->issuerSnSz,
                          esd->issuerSn, MAX_SN_SZ, MAX_SN_SZ);
signerInfoSz += esd->issuerSnSz;

The output of function SetSerialNumber usually returns the size of the serial number. But it can also return negative error codes, such as BAD_FUNC_ARG and BUFFER_E. In case that the serial number is zero, it returns BAD_FUNC_ARG that equals -173. There is no error checking and this value is used as-is in a call to wc_PKCS7_WriteOut:

wc_PKCS7_WriteOut(pkcs7, (output2)? (output2 + idx) : NULL,
        esd->issuerSn, esd->issuerSnSz);

The negative value -173 stored in esd->issuerSnSz is used as unsigned value 4294967123 for the size of the issuer serial number. Trying to copy this many bytes from stack to heap will most likely result in a segmentation fault that is difficult to recover from, because an unknown and possibly very large number of bytes has been written all over the heap. This behavior can be triggered by using a valid certificate where the issuer serial number is zero or negative and calling the function wolfSSL_SMIME_write_PKCS7 on that certificate.

According to RFC 5280 - 4.1.2.2:

Note: Non-conforming CAs may issue certificates with serial numbers that are negative or zero. Certificate users SHOULD be prepared to gracefully handle such certificates.

Thus, a "non-conforming" CA can crash applications that call wolfSSL_SMIME_write_PKCS7 from the wolfSSL library with the issued certificate. Since a CA is usually considered trustworthy, this vulnerability is considered impractical. However, a bug in the algorithm for serial number generation of the certificate has the potential to cause crashes on a large scale.

Finding vulnerability #2

This vulnerability was found using the fuzz test that has already discovered an issue in wolfSSL. We noticed that the coverage was shallow, because the fuzzer failed to generate valid and matching private key/certificate pairs. To help the fuzzer out, we used Claude Sonnet 4 to generate a script that

generates valid private keys (RSA, EC, ED25519, and DSA), CSRs, and the resulting self-signed certificates in PEM format
concatenates the private keys and their corresponding certificates together into one file—this is the format that the fuzz test harness expects

The files were then copied into the corpus directory of the fuzz test. This boosted the coverage and quickly found vulnerability #2.

AI in Vulnerability Detection

You can book a demo for our AI Test Agent, or fuzz wolfSSL on your own by downloading a free trial version. You will need access to an LLM and set several environmental variables so that cifuzz can access it, as described here. To set up the wolfSSL library for fuzzing, you can follow our quickstart guide.

Want to learn more about fuzz testing?

The uncovered real-world vulnerability proves that AI can effectively take over manual tasks in fuzz testing, such as analyzing code, identifying the most likely attack vectors, generating and running tests, and can thereby yield great results.

Learn more about fuzz testing tools available on the market and how they are different, and see a detailed comparison of CI Fuzz, one of the market leaders, with other white-box and protocol black-box tools.

Download Free Guide