Fuzz Testing the Compiled Code in R Packages

Akhila Chowdary Kolla, Alex Groce, Toby Dylan Hocking

Research output: Chapter in Book/Report/Conference proceedingConference contribution


R packages written in the widely used Rcpp frame-work are typically tested using expected input/output pairs that are manually coded by package developers. These manually written tests are validated under various CRAN checks, using both static and dynamic analysis. Such manually written tests allow for subtle bugs, since they do not anticipate all possible inputs and miss important code paths. Fuzzers pass random, unexpected, potentially invalid inputs to a function, in order to identify bugs missed by manually written tests. This paper presents RcppDeepState, an R package that uses the DeepState framework to provide automatic fuzzing and symbolic execution for $R$ packages written using the Rcpp framework. Using RcppDeepState, a package developer can systematically fuzz test their Rcpp functions, without having to manually write any inputs nor expected outputs. Randomly generated inputs are passed to each Rcpp function, and Valgrind is used to check for various memory access violations and memory leaks. In our system, a test harness can be used to fuzz test an Rcpp function using different backend fuzzers including afl, libFuzzer, and HonggFuzz. For even more flexibility, $R$ package developers can write their own random generation functions and assertions. We implemented random generation functions for 8 of the most common Rcpp data types, then used these functions to fuzz test 1,185 Rcpp packages. Valgrind reported issues for more than 2,000 functions (over nearly 500 packages) which were not detected using standard CRAN checks on manually specified test/example inputs. Developers confirmed for several of these issues that the problem was reproducible and represented missing or flawed code. These results suggest that RcppDeepState is useful for finding subtle flaws in Rcpp packages.

Original languageEnglish (US)
Title of host publicationProceedings - 2021 IEEE 32nd International Symposium on Software Reliability Engineering, ISSRE 2021
EditorsZhi Jin, Xuandong Li, Jianwen Xiang, Leonardo Mariani, Ting Liu, Xiao Yu, Nahgmeh Ivaki
PublisherIEEE Computer Society
Number of pages9
ISBN (Electronic)9781665425872
StatePublished - 2021
Event32nd IEEE International Symposium on Software Reliability Engineering, ISSRE 2021 - Wuhan, China
Duration: Oct 25 2021Oct 28 2021

Publication series

NameProceedings - International Symposium on Software Reliability Engineering, ISSRE
ISSN (Print)1071-9458


Conference32nd IEEE International Symposium on Software Reliability Engineering, ISSRE 2021


  • Automated test generation
  • C++ libraries
  • Fuzzing
  • Memory errors
  • R language
  • Statistical software

ASJC Scopus subject areas

  • Software
  • Safety, Risk, Reliability and Quality


Dive into the research topics of 'Fuzz Testing the Compiled Code in R Packages'. Together they form a unique fingerprint.

Cite this