Exploiting abstraction, learning from random simulation, and SVM classification for efficient dynamic prediction of software health problems

We present industrial experience on software health monitoring. Our goal was to determine whether we can predict abnormal behavior, based on data captured from software system interfaces. To analyze the system state and predict software health problems, we used Support Vector Machine (SVM) based analysis. To train the SVM, we exploited random testing with feedback and swarm testing with feedback to generate traces that exercise diverse scenarios, including both normal and abnormal behaviors that can be classified based on the system state after completing an API call. We then used the resulting classifier produced by the SVM-based analysis to predict whether an API call will result in abnormal behavior, given the input values to the API, and other system information. We applied this procedure to a subset of the API functions in the YAFFS2 flash file system, with the objective of predicting whether the health parameter of available free space will go below a threshold, relative to the total space in the flash file system. For several API functions, we achieved prediction accuracy of over 96%. We attribute the high prediction accuracy to using random testing with feedback that is optimized to produce execution traces with highly diverse behavior, which combined with the chosen representation of the system state and length of the traces resulted in a sufficient number of training vectors with diverse numeric values for the API functions of interest.