Visible to the public A comparison of 10 sampling algorithms for configurable systemsConflict Detection Enabled

TitleA comparison of 10 sampling algorithms for configurable systems
Publication TypeConference Proceedings
Year of Publication2016
AuthorsFlavio Medeiros, Christian Kästner, Marcio Ribeiro, Rohit Gheyi, Sven Apel
Conference NameICSE '16 Proceedings of the 38th International Conference on Software Engineering
Date Published05/2016
PublisherACM New York, NY, USA ©2016
Conference LocationAustin, TX
ISBN Number978-1-4503-3900-1
KeywordsCMU, July'16

Almost every software system provides configuration options to tailor the system to the target platform and application scenario. Often, this configurability renders the analysis of every individual system configuration infeasible. To address this problem, researchers have proposed a diverse set of sampling algorithms. We present a comparative study of 10 state-of-the-art sampling algorithms regarding their fault-detection capability and size of sample sets. The former is important to improve software quality and the latter to reduce the time of analysis. In a nutshell, we found that sampling algorithms with larger sample sets are able to detect higher numbers of faults, but simple algorithms with small sample sets, such as most-enabled-disabled, are the most efficient in most contexts. Furthermore, we observed that the limiting assumptions made in previous work influence the number of detected faults, the size of sample sets, and the ranking of algorithms. Finally, we have identified a number of technical challenges when trying to avoid the limiting assumptions, which questions the practicality of certain sampling algorithms.

Citation Keynode-30350

Other available formats:

Medeiros_Comparison_10_Sampling_CK.pdfPDF document301.91 KBDownloadPreview