Tue 6 Nov 2018 16:15 - 16:37 at Horizons 5 - Testing I Chair(s): David Lo

In spite of decades of research in bug detection tools, there is a surprising dearth of ground-truth corpora that can be used to evaluate the efficacy of such tools. Recently, systems such as LAVA and EvilCoder have been proposed to automatically inject bugs into software to quickly generate large bug corpora, but the bugs created so far differ from naturally occurring bugs in a number of ways. In this work, we propose a new automated bug injection system, Apocalypse, that uses formal techniques—symbolic execution, constraint-based program synthesis and model counting—to automatically inject fair (can potentially be discovered by current bug-detection tools), deep (requiring a long sequence of dependencies to be satisfied to fire), uncorrelated (each bug behaving independent of others), reproducible (a trigger input being available) and rare (can be triggered by only a few program inputs) bugs in large software code bases. In our evaluation, we inject bugs into thirty Coreutils programs as well as the TCAS test suite. We find that bugs synthesized by Apocalypse are highly realistic under a variety of metrics, that they do not favor a particular bug-finding strategy (unlike bugs produced by LAVA), and that they are more difficult to find than manually injected bugs, requiring up around 240× more tests to discover with a state-of-the-art symbolic execution tool.

Tue 6 Nov
Times are displayed in time zone: (GMT-05:00) Guadalajara, Mexico City, Monterrey change

15:30 - 17:00: Research Papers - Testing I at Horizons 5
Chair(s): David LoSingapore Management University
fse-2018-Journal-First15:30 - 15:52
Xintao Niu, Changhai Nie, Yu Lei, Hareton Leung, Xiaoyin WangUniversity of Texas at San Antonio, USA
fse-2018-research-papers15:52 - 16:15
Jiayi WeiUniversity of Texas at Austin, Jia ChenUniversity of Texas at Austin, Yu FengUniversity of California, Santa Barbara, USA, Kostas FerlesUT Austin, Isil DilligUT Austin
DOI Pre-print
fse-2018-research-papers16:15 - 16:37
Subhajit RoyIIT Kanpur, India, Awanish PandeyIIT Kanpur, India, Brendan Dolan-GavittNew York University, Yu HuNew York University, USA
fse-2018-research-papers16:37 - 17:00
Rachel Tzoref-BrillIBM Research, Shahar MaozTel Aviv University