Tue 6 Nov 2018 16:15 - 16:37 at Horizons 5 - Testing I Chair(s): David Lo

In spite of decades of research in bug detection tools, there is a surprising dearth of ground-truth corpora that can be used to evaluate the efficacy of such tools. Recently, systems such as LAVA and EvilCoder have been proposed to automatically inject bugs into software to quickly generate large bug corpora, but the bugs created so far differ from naturally occurring bugs in a number of ways. In this work, we propose a new automated bug injection system, Apocalypse, that uses formal techniques—symbolic execution, constraint-based program synthesis and model counting—to automatically inject fair (can potentially be discovered by current bug-detection tools), deep (requiring a long sequence of dependencies to be satisfied to fire), uncorrelated (each bug behaving independent of others), reproducible (a trigger input being available) and rare (can be triggered by only a few program inputs) bugs in large software code bases. In our evaluation, we inject bugs into thirty Coreutils programs as well as the TCAS test suite. We find that bugs synthesized by Apocalypse are highly realistic under a variety of metrics, that they do not favor a particular bug-finding strategy (unlike bugs produced by LAVA), and that they are more difficult to find than manually injected bugs, requiring up around 240× more tests to discover with a state-of-the-art symbolic execution tool.

Tue 6 Nov

Displayed time zone: Guadalajara, Mexico City, Monterrey change

15:30 - 17:00
Testing IJournal-First / Research Papers at Horizons 5
Chair(s): David Lo Singapore Management University
15:30
22m
Talk
Identifying failure-causing schemas in the presence of multiple faults
Journal-First
Xintao Niu , Changhai Nie , Yu Lei , Hareton Leung , Xiaoyin Wang University of Texas at San Antonio, USA
DOI
15:52
22m
Talk
Singularity: Pattern Fuzzing for Worst Case Complexity
Research Papers
Jiayi Wei University of Texas at Austin, Jia Chen University of Texas at Austin, Yu Feng University of California, Santa Barbara, USA, Kostas Ferles UT Austin, Işıl Dillig UT Austin
DOI Pre-print
16:15
22m
Talk
Bug Synthesis: Challenging Bug-Finding Tools with Deep Faults
Research Papers
Subhajit Roy IIT Kanpur, India, Awanish Pandey IIT Kanpur, India, Brendan Dolan-Gavitt New York University, Yu Hu New York University, USA
16:37
22m
Talk
Modify, Enhance, Select: Co-Evolution of Combinatorial Models and Test Plans
Research Papers
Rachel Tzoref-Brill IBM Research, Shahar Maoz Tel Aviv University