Tue 6 Nov 2018 16:37 - 17:00 at Horizons 5 - Testing I Chair(s): David Lo

The evolution of software introduces many challenges to its testing. Considerable test maintenance efforts are dedicated to the adaptation of the tests to the changing software. As a result, over time, the test repository may inflate and drift away from an optimal test plan for the software version at hand. Combinatorial Testing (CT) is a well-known test design technique to achieve a small and effective test plan. It requires a manual definition of the test space in the form of a combinatorial model, and then automatically generates a test plan design, which maximizes the added value of each
of the tests. CT is considered a best practice, however its applicability to evolving software is hardly explored.

In this work, we introduce a first co-evolution approach for combinatorial models and test plans. By combining three building blocks, to minimally modify existing tests, to enhance them, and to select from them, we provide five alternatives for co-evolving the test plan with the combinatorial model, considering tradeoffs between maximizing fine-grained reuse and minimizing total test plan size, all while meeting the required combinatorial coverage.

We use our solution to co-evolve test plans of 48 real-world industrial models with 68 version commits. The results demonstrate the need for co-evolution as well as the efficiency and effectiveness of our approach and its implementation. We further report on an industrial project that found our co-evolution solution necessary to enable adoption of CT with an agile development process.

Conference Day
Tue 6 Nov

Displayed time zone: Guadalajara, Mexico City, Monterrey change

15:30 - 17:00
Testing IJournal-First / Research Papers at Horizons 5
Chair(s): David LoSingapore Management University
Identifying failure-causing schemas in the presence of multiple faults
Xintao Niu, Changhai Nie, Yu Lei, Hareton Leung, Xiaoyin WangUniversity of Texas at San Antonio, USA
Singularity: Pattern Fuzzing for Worst Case Complexity
Research Papers
Jiayi WeiUniversity of Texas at Austin, Jia ChenUniversity of Texas at Austin, Yu FengUniversity of California, Santa Barbara, USA, Kostas FerlesUT Austin, Isil DilligUT Austin
DOI Pre-print
Bug Synthesis: Challenging Bug-Finding Tools with Deep Faults
Research Papers
Subhajit RoyIIT Kanpur, India, Awanish PandeyIIT Kanpur, India, Brendan Dolan-GavittNew York University, Yu HuNew York University, USA
Modify, Enhance, Select: Co-Evolution of Combinatorial Models and Test Plans
Research Papers
Rachel Tzoref-BrillIBM Research, Shahar MaozTel Aviv University