Thu 8 Nov 2018 15:52 - 16:15 at Horizons 5 - Testing II Chair(s): Tevfik Bultan

Test prioritization aims to detect regression faults faster via reordering test executions, and a large number of test prioritization techniques have been proposed accordingly. However, test prioritization effectiveness is usually measured in terms of the average percentage of faults detected concerned with the number of test executions, rather than the actual regression testing time, making it unclear which technique is optimal in actual regression testing time. To answer this question, this paper first conducts an empirical study to investigate the actual regression testing time of various prioritization techniques. The results reveal a number of practical guidelines. In particular, no prioritization technique can always perform optimal in practice.

To achieve the optimal prioritization effectiveness for any given project in practice, based on the findings of this study, we design learning-based Predictive Test Prioritization (PTP). PTP predicts the optimal prioritization technique for a given project based on the test distribution analysis (i.e., the distribution of test coverage, testing time, and coverage per unit time). The results show that PTP correctly predicts the optimal prioritization technique for 46 out of 50 open-source projects from GitHub, outperforming state-of-the-art techniques significantly in regression testing time, e.g., 43.16% to 94.92% improvement in detecting the first regression fault. Furthermore, PTP has been successfully integrated into the practical testing infrastructure of Baidu (a search service provider with over 600M monthly active users), and received positive feedbacks from the testing team of this company, e.g., saving beyond 2X testing costs with negligible overheads.

Thu 8 Nov
Times are displayed in time zone: Guadalajara, Mexico City, Monterrey change

15:30 - 17:00: Testing IIResearch Papers / Journal-First at Horizons 5
Chair(s): Tevfik BultanUniversity of California, Santa Barbara
15:30 - 15:52
On the correctness of electronic documents: studying, finding, and localizing inconsistency bugs in PDF readers and files
Tomasz Kuchta, Thibaud Lutellier, Edmund Wong, Lin TanUniversity of Waterloo , Cristian CadarImperial College London
15:52 - 16:15
Optimizing Test Prioritization via Test Distribution Analysis
Research Papers
Junjie ChenPeking University, Yiling LouPeking University, China, Lingming Zhang, Jianyi ZhouPeking University, China, Xiaoleng WangBaidu, China, Dan HaoPeking University, Lu ZhangPeking University
16:15 - 16:37
How Well Are Regular Expressions Tested in the Wild?
Research Papers
Peipei WangNorth Carolina State University, USA, Kathryn StoleeNorth Carolina State University
16:37 - 17:00
Which Generated Test Failures Are Fault Revealing? Prioritizing Failures Based on Inferred Precondition Violations using PAF
Research Papers
Mijung KimHong Kong University of Science and Technology, China, Shing-Chi CheungDepartment of Computer Science and Engineering, The Hong Kong University of Science and Technology, Sunghun KimHong Kong University of Science and Technology