Thu 8 Nov 2018 16:15 - 16:37 at Horizons 5 - Testing II Chair(s): Tevfik Bultan

Developers report testing their regular expressions less than the rest of their code. In this work, we explore how thoroughly tested regular expressions are by examining open source projects.

Using standard metrics of coverage, such as line and branch coverage, gives an incomplete picture of the test coverage of regular expressions. We adopt graph-based coverage metrics for the DFA representation of regular expressions, providing fine-grained test coverage metrics. Using over 15,000 tested regular expressions in 1,225 Java projects on GitHub, we measure node, edge, and edge-pair coverage. Our results show that only 17% of the regular expressions in the repositories are tested at all. For those that are tested, the median number of test inputs is two. For nearly 42% of the tested regular expressions, only one test input is used. Average node and edge coverage levels on the DFAs for tested regular expressions are 59% and 29%, respectively. Due to the lack of testing of regular expressions, we explore whether a string generation tool for regular expressions, Rex, achieves high coverage levels. With some exceptions, we found that tools such as Rex can be used to write test inputs with similar coverage to the developer tests.

Thu 8 Nov

Displayed time zone: Guadalajara, Mexico City, Monterrey change

15:30 - 17:00
Testing IIResearch Papers / Journal-First at Horizons 5
Chair(s): Tevfik Bultan University of California, Santa Barbara
15:30
22m
Talk
On the correctness of electronic documents: studying, finding, and localizing inconsistency bugs in PDF readers and files
Journal-First
Tomasz Kuchta , Thibaud Lutellier , Edmund Wong , Lin Tan University of Waterloo , Cristian Cadar Imperial College London
DOI
15:52
22m
Talk
Optimizing Test Prioritization via Test Distribution Analysis
Research Papers
Junjie Chen Peking University, Yiling Lou Peking University, China, Lingming Zhang , Jianyi Zhou Peking University, China, Xiaoleng Wang Baidu, China, Dan Hao Peking University, Lu Zhang Peking University
16:15
22m
Talk
How Well Are Regular Expressions Tested in the Wild?
Research Papers
Peipei Wang North Carolina State University, USA, Kathryn Stolee North Carolina State University
16:37
22m
Talk
Which Generated Test Failures Are Fault Revealing? Prioritizing Failures Based on Inferred Precondition Violations using PAF
Research Papers
Mijung Kim Hong Kong University of Science and Technology, China, Shing-Chi Cheung Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Sunghun Kim Hong Kong University of Science and Technology