Thu 8 Nov 2018 10:30 - 10:52 at Horizons 6-9F - Repair and Synthesis Chair(s): Shahar Maoz

It is common practice for developers of user-facing software to transform a mock-up of a graphical user interface (GUI) into code. This process takes place both at an application’s inception and in an evolutionary context as GUI changes keep pace with evolving features. Unfortunately, this practice is challenging and time-consuming. In this paper, we present an approach that automates this process by enabling accurate prototyping of GUIs via three tasks: detection, classification, and assembly. First, logical components of a GUI are detected from a mock-up artifact using either computer vision techniques or mock-up metadata. Then, software repository mining, automated dynamic analysis, and deep convolutional neural networks are utilized to accurately classify GUI-components into domain-specific types (e.g., toggle-button). Finally, a data-driven, K-nearest-neighbors algorithm generates a suitable hierarchical GUI structure from which a prototype application can be automatically assembled. We implemented this approach for Android in a system called ReDraw. Our evaluation illustrates that ReDraw achieves an average GUI-component classification accuracy of 91% and assembles prototype applications that closely mirror target mock-ups in terms of visual affinity while exhibiting reasonable code structure. Interviews with industrial practitioners illustrate ReDraw’s potential to improve real development workflows.

Thu 8 Nov

Displayed time zone: Guadalajara, Mexico City, Monterrey change

10:30 - 12:00
Repair and SynthesisJournal-First / Research Papers at Horizons 6-9F
Chair(s): Shahar Maoz Tel Aviv University
10:30
22m
Talk
Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps
Journal-First
Kevin Moran College of William & Mary, Carlos Bernal-Cárdenas William and Mary, Michael Curcio , Richard Bonett , Denys Poshyvanyk William and Mary
DOI Pre-print Media Attached
10:52
22m
Talk
Detecting Speech Act Types in Developer Question/Answer Conversations during Bug Repair
Research Papers
11:15
22m
Research paper
Visual Web Test Repair
Research Papers
Andrea Stocco University of British Columbia, Rahulkrishna Yandrapally University of British Columbia, Canada, Ali Mesbah University of British Columbia
Pre-print Media Attached
11:37
22m
Talk
Syntax-Guided Synthesis of Datalog Programs
Research Papers
Xujie Si University of Pennsylvania, Woosuk Lee University of Pennsylvania, USA, Richard Zhang University of Pennsylvania, Aws Albarghouthi University of Wisconsin-Madison, Paraschos Koutris University of Wisconsin-Madison, USA, Mayur Naik University of Pennsylvania