Tue 6 Nov 2018 14:37 - 15:00 at Horizons 6-9F - Deep Learning Chair(s): David Rosenblum

Artificial intelligence models are becoming an integral part of modern computing systems. Just like software inevitably has bugs, models have bugs too, leading to poor classification/prediction accuracy. Unlike software bugs, model bugs cannot be easily fixed by directly modifying models. Existing solutions work by providing additional training inputs. However, they have limited effectiveness due to the lack of understanding of model misbehaviors and hence the incapability of selecting proper inputs. Inspired by software debugging, we propose a novel model debugging technique that works by first conducting model state differential analysis to identify the internal features of the model that are responsible for model bugs and then performing training input selection that is similar to program input selection in regression testing. Our evaluation results on 29 different models for 6 different applications show that our technique can fix model bugs effectively and efficiently without introducing new bugs. For simple applications (e.g., digit recognition), MODE improves the test accuracy from 75% to 93% on average whereas the state-of-the-art can only improve to 85% with 11 times more training time. For complex applications and models (e.g., object recognition), MODE is able to improve the accuracy from 75% to over 91% in minutes to a few hours, whereas state-of-the-art fails to fix the bug or even degrades the test accuracy.

Tue 6 Nov

Displayed time zone: Guadalajara, Mexico City, Monterrey change

13:30 - 15:00
Deep LearningResearch Papers at Horizons 6-9F
Chair(s): David Rosenblum National University of Singapore
13:30
22m
Talk
Deep Learning Type Inference
Research Papers
Vincent J. Hellendoorn University of California at Davis, USA, Christian Bird Microsoft Research, Earl T. Barr , Miltiadis Allamanis Microsoft Research, Cambridge
13:52
22m
Talk
DeepSim: Deep Learning Code Functional Similarity
Research Papers
Gang Zhao , Jeff Huang Texas A&M University
14:15
22m
Talk
Code Vectors: Understanding Programs Through Embedded Abstracted Symbolic Traces
Research Papers
Jordan Henkel University of Wisconsin–Madison, Shuvendu K. Lahiri Microsoft Research, Ben Liblit University of Wisconsin–Madison, Thomas Reps University of Wisconsin - Madison and GrammaTech, Inc.
14:37
22m
Talk
MODE: Automated Neural Network Model Debugging via State Differential Analysis and Input Selection
Research Papers
Shiqing Ma Purdue University, USA, Yingqi Liu Purdue University, USA, Wen-Chuan Lee Purdue University, Xiangyu Zhang Purdue University, Ananth Grama Purdue University, USA