Thu 8 Nov 2018 13:52 - 14:14 at Horizons 5 - Probabilistic Reasoning Chair(s): Antonio Filieri

Probabilistic programming systems (PP systems) allow developers
to model stochastic phenomena and perform efficient inference on
the models. The number and adoption of probabilistic programming
systems is growing significantly. However, there is no prior study
of bugs in these systems and no methodology for systematically
testing PP systems. Yet, testing PP systems is highly non-trivial,
especially when they perform approximate inference.
In this paper, we characterize 118 previously reported bugs in
three open-source PP systems—Edward, Pyro and Stan—and pro-
pose ProbFuzz, an extensible system for testing PP systems. Prob-
Fuzz allows a developer to specify templates of probabilistic models,
from which it generates concrete probabilistic programs and data
for testing. ProbFuzz uses language-specific translators to generate
these concrete programs, which use the APIs of each PP system.
ProbFuzz finds potential bugs by checking the output from running
the generated programs against several oracles, including an accu-
racy checker. Using ProbFuzz, we found 67 previously unknown
bugs in recent versions of these PP systems. Developers already
accepted 51 bug fixes that we submitted to the three PP systems,
and their underlying systems, PyTorch and TensorFlow.

Thu 8 Nov (GMT-05:00) Guadalajara, Mexico City, Monterrey change

fse-2018-research-papers
13:30 - 15:00: Research Papers - Probabilistic Reasoning at Horizons 5
Chair(s): Antonio FilieriImperial College London
fse-2018-research-papers13:30 - 13:52
Talk
Sayali KatePurdue University, USA, John-Paul OreUniversity of Nebraska-Lincoln, USA, Xiangyu ZhangPurdue University, Sebastian ElbaumUniversity of Nebraska-Lincoln, USA, Zhaogui XuNanjing University, China
Pre-print
fse-2018-research-papers13:52 - 14:14
Talk
Saikat DuttaUniversity of Illinois at Urbana-Champaign, USA, Owolabi LegunsenUniversity of Illinois at Urbana-Champaign, Zixin HuangUniversity of Illinois at Urbana-Champaign, USA, Sasa MisailovicUniversity of Illinois at Urbana-Champaign
fse-2018-research-papers14:14 - 14:36
Talk
Yamilet R. Serrano LlerenaNational University of Singapore, Singapore, Marcel BöhmeMonash University, Marc Brüninknil, Singapore, Guoxin SuUniversity of Wollongong, Australia, David S. Rosenblum