Empirical studies on programming language stimuli
Document Type
Article
Department or Administrative Unit
Computer Science
Publication Date
8-21-2010
Abstract
Comprehending and debugging computer programs are inherently difficult tasks. The current approach to building program execution and debugging environments is to use exclusively visual stimuli on programming languages whose syntax and semantics has often been designed without empirical guidance. We present an alternative: Sodbeans, an open-source integrated development environment designed to output carefully chosen spoken auditory cues to supplement empirically evaluated visual stimuli. Originally designed for the blind, earlier work suggested that Sodbeans may benefit sighted programmers as well. We evaluate Sodbeans in two experiments. First, we report on a formal debugging experiment comparing (1) a visual debugger, (2) an auditory debugger, and (3) a multimedia debugger, which includes both visual and auditory stimuli. The results from this study indicate that while auditory debuggers on their own are significantly less effective for sighted users when compared with visual and multimedia debuggers, multimedia debuggers might benefit sighted programmers under certain circumstances. Specifically, we found that while multimedia debuggers do not provide instant usability, once programmers have some practice, their performance in answering comprehension questions improves. Second, we created and evaluated a pilot survey analyzing individual elements in a custom programming language (called HOP) to garner empirical metrics on their comprehensibility. Results showed that some of the most widely used syntax and semantics choices in commercial programming languages are extraordinarily unintuitive for novices. For example, at an aggregate level, the word for , as in a for loop, was rated reliably worse than repeat by more than 673% by novices. After completing our studies, we implemented the HOP programming language and integrated it into Sodbeans.
Recommended Citation
Stefik, A., & Gellenbeck, E. (2010). Empirical studies on programming language stimuli. Software Quality Journal, 19(1), 65–99. https://doi.org/10.1007/s11219-010-9106-7
Journal
Software Quality Journal
Rights
Copyright © 2010, Springer Science Business Media, LLC
Comments
This article was originally published in Software Quality Journal. The full-text article from the publisher can be found here.
Due to copyright restrictions, this article is not available for free download from ScholarWorks @ CWU.