Search Results

Documents authored by Stefik, Andreas

Document
DOI: 10.4230/DagRep.12.10.225
Toward Scientific Evidence Standards in Empirical Computer Science (Dagstuhl Seminar 22442)

Authors: Timothy Kluthe, Brett A. Becker, Christopher D. Hundhausen, Ciera Jaspan, Andreas Stefik, and Thomas Zimmermann

Published in: Dagstuhl Reports, Volume 12, Issue 10 (2023)

Abstract
Many scientific fields of study use formally established evidence standards during the peer review and evaluation process, such as Consolidated Standards of Reporting Trials (CONSORT) in medical research, the What Works Clearinghouse (WWC) used in education in the United States, or the APA Journal Article Reporting Standards (JARS) in psychology. The basis for these standards is community agreement on what to report in empirical studies. Such standards achieve two key goals. First, they make it easier to compare studies, facilitating replications, through transparent reporting and sharing of data, which can provide confidence that multiple research teams can obtain the same results. Second, they establish community agreement on how to report on and evaluate studies using different methodologies. The discipline of computer science does not have formalized evidence standards, even for major conferences or journals. This Dagstuhl Seminar has three primary objectives: 1) To establish a process for creating or adopting an existing evidence standard for empirical research in computer science. 2) To build a community of scholars that can discuss what a general standard should include. 3) To kickstart the discussion with scholars from software engineering, human-computer interaction, and computer science education. In order to better discuss and understand the implications of such standards across several empirical subfields of computer science and to facilitate adoption, we brought together participants from a range of backgrounds; including academia and industry, software engineering, computer-human interaction and computer science education, as well as representatives from several prominent journals. Funding: This material is based upon work supported by the National Science Foundation under Grant Numbers NSF HCC: 2106392 and NSF I-TEST: 2048356.
Cite as

Timothy Kluthe, Brett A. Becker, Christopher D. Hundhausen, Ciera Jaspan, Andreas Stefik, and Thomas Zimmermann. Toward Scientific Evidence Standards in Empirical Computer Science (Dagstuhl Seminar 22442). In Dagstuhl Reports, Volume 12, Issue 10, pp. 225-240, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023)

Copy BibTex To Clipboard

@Article{kluthe_et_al:DagRep.12.10.225,
  author =	{Kluthe, Timothy and Becker, Brett A. and Hundhausen, Christopher D. and Jaspan, Ciera and Stefik, Andreas and Zimmermann, Thomas},
  title =	{{Toward Scientific Evidence Standards in Empirical Computer Science (Dagstuhl Seminar 22442)}},
  pages =	{225--240},
  journal =	{Dagstuhl Reports},
  ISSN =	{2192-5283},
  year =	{2023},
  volume =	{12},
  number =	{10},
  editor =	{Kluthe, Timothy and Becker, Brett A. and Hundhausen, Christopher D. and Jaspan, Ciera and Stefik, Andreas and Zimmermann, Thomas},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/DagRep.12.10.225},
  URN =		{urn:nbn:de:0030-drops-178289},
  doi =		{10.4230/DagRep.12.10.225},
  annote =	{Keywords: Community evidence standards, Human factors}
}
Document
DOI: 10.4230/DagRep.12.1.119
The Human Factors Impact of Programming Error Messages (Dagstuhl Seminar 22052)

Authors: Brett A. Becker, Paul Denny, Janet Siegmund, Andreas Stefik, and Eddie Antonio Santos

Published in: Dagstuhl Reports, Volume 12, Issue 1 (2022)

Abstract
The impacts of many human factors on how people program are poorly understood and present significant challenges for work on improving programmer productivity and effective techniques for teaching and learning programming. Programming error messages are one factor that is particularly problematic, with a documented history of evidence dating back over 50 years. Such messages, commonly called compiler error messages, present difficulties for programmers with diverse demographic backgrounds. It is generally agreed that these messages could be more effective for all users, making this an obvious and high-impact area to target for improving programming outcomes. This report documents the program and the outputs of Dagstuhl Seminar 22052, "The Human Factors Impact of Programming Error Messages", which explores this problem. In total, 11 on-site participants and 17 remote participants engaged in intensive collaboration during the seminar, including discussing past and current research, identifying gaps, and developing ways to move forward collaboratively to address these challenges.
Cite as

Brett A. Becker, Paul Denny, Janet Siegmund, Andreas Stefik, and Eddie Antonio Santos. The Human Factors Impact of Programming Error Messages (Dagstuhl Seminar 22052). In Dagstuhl Reports, Volume 12, Issue 1, pp. 119-130, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2022)

Copy BibTex To Clipboard

@Article{becker_et_al:DagRep.12.1.119,
  author =	{Becker, Brett A. and Denny, Paul and Siegmund, Janet and Stefik, Andreas and Santos, Eddie Antonio},
  title =	{{The Human Factors Impact of Programming Error Messages (Dagstuhl Seminar 22052)}},
  pages =	{119--130},
  journal =	{Dagstuhl Reports},
  ISSN =	{2192-5283},
  year =	{2022},
  volume =	{12},
  number =	{1},
  editor =	{Becker, Brett A. and Denny, Paul and Siegmund, Janet and Stefik, Andreas and Santos, Eddie Antonio},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/DagRep.12.1.119},
  URN =		{urn:nbn:de:0030-drops-169246},
  doi =		{10.4230/DagRep.12.1.119},
  annote =	{Keywords: Accessibility; Compilers; Compiler Error Messages; Computer Science Education; Computer-Human Interaction; Computing Education; CS1; Error Messages; HCI; Human Factors; Human Computer Interaction; Novice Programmers; Programming Errors; Programming Error Messages; Programming Languages; Software Engineering; Syntax Errors}
}
Document
DOI: 10.4230/OASIcs.PLATEAU.2018.1
A Randomized Controlled Trial on the Impact of Polyglot Programming in a Database Context

Authors: Phillip Merlin Uesbeck and Andreas Stefik

Published in: OASIcs, Volume 67, 9th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2018)

Abstract
Using more than one programming language in the same project is common practice. Often, additional languages might be introduced to projects to solve specific issues. While the practice is common, it is unclear whether it has an impact on developer productivity. In this paper, we present a pilot study investigating what happens when programmers switch between programming languages. The experiment is a repeated measures double-blind randomized controlled trial with 3 groups with various kinds of code switching in a database context. Results provide a rigorous testing methodology that can be replicated by us or others and a theoretical backing for why these effects might exist from the linguistics literature.
Cite as

Phillip Merlin Uesbeck and Andreas Stefik. A Randomized Controlled Trial on the Impact of Polyglot Programming in a Database Context. In 9th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2018). Open Access Series in Informatics (OASIcs), Volume 67, pp. 1:1-1:8, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2019)

Copy BibTex To Clipboard

@InProceedings{uesbeck_et_al:OASIcs.PLATEAU.2018.1,
  author =	{Uesbeck, Phillip Merlin and Stefik, Andreas},
  title =	{{A Randomized Controlled Trial on the Impact of Polyglot Programming in a Database Context}},
  booktitle =	{9th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2018)},
  pages =	{1:1--1:8},
  series =	{Open Access Series in Informatics (OASIcs)},
  ISBN =	{978-3-95977-091-0},
  ISSN =	{2190-6807},
  year =	{2019},
  volume =	{67},
  editor =	{Barik, Titus and Sunshine, Joshua and Chasins, Sarah},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/OASIcs.PLATEAU.2018.1},
  URN =		{urn:nbn:de:0030-drops-101999},
  doi =		{10.4230/OASIcs.PLATEAU.2018.1},
  annote =	{Keywords: human-factors, randomized controlled trial, polyglot programming}
}
Document
DOI: 10.4230/DagRep.8.2.1
Evidence About Programmers for Programming Language Design (Dagstuhl Seminar 18061)

Authors: Andreas Stefik, Bonita Sharif, Brad. A. Myers, and Stefan Hanenberg

Published in: Dagstuhl Reports, Volume 8, Issue 2 (2018)

Abstract
The report documents the program and outcomes of Dagstuhl Seminar 18061 "Evidence About Programmers for Programming Language Design". The seminar brought together a diverse group of researchers from the fields of computer science education, programming languages, software engineering, human-computer interaction, and data science. At the seminar, participants discussed methods for designing and evaluating programming languages that take the needs of programmers directly into account. The seminar included foundational talks to introduce the breadth of perspectives that were represented among the participants; then, groups formed to develop research agendas for several subtopics, including novice programmers, cognitive load, language features, and love of programming languages. The seminar concluded with a discussion of the current SIGPLAN artifact evaluation mechanism and the need for evidence standards in empirical studies of programming languages.
Cite as

Andreas Stefik, Bonita Sharif, Brad. A. Myers, and Stefan Hanenberg. Evidence About Programmers for Programming Language Design (Dagstuhl Seminar 18061). In Dagstuhl Reports, Volume 8, Issue 2, pp. 1-25, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2018)

Copy BibTex To Clipboard

@Article{stefik_et_al:DagRep.8.2.1,
  author =	{Stefik, Andreas and Sharif, Bonita and Myers, Brad. A. and Hanenberg, Stefan},
  title =	{{Evidence About Programmers for Programming Language Design (Dagstuhl Seminar 18061)}},
  pages =	{1--25},
  journal =	{Dagstuhl Reports},
  ISSN =	{2192-5283},
  year =	{2018},
  volume =	{8},
  number =	{2},
  editor =	{Stefik, Andreas and Sharif, Bonita and Myers, Brad. A. and Hanenberg, Stefan},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/DagRep.8.2.1},
  URN =		{urn:nbn:de:0030-drops-92887},
  doi =		{10.4230/DagRep.8.2.1},
  annote =	{Keywords: programming language design, computer science education, empirical software engineering, eye tracking, evidence standards}
}
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact