How to Gamify Continuous Integration

Table of Links

Abstract and 1 Introduction

2.1 Software Testing

2.2 Gamification of Software Testing

3 Gamifying Continuous Integration and 3.1 Challenges in Teaching Software Testing

3.2 Gamification Elements of Gamekins

3.3 Gamified Elements and the Testing Curriculum

4 Experiment Setup and 4.1 Software Testing Course

4.2 Integration of Gamekins and 4.3 Participants

4.4 Data Analysis

4.5 Threats to Validity

5.1 RQ1: How did the students use Gamekins during the course?

5.2 RQ2: What testing behavior did the students exhibit?

5.3 RQ3: How did the students perceive the integration of Gamekins into their projects?

6 Related Work

7 Conclusions, Acknowledgments, and References

3 GAMIFYING CONTINUOUS INTEGRATION

Despite its importance, testing is not tightly integrated into most computer science curricula. Very often, it is part of general software engineering courses, where time dedicated to testing tends to be limited [25]. However, dedicated software testing courses are becoming more common. Dedicating an entire course to software testing provides the opportunity to cover many different facets of testing, ranging from testing techniques to automating testing, and also how to firmly integrate testing during software development.

3.1 Challenges in Teaching Software Testing

At the University of Passau, we established a mandatory undergraduate course on software testing in 2018. Grading is based entirely on coursework, forcing the students to engage practically with testing challenges at different levels. Besides covering the standard portfolio of techniques common in software quality assurance, where testing in practice is often conducted by dedicated software testers, we also aim to cover developer testing, i.e., the tight integration of testing into regular software development activities. Thus, one of the course aims is to establish testing as a routine activity that students perform when developing their own software.

\ However, throughout multiple iterations of the course, we observed several issues inhibiting this aim: Students exhibit a lack of motivation to write tests, which results in them writing only the bare minimum number of tests required to meet a certain goal (e.g., if code coverage is required). This also indicates that students tend to overly focus on metrics, in particular code coverage, which is easy to obtain and understand. Further evidence for the lack of engagement can be found in the students’ commit logs: Rather than including tests throughout the process and using fine-grained commits [42], testing is usually conducted as a post-hoc exercise. Anecdotally, we also observed general weaknesses such as hardcoded test data (e.g., paths to test resources) which may not be reflected in the metrics students aim for but suggest a lack of proper engagement with testing. Consequently, there is a need for further incentives to establish testing in regular software development.

\

:::info This paper is available on arxiv under CC BY-SA 4.0 DEED license.

:::

:::info Authors:

(1) Philipp Straubinger, University of Passau, Germany;

(2) Gordon Fraser, University of Passau, Germany.

:::

\