Although software testing is crucial in safeguarding our rapidly digitizing society against accidental and malefic threats, students often perceive it as an unattractive activity when compared to design or coding. A more abundant exposure to bugs and their impact can improve the situation. To test this radically new hypothesis, the VU-BugZoo is developed, an interactive software testing learning platform. Built upon a repository of fault-seeded standalone and embedded software, the platform engages students and teachers in a dynamic bug-hunting experience. What counts the most in this game is the quality of the test strategy and fault diagnosis, rather than just the quantity of devised test cases.
Development of the VU-BugZoo project
The VU-BugZoo project investigated a new way to teach software testing by engaging students in a bug-hunting exercise. The project produced the following didactical (software and hardware) materials, initially intended to accompany the course Software testing, which was taught to 50 MSc Computer Science students at the VU:
- DBugIT, an online software testing instruction tool. The main attributes of the tool are that it is fun to use, and it is extremely simple. Based on a repository of fault-seeded, executable code artifacts, this web-application engages users in an exciting bug-hunting game, while challenging them to design and justify an efficient test strategy. Deployment to ca 100 computer science students during the Covid-19 pandemic and multiple lockdowns showed that in DBugIT Testing becomes more exciting. Learning software testing is more efficient and assessment is more adequate than a pen-and-paper exam. A big advantage is that it is online and scalable. It worked very well in a test in June 2024 for an exam Phyton programming for Economy with 250 students.
- A miniature Smart-home, with smart lights, temperature regulation, smart curtains, and smart garage.
- A physical replica of Therac-25 (hardware and software), a radiotherapy machine that malfunctioned in 1985 and resulted in a few overdose casualties due to bugs in software. This does not work yet.
- A multifunctional, microcontroller-based control unit with multiple input and output ports where software can be injected with bugs. The control unit can be connected to both Therac-25 and smart home, and due to its versatile design, to any other system that may be built in the future.
Continued use and further development of teaching materials and products
Students appreciate the BugZoo approach, becoming more enthusiastic about learning software testing and even expressing interest in contributing to the platform’s development. The teaching materials and products created have a potentially broader audience. For example, DBugIT can be used to teach software testing to any Computer Science (CS) student beyond VU, as well as to any student with programming knowledge, even if they are not CS specialists. Additionally, the tool can be used by ICT companies to train developers on the job, during job interviews, or in programming courses.
The smart home can be used for a course on Pervasive computing for BSc students Computer Science. The Therac-25 replica can be used to teach a course on software testing to students Medical Physics and even to radiotherapy practitioners.
Although software testing is much more than just bug-hunting, the proposed approach is very effective in increasing the motivation of beginners. Therefore, we will continue to develop the tool and use it in our teaching. In the future we will be extending the corpus of corrupted artifacts with both synthetic and historical, representative, well-hidden bugs. Moreover, we will extend the tool’s functionality with automatic grading and feedback generation. Finally, we will investigate ways to make the tool available to interested communities, both in academia and industry.
| Opleiding(en) en aantal studenten waarbij de innovatie is toegepast: | 400 Computer Science at VU 250 Economy at VU 90 MakeIt Work at HvA |
|---|
| Ontwikkelperiode: | 2019-2022 |
|---|
| Opzet projectteam: | Natalia Silvis Software: Mark Went, Robert Jansma, Viktor Bonev, Emil Apostolov, Martynas Rimkevicius Hardware: Rob Limburg, Niels Althuisius, Mario Molenaar, Lars Eeuwijk, Ramon Parmassar (Beta elektronica and FMI), Glenn Visser (VU) |
|---|
| Totale budget: | EUR 100.000 (waarvan 50.000 van de afdeling Computer Science) |
|---|
Relevante publicaties
- T. Astigarraga, E. M. Dow, C. Lara, R. Prewitt &M. R. Ward, "The Emerging Role of Software Testing in Curricula,". In 2010 IEEE Transforming Engineering Education: Creating Interdisciplinary Skills for Complex Global Environments, Dublin, Ireland, 2010, pp. 1-26, https://doi.org/10.1109/tee.2010.5508833
- Silvis-Cividjian, N., Limburg, R., Althuisius, N., Apostolov, E., Bonev, V., Jansma, R., Visser, G., & Went, M. (2020). VU-BugZoo: A Persuasive Platform for Teaching Software Testing. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education (pp. 553–553). ITiCSE ’20: Innovation and Technology in Computer Science Education. ACM. https://doi.org/10.1145/3341525.3393975
- Leveson, N. G., & Turner, C. S. (1993). An investigation of the Therac-25 accidents. Computer, 26(7), 18-41. https://doi.org/10.1109/MC.1993.274940
- Silvis-Cividjian, N., Went, M., Jansma, R., Bonev, V., & Apostolov, E. (2021, June). Good bug hunting: Inspiring and motivating software testing novices. In Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1 (pp. 171-177). https://doi.org/10.1145/3430665.3456330
- Silvis-Cividjian, N. (2021, May). Awesome bug manifesto: Teaching an engaging and inspiring course on software testing (position paper). In 2021 Third International Workshop on Software Engineering Education for the Next Generation (SEENG) (pp. 16-20). IEEE. https://doi.org/10.1109/SEENG53126.2021.00010
Producten of presentaties over dit project
- Video explaining DBugIT
- Silvis-Cividjian, N. (2024, September). Bug-Hunting Games to Assist Software Development and Testing Education (Keynote). In Proceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation (pp. 1-1).https://doi.org/10.1145/3678869.3695732
- Silvis-Cividjian, N., Kenyon, J., Nazarian, E., Sluis, S., & Gevonden, M. (2024). On using physiological sensors and ai to monitor emotions in a bug-hunting game. In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 (pp. 429-435). https://doi.org/10.1145/3649217.3653611
- Silvis-Cividjian, N., Visser, G., Veltman, J., Althuisius, N., Limburg, R., & Molenaar, M. (2023, June). House of the Rising Flames: A Hands-on, Bug-centered Tutorial on Embedded Software Testing. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 2 (pp. 581-582). https://doi.org/10.1145/3587103.3594153
- Silvis-Cividjian, N., & Hager, F. (2023, May). Speak, memory! Analyzing historical accidents to sensitize software testing novices. In 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) (pp. 70-81). IEEE. https://doi.org/10.1109/ICSE-SEET58685.2023.00013
- Natalia Silvis-Cividjian, Jasper Veltman, Auke Buchel, Erik Link, Joshua Kenyon and Michel Oey, Bug Hunting Games to Add Enthusiasm in Software Testing and Programming Classes, 2025 IEEE/ACM 37th International Conference on Software Engineering Education and Training (CSEE&T), 359-364. 2025. https://doi.org/10.1109/CSEET66350.2025.00044
- Harmon, A. (Host), Cooper, L. (Host), & Silvis‑Cividjian, N. (2023, May 2). Therac‑25 in the classroom with Natalia Silvis‑Cividjian (Season 3, No. 9) [Audio podcast episode]. In Inside Out Quality. Apple Podcasts. https://podcasts.apple.com/us/podcast/therac-25-in-the-classroom-with-natalia-silvis-cividjian/id1542459355?i=1000611431098
Comenius Teaching Fellowship 2019
