When Dynamic Slicing Helps and Hurts Spectrum-Based Fault Localization: Evidence from an Expanded TCAS Benchmark (Short Paper)

Abstract

Spectrum-based fault localization and dynamic program slicing are two established approaches to automated software debugging. A recent study provided a first experimental evaluation of a hybrid approach that combined both techniques, reporting that the combination improves the wasted effort metric for programs with multiple output variables. However, as stated in the study, further experiments are required for a final judgment of the introduced debugging methodology. In this paper, we review and extend that evaluation through three rounds of experiments, all carried out under a strict single-fault assumption. First, we replicate the third experiment from the original study to validate the obtained findings. Second, we expand the TCAS benchmark from 2 to 11 faulty variants by systematically generating nine new mutants spanning six distinct mutation operator families, and verify their quality through a dedicated kill-ability assessment. Eleven of the twelve mutants pass our quality filter and are used in two independent rounds of extended experiments , each evaluated with three spectrum-based fault localization coefficients (Ochiai, Tarantula, Sarhan-Beszédes) in both the plain and hybrid configurations. Our results confirm that the slicer’s behavior has a decisive impact on the quality of the hybrid approach and that improvements from slicing are far from uniform across mutation types. In particular, bugs that suppress the execution of entire program paths present a fundamental challenge for dynamic slicing, providing concrete evidence for the limitations reported in the original study.