Loading...

A Probabilistic Approach to Assessing and Interpreting Test Suite Effectiveness

Agha Mohammadi, Alireza | 2022

376 Viewed
  1. Type of Document: Ph.D. Dissertation
  2. Language: Farsi
  3. Document No: 55361 (19)
  4. University: Sharif University of Technology
  5. Department: Computer Engineering
  6. Advisor(s): Mirian Hosseinabadi, Hassan
  7. Abstract:
  8. The test suite effectiveness concerns the ability of test suites to reveal faults. Mutation testing is a de facto standard to assess the test suite effectiveness. However, mutation testing is a time-consuming process. Over the years, researchers have proposed two kinds of approaches. The first category is related to code coverage criteria and assess the total test suite effectiveness. The second is known as Predictive Mutation Testing (PMT). The suggested approach is probabilistic, being in different levels of abstraction (macro and micro). First, in the macro level, there is a code coverage criterion that not only does outperform existing code coverage but also does not have a statistically significant difference with mutation score — based on the evaluation results. Kendall’s Tau and Spearman’s Rho rank correlations between the proposed approach and mutation score are 0.599 and 0.761 respectively. Second, in the micro level, a method based on the combination of Random Forest and Gradient Boosting for PMT is presented, considering the impact of unreached mutants. This method is exploited to predict individual mutant execution results rather than assessing overall test suite effectiveness. Also an interpretable model for assessing the test suite effectiveness is suggested, served as guidance for developers to take meaningful actions as for killing the very mutant. Our results indicate that the performance of PMT drastically decreases in terms of area under a receiver operating characteristic curve (AUC) from 0.833 to 0.517, when considering the impact of unreached mutants. The proposed approach improves the PMT results, achieving the median AUC value of 0.609.
  9. Keywords:
  10. Software Testing ; Interpretability ; Test Suite Effectiveness ; Predictive Mutation Testing

 Digital Object List

 Bookmark

...see more