Optimization of Mutation Testing Challenges to Fixing Faults

Sasa Ani Arnomo, Noraini Ibrahim, Anggia Dasa Putri, Ellbert Hutabri

Abstract


One of the challenges of mutation testing is fixing faults. In the debugging phase, all live mutants were repaired. Programs need high mutation scores to be declared reliable program codes. Each mutation test can allow the identification of multiple mutants. This is what confuses the faults fixing process. The objective of this research is to get the shortest route so that it can help in sorting the mutant types during application improvement after testing. The optimization is needed considering the number of mutants in each mutation testing. The problems related to optimization are very complex. It takes a suitable method to find the shortest path by paying attention to each point. There are 30 projects chosen randomly. The operator mutations that are often killed when testing mutations are AOIU and COI. The proposed optimization for mutant repair sequence is the ant colony system (ACS). The route selection using the Ant Colony System algorithm resulted in route optimization of 1.528254. Meanwhile, if the genetic algorithm is used, the score is 1.767643. Optimization results are very helpful for developers in improving code in mutation testing. Research states the best order for handling mutants using ACS. This research can be further developed with the addition of class-level mutant cases which are produced using class mutation operators. Class mutation operators have different characteristics from traditional mutation operators. In particular, it requires changes to the program structure, such as the definition of class variables.

Keywords


Fixing faults; mutation testing; optimization; ACS.

Full Text:

PDF

References


A. Aghamohammadi, S. H. Mirian-Hosseinabadi, and S. Jalali, “Statement frequency coverage: A code coverage criterion for assessing test suite effectiveness,†Inf. Softw. Technol., vol. 129, no. September 2020, p. 106426, 2021.

A. Mustafa, W. M. N. Wan-Kadir, and N. Ibrahim, “Comparative evaluation of the state-of-art requirements-based test case generation approaches,†Int. J. Adv. Sci. Eng. Inf. Technol., vol. 7, no. 4–2 Special Issue, pp. 1567–1573, 2017.

F. F. Ismail, R. Razali, and Z. Mansor, “Considerations for cost estimation of software testing outsourcing projects,†Int. J. Adv. Sci. Eng. Inf. Technol., vol. 9, no. 1, pp. 142–152, 2019.

P. Delgado-Pérez and F. Chicano, “An experimental and practical study on the equivalent mutant connection: An evolutionary approach,†Inf. Softw. Technol., vol. 124, no. April, 2020.

X. Dang, X. Yao, D. Gong, T. Tian, and B. Sun, “Multi-Task Optimization-Based Test Data Generation for Mutation Testing via Relevance of Mutant Branch and Input Variable,†IEEE Access, vol. 8, pp. 144401–144412, 2020.

P. Pinheiro et al., “Mutating code annotations: An empirical evaluation on Java and C# programs,†Sci. Comput. Program., vol. 191, p. 102418, 2020.

N. Gupta, A. Sharma, and M. K. Pachariya, “Multi-objective test suite optimization for detection and localization of software faults,†J. King Saud Univ. - Comput. Inf. Sci., no. xxxx, 2020.

A. Usman, N. Ibrahim, and I. A. Salihu, “TEGDroid: Test case generation approach for android apps considering context and GUI events,†Int. J. Adv. Sci. Eng. Inf. Technol., vol. 10, no. 1, pp. 16–23, 2020.

S. A. Arnomo and N. Binti Ibrahim, “Priority path for mutant repairs on mutation testing,†Proc. ICAITI 2019 - 2nd Int. Conf. Appl. Inf. Technol. Innov. Explor. Futur. Technol. Appl. Inf. Technol. Innov., pp. 71–76, 2019.

J. A. do Prado Lima and S. R. Vergilio, “A systematic mapping study on higher order mutation testing,†J. Syst. Softw., vol. 154, pp. 92–109, 2019.

A. V. Pizzoleto, F. C. Ferrari, J. Offutt, L. Fernandes, and M. Ribeiro, “A systematic literature review of techniques and metrics to reduce the cost of mutation testing,†J. Syst. Softw., vol. 157, p. 110388, 2019.

A. M. Kazerouni, J. C. Davis, A. Basak, C. A. Shaffer, F. Servant, and S. H. Edwards, “Fast and accurate incremental feedback for students’ software tests using selective mutation analysis,†J. Syst. Softw., vol. 175, p. 110905, 2021.

X. Yao, G. Zhang, F. Pan, D. Gong, and C. Wei, “Orderly Generation of Test Data via Sorting Mutant Branches Based on Their Dominance Degrees for Weak Mutation Testing,†IEEE Trans. Softw. Eng., vol. 5589, no. c, pp. 1–17, 2020.

R. Gheyi et al., “Identifying method-level mutation subsumption relations using Z3,†Inf. Softw. Technol., vol. 132, no. April 2020, p. 106496, 2021.

L. Villalobos-Arias, C. Quesada-López, A. Martínez, and M. Jenkins, “Evaluation of a model-based testing platform for Java applications,†IET Softw., vol. 14, no. 2, pp. 115–128, 2020.

H. Wang, B. Du, J. He, Y. Liu, and X. Chen, “IETCR: An Information Entropy Based Test Case Reduction Strategy for Mutation-Based Fault Localization,†IEEE Access, vol. 8, pp. 124297–124310, 2020.

J. M. Zhang, L. Zhang, D. Hao, L. Zhang, and M. Harman, “An empirical comparison of mutant selection assessment metrics,†Proc. - 2019 IEEE 12th Int. Conf. Softw. Testing, Verif. Valid. Work. ICSTW 2019, pp. 90–101, 2019.

L. Gutierrez-Madronal, A. Garcia-Dominguez, and I. Medina-Bulo, “Combining Evolutionary Mutation Testing with Random Selection,†2020 IEEE Congr. Evol. Comput. CEC 2020 - Conf. Proc., 2020.

M. B. Bashir and A. Nadeem, “Improved Genetic Algorithm to Reduce Mutation Testing Cost,†IEEE Access, vol. 5, no. c, pp. 3657–3674, 2017.

N. Jatana and B. Suri, “Particle Swarm and Genetic Algorithm applied to mutation testing for test data generation: A comparative evaluation,†J. King Saud Univ. - Comput. Inf. Sci., vol. 32, no. 4, pp. 514–521, 2020.

M. Nosrati, H. Haghighi, and M. Vahidi Asl, “Test data generation using genetic programming,†Inf. Softw. Technol., vol. 130, no. September, p. 106446, 2021.

R. Jangra and R. Kait, “Analysis and comparison among Ant System; Ant Colony System and Max-Min Ant System with different parameters setting,†3rd IEEE Int. Conf. , pp. 1–4, 2017.

D. N. Mudaliar and N. K. Modi, “Design and Application of m-Mutation Operator in Genetic Algorithm to Solve Traveling Salesman Problem,†8th Int. Conf. Comput. Power, Energy, Inf. Commun. ICCPEIC 2019, pp. 94–96, 2019.

Q. Zhu, A. Zaidman, and A. Panichella, “How to kill them all: An exploratory study on the impact of code observability on mutation testing,†J. Syst. Softw., vol. 173, p. 110864, 2021.

Z. X. Lu, S. Vercammen, and S. Demeyer, “Semi-Automatic Test Case Expansion for Mutation Testing,†VST 2020 - Proc. 2020 IEEE 3rd Int. Work. Validation, Anal. Evol. Softw. Tests, pp. 1–7, 2020.

N. Yang and Y. Shi, “Research on Tourist Route based on a Novel Ant Colony Optimization Algorithm,†2019 IEEE Int. Conf. Power, Intell. Comput. Syst. ICPICS 2019, no. 3, pp. 160–163, 2019.




DOI: http://dx.doi.org/10.18517/ijaseit.12.6.14522

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development