迴歸測試是一種確保軟體品質的活動，因為新功能的開發需要在軟體開發生命週期 (SDLC) 中進行。隨著軟體產品的開發，迴歸測試套件正在增長。測試套件精簡是一種透過刪除迴歸測試套件中的冗餘測試案例來加速迴歸測試過程的技術。藉由使用這種技術，可以減少迴歸測試的執行時間和軟體開發成本。然而，這種技術對於大型軟體或測試套件可能很耗時，並且在僅使用一個測試標準時會丟失一些重要的測試案例。本研究的目的是應用兩個測試標準（函數覆蓋率和代碼覆蓋）的基於叢集的技術改編迴歸測試套件精簡演算法，來達成相同或更高的故障檢測能力精簡測試套件的大小。首先，我們根據函數覆蓋率的相似性，應用三種叢集演算法對測試案例進行叢集，並利用輪廓係數來尋找最佳聚類結果。其次，基於代碼覆蓋率的四種測試套件精簡算法將應用於每個測試案例叢集。基於真實軟體專案的實驗結果將使用故障檢測損失（FDE Loss）和其他比較標准進行評估。研究表明，與原始測試套件精簡算法相比，我們提出的方法的測試套件精簡處理時間至少下降了 40%，而檢測到的故障數量卻在增加。該結果表明我們的方法 CB-TSR（CB-Greedy、CB-GE、CB-GRE 和 CB-HGS）提供了更低的處理成本並提高了有效性，同時提供相同或更好的精簡測試套件的故障檢測能力。

Regression testing is an activity that ensures software quality as new features develop and needs to be conducted in the software development life cycle (SDLC). Test suite reduction is a classic technique that speeds up the regression test process by removing redundant test cases in the regression test suite. By using this technique, the execution time of the regression tests and software development costs can be reduced. However, this technique may be time-consuming with large software or test suites and may lose some essential test cases while using only one testing criteria. The aim of this paper is to reduce test suite size with the same or higher fault detection capability by applying cluster-based techniques in the adapted regression test suite reduction process. First, we applied three cluster algorithms to cluster test cases based on the similarity of function coverage and considered the silhouette coefficient to find the best cluster result. Second, the four test suite reduction algorithms were applied with statement coverage against each cluster of the test case. The experimental results based on a real subject program were evaluated using six widely-used comparison criteria. This study revealed that our proposed method reduced the processing time of the test suite by at least 40% with higher the number of detected faults. This result suggests that our approach provides a lower processing cost and improves effectiveness with the same or better fault detection capability of the reduced test suite.

Abstract i
摘要 ii
Acknowledgement iii
List of Figures v
List of Tables vi
List of Notation vii
List of Acronyms/Abbreviations viii
1 Introduction 1
2 Background and Related Works 6
2.1 Test Suite Reduction 6
2.2 Clustering Algorithms 12
3 Cluster-based Test Suite Reduction 16
3.1 Cluster-based Greedy 18
3.2 Cluster-based GE 21
3.3 Cluster-based GRE 23
3.4 Cluster-based HGS 25
4 Experiments and Analysis 27
4.1 Experimental Setup and Subject Program 27
4.2 Comparison Criteria 28
4.3 Experimental Results of CB-TSR 31
5 Discussion and Threat to Validity 43
5.1 Research Questions 43
5.2 Thread to Validity 49
6 Conclusion and Future Work 54
Reference 56
Appendix 61

[1] P. Ammann and J. Offutt, Introduction to Software Testing, 1st ed. USA: Cambridge University Press, 2008.
[2] “IEEE Standard Glossary of Software Engineering Terminology,” IEEE Std 610.12-1990, Dec. 1990, pp. 1–84.
[3] S. Yoo and M. Harman, “Regression testing minimization, selection and prioritization: a survey,” Software Testing, Verification & Reliability, vol. 22, no. 2, Mar. 2012, pp. 67–120.
[4] G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold, “Test Case Prioritization: An Empirical Study,” in Proceedings of the IEEE International Conference on Software Maintenance, USA, Aug. 1999, p. 179.
[5] C. L. Chiang, C. Y. Huang, C. H. Lee, and K. W. Chen, “Analysis and Assessment of Weighted Combinatorial Criterion for Test Suite Reduction,” Quality and Reliability Engineering International, 2021.
[6] V. Chvatal, “A Greedy Heuristic for the Set-Covering Problem,” Mathematics of Operations Research, vol. 4, no. 3, Aug. 1979, pp. 233–235.
[7] M. J. Harrold, R. Gupta, and M. L. Soffa, “A methodology for controlling the size of a test suite,” ACM Transactions on Software Engineering and Methodology, vol. 2, no. 3, Jul. 1993, pp. 270–285.
[8] T. Y. Chen and M. F. Lau, “A new heuristic for test suite reduction,” Information and Software Technology, vol. 40, no. 5, Jul. 1998, pp. 347–354.
[9] T. Y. Chen and M. F. Lau, “A simulation study on some heuristics for test suite reduction,” Information and Software Technology, vol. 40, no. 13, Nov. 1998, pp. 777–787.
[10] B. Beizer, Software Testing Techniques, 2nd edition. New York: Van Nostrand Reinhold, 1990.
[11] J.-W. Lin and C.-Y. Huang, “Analysis of test suite reduction with enhanced tie-breaking techniques,” Information and Software Technology, vol. 51, no. 4, Apr. 2009, pp. 679–690.
[12] G. Rothermel, M. J. Harrold, J. von Ronne, and C. Hong, “Empirical studies of test-suite reduction,” Software Testing, Verification and Reliability, vol. 12, no. 4, Dec. 2002, pp. 219–249.
[13] D. Jeffrey and N. Gupta, “Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction,” IEEE Transactions on Software Engineering, vol. 33, no. 2, Feb. 2007, pp. 108–123.
[14] A. Shi, A. Gyori, M. Gligoric, A. Zaytsev, and D. Marinov, “Balancing trade-offs in test-suite reduction,” in Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, New York, NY, USA, Nov. 2014, pp. 246–256.
[15] S. Tallam and N. Gupta, “A concept analysis inspired greedy algorithm for test suite minimization,” in Proceedings of the 6th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering, New York, NY, USA, Sep. 2005, pp. 35–42.
[16] S. Sampath, S. Sprenkle, E. Gibson, and L. Pollock, “Integrating customized test requirements with traditional requirements in web application testing,” in Proceedings of the 2006 workshop on Testing, analysis, and verification of web services and applications, New York, NY, USA, Jul. 2006, pp. 23–32.
[17] C.-Y. Huang, C.-S. Chen, and C.-E. Lai, “Evaluation and analysis of incorporating Fuzzy Expert System approach into test suite reduction,” Inf. Softw. Technol., vol. 79, no. C, Nov. 2016, pp. 79–105.
[18] N. Mansour and K. El‐Fakih, “Simulated Annealing and Genetic Algorithms for Optimal Regression Testing,” Journal of Software Maintenance: Research and Practice, vol. 11, no. 1, 1999, pp. 19–34.
[19] H. Ural and H. Yenigün, “Regression test suite selection using dependence analysis,” Journal of Software: Evolution and Process, vol. 25, no. 7, 2013, pp. 681–709.
[20] J. Black, E. Melachrinoudis, and D. Kaeli, “Bi-criteria models for all-uses test suite reduction,” in Proceedings. 26th International Conference on Software Engineering, May 2004, pp. 106–115.
[21] S. Eghbali, V. Kudva, G. Rothermel, and L. Tahvildari, “Supervised tie breaking in test case prioritization,” in 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 2019, pp. 242–243.
[22] J. Farjo and W. Masri, “Weighted Execution Profiles for Software Testing,” in 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops, Cleveland, OH, USA, Mar. 2014, pp. 298–301.
[23] E. Cruciani, B. Miranda, R. Verdecchia, and A. Bertolino, “Scalable approaches for test suite reduction,” in Proceedings of the 41st International Conference on Software Engineering, Montreal, Quebec, Canada, May 2019, pp. 419–429.
[24] C. Coviello, S. Romano, G. Scanniello, A. Marchetto, G. Antoniol, and A. Corazza, “Clustering support for inadequate test suite reduction,” in 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER), Campobasso, Italy, Mar. 2018, pp. 95–105.
[25] T. Kanungo, D. M. Mount, N. S. Netanyahu, C. D. Piatko, R. Silverman, and A. Y. Wu, “A local search approximation algorithm for k-means clustering,” in Proceedings of the eighteenth annual symposium on Computational geometry, New York, NY, USA, Jun. 2002, pp. 10–18.
[26] M. Steinbach, G. Karypis, and V. Kumar, “A Comparison of Document Clustering Techniques,” Proceedings of the International KDD Workshop on Text Mining, Jun. 2000.
[27] D. Yadav and S. Dutta, “Test Case Prioritization Using Clustering Approach for Object Oriented Software,” International Journal of Information System Modeling and Design, vol. 10, Jul. 2019, pp. 92–109.
[28] M. J. Arafeen and H. Do, “Test Case Prioritization Using Requirements-Based Clustering,” in Verification and Validation 2013 IEEE Sixth International Conference on Software Testing, Mar. 2013, pp. 312–321.
[29] A. Upadhyay and A. K. Misra, “Prioritizing Test Suites Using Clustering Approach in Software Testing.” /paper/Prioritizing-Test-Suites-Using-Clustering-Approach-Upadhyay-Misra/7fad196cf21151de8cb05e3f9cd839d610b0afbd (accessed Feb. 02, 2021).
[30] S. Elbaum, G. Rothermel, S. Kanduri, and A. G. Malishevsky, “Selecting a Cost-Effective Test Case Prioritization Technique,” Software Quality Journal, vol. 12, no. 3, Sep. 2004, pp. 185–210.
[31] Z. Huang, “Extensions to the k-Means Algorithm for Clustering Large Data Sets with Categorical Values,” Data Mining and Knowledge Discovery, vol. 2, no. 3, Sep. 1998, pp. 283–304.
[32] M. Murty, “Cluster Analysis on Different Data Sets Using K-Modes and K-Prototype Algorithms,” in ICT and Critical Infrastructure: Proceedings of the 48th Annual Convention of Computer Society of India- Vol II, vol. 249, Dec. 2013, pp. 137–144.
[33] Z. He, S. Deng, and X. Xu, “Improving K-Modes Algorithm Considering Frequencies of Attribute Values in Mode,” in Computational Intelligence and Security, Berlin, Heidelberg, vol. 3801, 2005, pp. 157–162.
[34] R. S. Sangam and H. Om, “The k-modes Algorithm with Entropy Based Similarity Coefficient,” Procedia Computer Science, vol. 50, Jan. 2015, pp. 93–98.
[35] M. Mahmoodi, S. H. Tavassoli, O. Takayama, J. Sukham, R. Malureanu, and A. V. Lavrinenko, “Existence Conditions of High-k Modes in Finite Hyperbolic Metamaterials,” Laser & Photonics Reviews, vol. 13, no. 3, 2019, p. 1800253.
[36] Y. Xiao, C. Huang, J. Huang, I. Kaku, and Y. Xu, “Optimal mathematical programming and variable neighborhood search for k-modes categorical data clustering,” Pattern Recognition, vol. 90, Jun. 2019, pp. 183–195.
[37] C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge: Cambridge University Press, 2008.
[38] K. Chidananda Gowda and G. Krishna, “Agglomerative clustering using the concept of mutual nearest neighbourhood,” Pattern Recognition, vol. 10, no. 2, Jan. 1978, pp. 105–112.
[39] P. Franti, O. Virmajoki, and V. Hautamaki, “Fast Agglomerative Clustering Using a k-Nearest Neighbor Graph,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 11, Nov. 2006, pp. 1875–1881.
[40] H. Zhou and J. Gao, “Automatic Method for Determining Cluster Number Based on Silhouette Coefficient,” Advanced Materials Research, vol. 951, May 2014, pp. 227–230.
[41] D.-T. Dinh, T. Fujinami, and V.-N. Huynh, “Estimating the Optimal Number of Clusters in Categorical Data Clustering by Silhouette Coefficient,” in Knowledge and Systems Sciences, Singapore, vol. 1103, Nov. 2019, pp. 1–17.
[42] S. Aranganayagi and K. Thangavel, “Clustering Categorical Data Using Silhouette Coefficient as a Relocating Measure,” in International Conference on Computational Intelligence and Multimedia Applications (ICCIMA 2007), Sivakasi, Tamil Nadu, India, vol. 2, Dec. 2007, pp. 13–17.
[43] M. Hutchins, H. Foster, T. Goradia, and T. Ostrand, “Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria,” in Proceedings of 16th International Conference on Software Engineering, May 1994, pp. 191–200.
[44] R. Just, D. Jalali, and M. D. Ernst, “Defects4J: a database of existing faults to enable controlled testing studies for Java programs,” in Proceedings of the 2014 International Symposium on Software Testing and Analysis, New York, NY, USA, Jul. 2014, pp. 437–440.
[45] “Software-artifact Infrastructure Repository: Home.” http://sir.csc.ncsu.edu/portal/ (accessed Dec. 11, 2020).
[46] R. Just, “rjust/defects4j.” https://github.com/rjust/defects4j (accessed Mar. 06, 2021).
[47] G. Rothermel, R. H. Untch, Chengyun Chu, and M. J. Harrold, “Prioritizing test cases for regression testing,” IEEE Transactions on Software Engineering, vol. 27, no. 10, Oct. 2001, pp. 929–948.
[48] P. G. Sapna and H. Mohanty, “Clustering test cases to achieve effective test selection,” in Proceedings of the 1st Amrita ACM-W Celebration on Women in Computing in India, New York, NY, USA, Sep. 2010, pp. 1–8.
[49] S. Elbaum, A. G. Malishevsky, and G. Rothermel, “Test Case Prioritization: A Family of Empirical Studies,” IEEE Trans. Softw. Eng., vol. 28, no. 2, Feb. 2002, pp. 159–182.
[50] J. Rodgers and A. Nicewander, “Thirteen Ways to Look at the Correlation Coefficient,” American Statistician - AMER STATIST, vol. 42, Feb. 1988, pp. 59–66.
[51] M. Inaba, N. Katoh, and H. Imai, “Applications of weighted Voronoi diagrams and randomization to variance-based k-clustering,” in Proceedings of the tenth annual symposium on Computational geometry, New York, NY, USA, Jun. 1994, pp. 332–339.
[52] Z. Huang, “Clustering large data sets with mixed numeric and categorical values,” in In The First Pacific-Asia Conference on Knowledge Discovery and Data Mining, 1997, pp. 21–34.
[53] R. Sibson, “SLINK: An optimally efficient algorithm for the single-link cluster method,” The Computer Journal, vol. 16, no. 1, Jan. 1973, pp. 30–34.
[54] D. Defays, “An efficient algorithm for a complete link method,” The Computer Journal, vol. 20, no. 4, Jan. 1977, pp. 364–366.
[55] “Gcov (Using the GNU Compiler Collection (GCC)).” https://gcc.gnu.org/onlinedocs/gcc/Gcov.html (accessed Dec. 11, 2020).
[56] “PIT Mutation Testing.” https://pitest.org/ (accessed Mar. 24, 2021).
[57] A. Shi, T. Yung, A. Gyori, and D. Marinov, “Comparing and combining test-suite reduction and regression test selection,” in Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, New York, NY, USA, Aug. 2015, pp. 237–247.
[58] L. Zhang, D. Marinov, L. Zhang, and S. Khurshid, “An Empirical Study of JUnit Test-Suite Reduction,” in 2011 IEEE 22nd International Symposium on Software Reliability Engineering, Hiroshima, Japan, Nov. 2011, pp. 170–179.