Tabu Search in Covering Array Generation
Author:
Affiliation:

Fund Project:

National Natural Science Foundation of China (61272079, 61321491, 91318301); Ministry of Education PhD fund(20130091110032)

  • Article
  • | |
  • Metrics
  • |
  • Reference [39]
  • |
  • Related
  • | | |
  • Comments
    Abstract:

    Combinatorial testing can effectively detect faults caused by the interaction among the parameters of the system under test. In its 30 years of the development, covering array generation has been one of the key research areas, and relevant research articles have reached more than 200. As effective algorithms to generate covering arrays, existing tabu search algorithms have some advantages on the size of covering array, but there is still much room for improving the solution quality and calculation speed. Furthermore, the practical application of the existing algorithms is poor, because they can neither take account of constrains nor generate variable strength covering arrays. To solve the above problems, this paper proposes a new tabu search algorithm. Three improved aspects are presented. 1) The process of parameter tuning is divided into two stages:pair-wise and climbing to ensure that the optimal configuration is hit with a minimum number of configurations so as to further improve the size of covering arrays. 2) In order to improve the speed, the algorithm is parallelized. 3) Constrains and variable strength handling are added to make the algorithm adapt to various test scenarios. Experimental results show that the proposed algorithm has the advantage on the size of various covering arrays, such as fixed strength covering arrays, variable strength covering arrays and covering arrays with constraints. At the same time, the parallelization results in the increase of average speed of the algorithm by about 2.6 times.

    Reference
    [1] Nie CH, Leung H. A survey of combinatorial testing. ACM Computing Surveys, 2011,43(2):1-29.[doi:10.1145/1883612.1883618]
    [2] Nie CH, Wu HY, Niu XT, Kuo FC, Leung H, Colbourn CJ. Combinatorial testing, random testing, and adaptive random testing for detecting interaction triggered failures. Information and Software Technology, 2015,62:198-213.[doi:10.1016/j.infsof.2015.02. 008]
    [3] Kuhn DR, Reilly MJ. An investigation of the applicability of design of experiments to software testing. In:Proc. of the 27th Annual NASA Goddard Software Engineering Workshop. IEEE, 2002. 91-95.[doi:10.1109/SEW.2002.1199454]
    [4] Bryce RC, Colbourn CJ, Cohen MB. A framework of greedy methods for constructing interaction test suites. In:Proc. of the 27th Int'l Conf. on Software Engineering. IEEE, 2005. 146-155.[doi:10.1109/ICSE.2005.1553557]
    [5] Colbourn CJ, Cohen MB, Turban R. A deterministic density algorithm for pairwise interaction coverage. In:Proc. of the IASTED Conf. on Software Engineering. 2004. 345-352.
    [6] Tai KC, Lei Y. A test generation strategy for pairwise testing. IEEE Trans. on Software Engineering, 2002,28(1):109-111.[doi:10.1109/32.979992]
    [7] Lei Y, Kacker R, Kuhn DR, Okun V, Lawrence J. IPOG:A general strategy for t-way software testing. In:Proc. of the 14th Annual IEEE Int'l Conf. and Workshops on the Engineering of Computer-Based Systems. IEEE, 2007. 549-556.[doi:10.1109/ECBS. 2007.47]
    [8] Nie CH, Wu HY, Liang YL. Search based combinatorial testing. In:Proc. of the 19th Asia-Pacific Software Engineering Conf. IEEE, 2012. 778-783.[doi:10.1109/APSEC.2012.16]
    [9] Willams AW. Determination of test configurations for pair-wise interaction coverage. In:Proc. of the IFIP TC6/WG6.113th Int'l Conf. on Testing Communicating Systems, Vol.48. Boston:Springer-Verlag, 2000. 59-74.[doi:10.1007/978-0-387-35516-0\_4]
    [10] Kobayashi N. Design and evaluation of automatic test generation strategies for functional testing of software[Ph.D. Thesis]. Osaka:Osaka University, 2002.
    [11] Gonzalez-Hernandez L. New bounds for mixed covering arrays of t-way testing with uniform strength. Information and Software Technology, 2015,59:17-32.[doi:10.1016/j.infsof.2014.10.009]
    [12] Liang YL, Nie CH. The optimization of configurable genetic algorithm for covering arrays generation. Chinese Journal of Computers, 2012,35(7):1522-1538(in Chinese with English abstract).
    [13] Cohen MB, Gibbons PB, Mugridge WB, Colbourn CJ, Collofello JS. Variable strength interaction testing of components. In:Proc. of the 27th Annual Int'l Conf. on Computer Software and Applications. IEEE, 2003:413-418.[doi:10.1109/CMPSAC.2003. 1245373]
    [14] Chen X, Gu Q, Li A, Chen D. Variable strength interaction testing with an ant colony system approach. In:Proc. of the 200916th Asia-Pacific Software Engineering Conf. IEEE, 2009. 160-167.[doi:10.1109/APSEC.2009.18]
    [15] Ahmed BS, Zamli KZ. A variable strength interaction test suites generation strategy using particle swarm optimization. Journal of Systems and Software, 2011,84(12):2171-2185.[doi:10.1016/j.jss.2011.06.004]
    [16] Lakhotia K, Harman M, Gross H. AUSTIN:An open source tool for search based software testing of C programs. Information and Software Technology, 2013,55(1):112-125.[doi:10.1016/j.infsof.2012.03.009]
    [17] Glover F. Tabu Search-Part 2. ORSA Journal on Computing, 1990,2(1):4-32.[doi:10.1287/ijoc.2.1.4]
    [18] Nurmela KJ. Upper bounds for covering arrays by tabu search. Discrete Applied Mathematics, 2004,138(1-2):143-152.[doi:10.1016/S0166-218X(03)00291-9]
    [19] Jia Y, Cohen MB, Harman M, Petke J. Learning combinatorial interaction testing strategies using hyperheuristic search. In:Proc. of the 2015 IEEE/ACM 37th IEEE Int'l Conf. on Software Engineering. IEEE, 2015. 540-550.[doi:10.1109/ICSE.2015.71]
    [20] Yu L, Duan F, Lei Y, Kacher RN, Kuhn DR. Constraint handling in combinatorial test generation using forbidden tuples. In:Proc. of the IEEE 8th Int'l Conf. on Software Testing. IEEE, 2015. 1-9.[doi:10.1109/ICSTW.2015.7107441]
    [21] Cohen MB, Dwyer MB, Shi J. Interaction testing of highly-configurable systems in the presence of constraints. In:Proc. of the 5th Int'l Symp. on Software Testing and Analysis. London:ACM Press, 2007. 129-139.[doi:10.1145/1273463.1273482]
    [22] Cohen MB, Dwyer MB, Shi J. Constructing interaction test suites for highly-configurable systems in the presence of constraints:A greedy approach. IEEE Trans. on Software Engineering, 2008,34(5):633-650.[doi:10.1109/TSE.2008.50]
    [23] Garvin BJ, Cohen MB, Dwyer MB. An improved meta-heuristic search for constrained interaction testing. In:Proc. of the 1st Int'l Symp. on Search Based Software Engineering. IEEE, 2009. 13-22.[doi:10.1109/SSBSE.2009.25]
    [24] Gonzalez-Hernandez L, Torres-Jimenez J, Rangel-Valdez N. MiTS in depth:An analysis of distinct tabu search configurations for constructing mixed covering arrays. Artificial Intelligence Evolutionary Computing and Metaheuristics, 2013,427:371-402.[doi:10.1007/978-3-642-29694-9_15]
    [25] Walker Ⅱ RA, Colbourn CJ. Tabu search for covering arrays using permutation vectors. Journal of Statistical Planning and Inference, 2009,139(1):69-80.[doi:10.1007/978-3-642-29694-9\_15]
    [26] Yamada A, Kitamura T, Artho C, Choi EH, Oiwa Y. Optimization of combinatorial testing by incremental SAT solving. In:Proc. of the 2015 IEEE 8th Int'l Conf. on Software Testing, Verification and Validation. 2015. 1-10.[doi:10.1109/ICST.2015.7102599]
    [27] Garvin BJ, Cohen MB, Dwyer MB. Evaluating improvements to a meta-heuristic search for constrained interaction testing. Empirical Software Engineering, 2011,16(1):61-102.[doi:10.1007/s10664-010-9135-7]
    [28] Cohen MB, Gibbons PB, Mugridge WB, Colbourn CJ. Constructing test suites for interaction testing. In:Proc. of the 25th Int'l Conf. on Software Engineering. IEEE, 2003. 38-48.[doi:10.1109/ICSE.2003.1201186]
    [29] Qi RZ, Wang ZJ, Huang YH, Li SY. Generating combinatorial test suite with spark based parallel approach. Chinese Journal of Computers, 2018,41(6):1284-1299(in Chinese with English abstract).
    [30] Zamli KZ, Alkazem BY, Kendall GA. Tabu search hyper-heuristic strategy for t-way test suite generation. Applied Soft Computing, 2016,44:57-74.[doi:10.1016/j.asoc.2016.03.021]
    [31] Wang Z, Qian J, Chen L, Xu BW. Generating variable strength combinatorial test suite with one-test-at-a-time strategy. Chinese Journal of Computers, 2012,35(12):2541-2552(in Chinese with English abstract).
    [32] Czerwonka J. Pairwise testing in real world:Practical extensions to test case scenarios. In:Proc. of the 24th Pacific Northwest Software Quality Conf. 2006. 419-430.
    [33] Himer AG, Jose TJ, Hernandez V, Nelson RV. A parallel algorithm for the verification of covering arrays. In:Proc. of the Int'l Conf. on Parallel and Distributed Processing Techniques and Applications (PDPTA). 2011. 879-885.
    [34] Lopez-Herrejon RE, Ferrer J, Chicano F. A parallel evolutionary algorithm for prioritized pairwise testing of software product lines. In:Proc. of the 16th Genetic and Evolutionary Computation Conf. Vancouver:ACM Press, 2014. 1255-1262.[doi:10.1145/2576768.2598305]
    [35] Geronimo LD, Ferrucci F, Murolo A. A parallel genetic algorithm based on hadoop mapReduce for the automatic generation of JUnit test Suites. In:Proc. of the 2012 IEEE 5th Int'l Conf. on Software Testing, Verification and Validation. IEEE, 2012. 785-793.[doi:10.1109/ICST.2012.177]
    附中文参考文献:
    [12] 梁亚澜,聂长海.覆盖表生成的遗传算法配置参数优化.计算机学报,2012,35(7):1522-1538.
    [29] 戚荣志,王志坚,黄宜华,李水艳.基于Spark的并行化组合测试用例集生成方法.计算机学报,2018,41(6):1284-1299.
    [31] 王子元,钱巨,陈林,徐宝文.基于One-test-at-a-time策略的可变力度组合测试用例生成方法.计算机学报,2012,35(12):2541-2552.
    Related
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

王燕,聂长海,钮鑫涛,吴化尧,徐家喜.覆盖表生成的禁忌搜索算法.软件学报,2018,29(12):3665-3691

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:March 28,2017
  • Revised:May 31,2017
  • Online: December 05,2018
You are the first2043811Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063