微信服务号

微信订阅号

2025年3月24日 8:33 星期一
首页 > 过刊浏览>2013年第24卷第4期 >663-674. DOI:10.3724/SP.J.1001.2013.04379
PDF HTML阅读 XML下载 导出引用 引用提醒
测试用例演化研究进展
作者:
基金项目:

国家自然科学基金(61003024, 61170067, 61170071)


Research Progress on Test Case Evolution
Author:
  • 摘要
    • | |
  • 访问统计
    • |
  • 参考文献 [61]
    • |
  • 相似文献 [20]
    • | | |
  • 文章评论
    摘要:

    由于功能增加、性能调优、错误修复等原因,软件常常动态演化.现有测试技术难以满足软件演化过程中变化的测试需求,因此需要系统的测试用例演化技术,以有效保障演化软件的质量.回顾测试用例演化技术的研究现状,分别对测试用例选择、测试用例修复和测试用例集扩增这3 部分内容进行了详细的比较和分析.最后提出测试用例演化技术领域存在的挑战和未来的研究方向.

    Abstract:

    Software is always dynamically evolving due to additional functionalities, performance tuning, bug fixing, and others. The existing testing techniques are difficult to satisfy the changing testing requirements. Hence, systematic techniques are needed of test case evolution to ensure the quality of evolving software. This paper surveys the test case evolution and compares and analyzes these techniques in detail, including test case selection, test case repairing and test suite augmentation. Finally, the challenges and research directions of test case evolution are proposed.

    参考文献
    [1] Bertolino A. Software testing research: Achievements, challenges, dreams. In: Briand LC, Wolf AL, eds. Proc. of the Future ofSoftware Engineering (FOSE 2007). Minneapolis: ACM Press, 2007. 85-103. [doi: 10.1002/stvr.430]
    [2] Yoo S, Harman M. Regression testing minimization, selection and prioritization: A survey. Journal of Software Testing,Verification and Reliability, 2012,22(2):67-120. [doi: 10.1002/stv.430]
    [3] Harrold MJ, Orso A. Retesting software during development and maintenance. In: Müller HA, ed. Proc. of the Frontiers ofSoftware Maintenance (FoSM 2008). Beijing: IEEE Computer Society Press, 2008. 99-108. [doi: 10.1109/FOSM.2008.4659253]
    [4] Rothermel G, Harrold MJ. A safe, efficient regression test selection technique. ACM Trans. on Software Engineering andMethodology, 1997,6(2):173-210. [doi: 10.1145/248233.248262]
    [5] Bible J, Rothermel G, Rosenblum DS. A comparative study of coarse- and fine-grained safe regression test-selection techniques.ACM TOSEM, 2001,10(2):149-183. [doi: 10.1145/367008.367015]
    [6] Rothermel G, Harrold MJ. A safe, efficient algorithm for regression test selection. In: Proc. of the Int’l Conf. on SoftwareMaintenance (ICSM 2003). Montreal: IEEE Computer Society Press, 1993. 358-367. [doi: 10.1109/ICSM.1993.366926]
    [7] Rothermel G, Harrold MJ. Selecting tests and identifying test coverage requirements for modified software. In: Proc. of the Int’lSymp. on Software Testing and Analysis (ISSTA’94). Seattle: ACM Press, 1994. 169-184. [doi: 10.1145/186258.187171]
    [8] Zhang C, Chen Z, Zhao Z, Yan S, Zhang J, Xu B. An improved regression test selection technique by clustering execution profiles.In: Proc. of the Int’l Conf. on Quality Software (QSIC 2010). Zhangjiajie: IEEE Reliability Society Press, 2010. 171-179. [doi: 10.1109/QSIC.2010.16]
    [9] Chen Z, Duan Y, Zhao Z, Xu B, Qian J. Using program slicing to improve the efficiency and effectiveness of cluster test selection.Int’l Journal of Software Engineering and Knowledge Engineering, 2011,21(6):759-777. [doi: 10.1142/S0218194011005487]
    [10] Fischer K. A test case selection method for the validation of software maintenance modifications. In: Proc. of the Int’l ComputerSoftware and Applications Conf. (COMPSAC’77). Chicago: IEEE Computer Society Press, 1977. 421-426.
    [11] Fischer K, Raji F, Chruscicki A. A methodology for retesting modified software. In: Proc. of the National TelecommunicationsConf. New Orleans: IEEE Computer Society Press, 1981. 1-6.
    [12] Harrold MJ, Soffa ML. Interprocedual data flow testing. In: Proc. of the 3rd ACM SIGSOFT Symp. on Software Testing, Analysis,and Verification (TAV3). ACM Press, 1989. 158-167. [doi: 10.1145/75308.75327]
    [13] Yau SS, Kishimoto Z. A method for revalidating modified programs in the maintenance phase. In: Proc. of the Int’l ComputerSoftware and Applications Conf. (COMPSAC’87). Tokyo: IEEE Computer Society Press, 1987. 272-277.
    [14] Agrawal H, Horgan JR, Krauser EW, London SA. Incremental regression testing. In: Proc. of the Int’l Conf. on SoftwareMaintenance (ICSM’93). Montréal: IEEE Computer Society Press, 1993. 348-357. [doi: 10.1109/ICSM.1993.366927]
    [15] Vokolos F, Pythia FP. A regression test selection tool based on text differencing. In: Proc. of the Int’l Conf. on Reliability Qualityand Safety of Software Intensive Systems. London: ROYAUME-UNI, 1997.
    [16] Vokolos F, Frankl P. Empirical evaluation of the textual differencing regression testing technique. In: Proc. of the IEEE Int’l Conf.on Software Maintenance (ICSM’98). Bethesda: IEEE Computer Society Press, 1998. 44-53. [doi: 10.1109/ICSM.1998.738488]
    [17] Bates S, Horwitz S. Incremental program testing using program dependence graphs. In: Proc. of the 20th ACM SIGPLAN-SIGACTSymp. on Principles of Programming Languages. Charleston: ACM Press, 1993. 384-396. [doi: 10.1145/158511.158694]
    [18] Benedusi P, Cmitile A, De Carlini U. Post-Maintenance testing based on path change analysis. In: Proc. of the Int’l Conf. onSoftware Maintenance (ICSM’88). Santa Fe: IEEE Computer Society Press, 1988. 352-361. [doi: 10.1109/ICSM.1988.10187]
    [19] Chen YF, Rosenblum D, Vo KP. Testtube: A system for selective regression testing. In: Proc. of the 16th Int’l Conf. on SoftwareEngineering (ICSE’94). Sorrento: ACM Press, 1994. 211-220. [doi: 10.1109/ICSE.1994.296780]
    [20] Laski J, SzermerW. Identification of program modifications and its applications in software maintenance. In: Proc. of the Int’l Conf.on Software Maintenance (ICSM’92). Chicago: IEEE Computer Society Press, 1992. 282-290. [doi: 10.1109/ICSM.1992.242533]
    [21] Leung HKN, White L. A study of integration testing and software regression at the integration level. In: Proc. of the Int’l Conf. onSoftware Maintenance (ICSM’90). San Diego: IEEE Computer Society Press, 1990. 290-301. [doi: 10.1109/ICSM.1990.131377]
    [22] White L, Robinson B. Industrial real-time regression testing and analysis using firewalls. In: Proc. of the Int’l Conf. on SoftwareMaintenance (ICSM 2004). Chicago: IEEE Computer Society Press, 2004. 18-27. [doi: 10.1109/ICSM.2004.1357786]
    [23] White L, Almezen H, Sastry S. Firewall regression testing of GUI sequences and their interactions. In: Proc. of the Int’l Conf. onSoftware Maintenance (ICSM 2003). Amsterdam: IEEE Computer Society Press, 2003. 398-409. [doi: 10.1109/ICSM.2003.1235450]
    [24] Zheng J, Robinson B, Williams L, Smiley K. Applying regression test selection for COTS-based applications. In: Proc. of the Int’lConf. on Software Engineering (ICSE 2006). Shanghai: ACM Press, 2006. 512-522. [doi: 10.1145/1134357]
    [25] Briand LC, Labiche Y, Buist K, Soccar G. Automating impact analysis and regression test selection based on UML designs. In:Proc. of the Int’l Conf. on Software Maintenance (ICSM 2002). Montréal: IEEE Computer Society Press, 2002. 252-261.[Regression testing UML designs]
    [26] Briand LC, Labiche Y, He S. Automating regression test selection based on UML designs. Information and Software Technology,2009,51(1):16-30. [doi: 10.1016/j.infsof.2008.09.010]
    [27] Pilskalns O, Uyan G, Andrews A. Regression testing UML designs. In: Proc. of the Int’l Conf. on Software Maintenance (ICSM2006). Dublin: IEEE Computer Society Press, 2006. 254-264. [doi: 10.1109/ICSM.2006.53]
    [28] Engstrom E, Runeson P, Skoglund M. A systematic review on regression test selection techniques. Information and SoftwareTechnology, 2010,52(1):14-30. [doi: 10.1016/j.infsof.2009.07.001]
    [29] Zhang XF, Chen L, Xu BW, Nie CH. Survey of test suite reduction problem. Journal of Frontiers of Computer Science andTechnology, 2008,2(3):235-247 (in Chinese with English abstract).
    [30] Qu B, Nie CH, Xu BW. Survey of test case prioritization for regression testing. Journal of Frontiers of Computer Science andTechnology, 2009,3(3):225-233 (in Chinese with English abstract).
    [31] Memon A, Soffa ML. Regression testing of GUIs. In: Proc. of the 9th European Software Engineering Conf. on Held Jointly withthe 11th ACM SIGSOFT Int’l Symp. on Foundations of Software Engineering (ESEC/SIGSOFT FSE 2003). New York: ACMPress, 2003. 118-127. [doi: 10.1145/940071.940088]
    [32] Stack Overflow. Program evolution and broken tests. http://stackoverflow.com/questions/2054171/
    [33] C2 Wiki. Deleting broken unit tests. http://c2.com/cgi-bin/wiki?DeletingBrokenUnitTests
    [34] Memon A. Automatically repairing event sequence-based GUI test suites for regression testing. ACM Trans. on SoftwareEngineering and Methodology, 2008,18(2):Article 4. [doi: 10.1145/1416563.1416564]
    [35] Xie Q, Grechanik M, Fu C. Rest: A tool for reducing effort in script-based testing. In: Proc. of the Int’l Conf. on SoftwareMaintenance (ICSM 2008). Beijing: IEEE Computer Society Press, 2008. 468-469. [doi: 10.1109/ICSM.2008.4658108]
    [36] Grechanik M, Xie Q, Fu C. Maintaining and evolving GUI-directed test scripts. In: Proc. of the Int’l Conf. on Software Engineering(ICSE 2006). Vancouver: ACM Press, 2006. 408-418. [doi: 10.1109/ICSE.2009.5070540]
    [37] Huang S, Cohen MB, Memon AM. Repairing GUI test suites using a genetic algorithm. In: Proc. of the 1st Int’l Conf. on SoftwareTesting, Verification and Validation (ICST 2010). Paris: IEEE Computer Society Press, 2010. 245-254. [doi: 10.1109/ICST.2010.39]
    [38] Daniel B, Luo Q, Mirzaaghaei M, Dig D, Marinov D, Pezzè M. Automated GUI refactoring and test script repair. In: Proc. of the1st Int’l Workshop on End-to-End Test Script Engineering (ETSE 2011). Toronto: ACM Press, 2011. [doi: 10.1145/2002931.2002937]
    [39] Harman M, Alshahwan N. Automated session data repair for Web application regression testing. In: Proc. of the 1st Int’l Conf. onSoftware Testing, Verification and Validation (ICST 2008). Lillehammer: IEEE Computer Society Press, 2008. 298-307. [doi: 10.1109/ICST.2008.56]
    [40] Choudhary SR, Zhao D, Versee H, Orso A. WATER: Web application TEst repair. In: Proc. of the 1st Int’l Workshop onEnd-to-End Test Script Engineering (ETSE 2011). Toronto: ACM Press, 2011. [doi: 10.1145/2002931.2002935]
    [41] Mirzaaghaei M, Pastore F, Pezzè M. Automatically repairing test cases for evolving method declarations. In: Proc. of the Int’l Conf.on Software Maintenance (ICSM 2010). Timisoara: IEEE Computer Society Press, 2010. 1-5. [doi: 10.1109/ICSM.2010.5609549]
    [42] Daniel B, Jagannath V, Dig D, Marinov D. ReAssert: Suggesting repairs for broken unit tests. In: Proc. of the 24th IEEE/ACM Int’lConf. on Automated Software Engineering (ASE 2009). Auckland: IEEE Computer Society Press, 2009. 433-444. [doi: 10.1109/ASE.2009.17]
    [43] Daniel B, Dig D, Gvero T, Jagannath V, Jiaa J, Mitchell D, Nogiec J, Tan SH, Marinov D. ReAssert: A tool for repairing brokenunit tests. In: Proc. of the Int’l Conf. on Software Engineering (ICSE 2011). Hawaii: ACM Press, 2011. 1010-1012. [doi: 10.1145/1985793.1985978]
    [44] Daniel B, Gvero T, Marinov D. On test repair using symbolic execution. In: Proc. of the Int’l Symp. on Software Testing andAnalysis (ISSTA 2010). Trento: ACM Press, 2010. 207-218. [doi: 10.1145/1831708.1831734]
    [45] Visser W, Pasareanu C, Khurshid S. Test input generation with Java pathfinder. In: Proc. of the Int’l Symp. on Software Testingand Analysis (ISSTA 2004). Boston: ACM Press, 2004. 97-107. [doi: 10.1145/1007512.1007526]
    [46] Sen K, Marinov D, and Agha G. CUTE: A concolic unit testing engine for C. In: Proc. of the 10th European Software EngineeringConf. on Held Jointly with the 13th ACM SIGSOFT Int’l Symp. on Foundations of Software Engineering (ESEC/FSE 2005).Lisbon: ACM Press, 2005. 263-272. [doi: 10.1145/1081706.1081750]
    [47] Chen TY, Merkel R. Quasi-Random testing. IEEE Trans. on Reliability, 2007,56(3):562-568. [doi: 10.1109/TR.2007.903293]
    [48] Harder M, Mellen J, Ernst MD. Improving test suites via operational abstraction. In: Proc. of the Int’l Conf. on SoftwareEngineering (ICSE 2003). Hilton Portland: ACM Press, 2003. 60-71. [doi: 10.1109/ICSE.2003.1201188]
    [49] Santelices RA, Chittimalli PK, Apiwattanapong T, Orso A, Harrold MJ. Test-Suite augmentation for evolving software. In: Proc. ofthe 23th IEEE/ACM Int’l Conf. on Automated Software Engineering (ASE 2008). L’Aquila: IEEE Computer Society Press, 2008.218-227. [doi: 10.1109/ASE.2008.32]
    [50] Santelices RA, Harrold MJ. Automated scalable test-suite augmentation for evolving software. In: Proc. of the Int’l Conf. onSoftware Engineering-Companion (ICSE Companion 2009). Vancouver: ACM Press, 2009. 379-382. [doi: 10.1109/ICSECOMPANION.2009.5071026]
    [51] Santelices RA, Harrold MJ. Applying aggressive propagation-based strategies for testing changes. In: Proc. of the Int’l Conf. ofSoftware Testing, Verification and Validation (ICST 2011). Berlin: IEEE Computer Society Press, 2011. 11-20. [doi: 10.1109/ICST.2011.46]
    [52] Xu Z, Rothermel G. Directed test suite augmentation. In: Proc. of the Asia-Pacific Software Engineering Conf. (APSEC 2009).Penang: ACM Press, 2009. 406-413.
    [53] Xu ZH, Cohen MB, Rothermel G. Factors affecting the use of genetic algorithms in test suite augmentation. In: Proc. of the 12thAnnual Conf. on Genetic and Evolutionary Computation (GECCO 2010). Portland: ACM Press, 2010. 1365-1372. [doi: 10.1145/1830483.1830734]
    [54] Xu Z, Kim Y, Kim M, Rothermel G, Cohen MB. Directed test suite augmentation: Techniques and tradeoffs. In: Proc. of the ACMSIGSOFT Int’l Symp. on Foundations of Software Engineering (SIGSOFT FSE 2010). Santa Fe: ACM Press, 2010. 257-266. [doi:10.1145/1882291.1882330]
    [55] Taneja K, Xie T, Tillmann N, Halleux J, Schulte W. Guided path exploration for regression test generation. In: Proc. of the Int’lConf. on Software Engineering (ICSE 2009). Vancouver: ACM Press, 2009. 311-314. [doi: 10.1109/ICSE-COMPANION.2009.5071009]
    [56] Qi D, Roychoudhury A, Liang Z. Test generation to expose changes in evolving programs. In: Proc. of the 25th IEEE/ACM Int’lConf. on Automated Software Engineering (ASE 2010). UniMail: IEEE Computer Society Press, 2010. 397-406. [doi: 10.1145/1858996.1859083]
    [57] Watt SK. Difficult dialogues, privilege, and social justice: Uses of the privileged identity exploration (PIE) model in student affairspractice. College Student Affairs Journal, 2007,26(2):114-126.
    [58] Apiwattanapong T, Santelices R, Chittimalli PK, Orso A, Harrold MJ. Matrix: Maintenance-Oriented testing requirement identifierand examiner. In: Proc. of the Testing: Academic and Industrial Conf.—Practice and Research Techniques (TAIC-PART 2006).Windsor: Swell Press, 2006. 137-146.
    [59] Jiang B, Tse TH, Grieskamp W, Kicillof N, Cao Y, Li X. Regression testing process improvement for specification evolution ofreal-world protocol software. In: Proc. of the Int’l Conf. on Quality Software (QSIC 2010). Zhangjiajie: IEEE Reliability SocietyPress, 2010. 62-71. [doi: 10.1109/QSIC.2010.55]
    [60] Cadar C, Ganesh V, Pawlowski PM, Dill DL, Engler DR. Exe: Automatically enerating inputs of death. In: Proc. of the 13th ACMConf. on Computer and Communications Security (CCS 2006). Alexandria: ACM Press, 2006. 322-335. [doi: 10.1145/1180405.1180445]
    [61] McMinn P. Search-Based software test data generation: A survey. Software Testing, Verification and Reliability, 2004,14(2):105-156. [doi: 10.1002/stvr.294]
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

张智轶,陈振宇,徐宝文,杨瑞.测试用例演化研究进展.软件学报,2013,24(4):663-674

复制
分享
文章指标
  • 点击次数:9429
  • 下载次数: 10436
  • HTML阅读次数: 0
  • 引用次数: 0
历史
  • 收稿日期:2012-07-01
  • 最后修改日期:2012-09-29
  • 在线发布日期: 2013-03-26
文章二维码
友情链接:中国科学院软件研究所中国计算机学会中国科学院国家自然科学基金委员会CCF数字图书馆Int'l J of Softw Info计算机系统应用
您是第19727256位访问者
版权所有：中国科学院软件研究所 京ICP备05046678号-3
地址：北京市海淀区中关村南四街4号,邮政编码：100190
电话：010-62562563 传真：010-62562533 Email：jos@iscas.ac.cn
技术支持：北京勤云科技发展有限公司

京公网安备 11040202500063号