微信服务号

微信订阅号

2025年3月26日 7:37 星期三
首页 > 过刊浏览>2009年第20卷第6期 >1425-1443
PDF HTML阅读 XML下载 导出引用 引用提醒
软件错误注入测试技术研究
作者:
基金项目:

Supported by the National Research Project Foundation of China under Grant No.513150601 (国家部委项目)

  • 摘要
    • | |
  • 访问统计
    • |
  • 参考文献 [74]
    • |
  • 相似文献
    • |
  • 引证文献
    • | |
  • 文章评论
    摘要:

    软件错误注入测试(software fault injection testing,简称SFIT)技术经过近30年的发展,一直是软件测试领域最活跃的研究内容之一.作为一种非传统的测试技术,在提高软件质量、减少软件危害及改进软件开发过程等方面起着重要作用.对软件错误注入测试的研究现状及动态进行了调研,对该领域相关技术进行了归类及介绍,并对当前较为有效的测试框架和原型工具进行了总结,同时介绍了正在研发的基于SFIT技术的构件安全性测试系统CSTS.在认真分析现有技术的基础上,总结了当前软件错误注入测试存在的问

    Abstract:

    The software fault injection testing (SFIT) technique has been developed for thirty years. It is one of the most active parts in software testing research. As a non-traditional testing technique, it plays a very important role in enhancing software quality, eliminating software failures and improving the process of software development. A detailed review of the research on SFIT is presented based on the survey and classification of the current SFIT techniques. Then, some important testing frameworks and tools that are effective at present are also discussed. Meanwhile, a brief description of the testing system CSTS (Component Security Testing System) is provided as well. Based on the precise investigation on SFIT, the issues and challenges of SFIT are pointed out and the future development trend for SFIT is proposed.

    参考文献
    [1] Jean A, Martine A, Louis A, Yves C, Jean-Charles F, Jean-Claude L, Eliane M, David P. Fault injection for dependability validation: A methodology and some applications. IEEE Trans. on Software Engineering, 1990,16(2):166?182.
    [2] Carreira J, Madeira H, Silva JG. Xception: A technique for the experimental evaluation of dependability in modern computers. IEEE Trans. on Software Engineering, 1998,24(2):125?136.
    [3] Ghani AK, Nasser AK, Jacob AA. FERRARI: A flexible software-based fault and error injection system. IEEE Trans. on Computers, 1995,44(2):248?260.
    [4] Goswami KK. DEPEND: A simulation-based environment for system level dependability analysis. IEEE Trans. on Computers, 1997,46(1):60?74.
    [5] Looker N, Munro M, Xu J. Simulating errors in Web services. Int’l Journal of Simulation Systems, Science, 2004,5(5):29?37.
    [6] DeMillo RA, Lipton RJ, Sayward FG. Hints on test data selection: Help for the practicing programmer. IEEE Computer, 1978,11(4):34?41.
    [7] Voas J. Fault injection for the masses. IEEE Computer, 1997,30:129?130.
    [8] Hsueh MC, Tsai TK, Lyer RK. Fault injection techniques and tools. IEEE Computer, 1997,30(4):75?82.
    [9] Bieman JM, Dreilinger D, Lijun L. Using fault injection to increase software test coverage. In: Proc. of the 7th Int’l Symp. on Software Reliability Engineering (ISSRE’96). Washington: IEEE Computer Society, 1996. 166?174.
    [10] Delamaro ME, Maidonado JC, Mathur AP. Interface mutation: An approach for integration testing. IEEE Trans. on Software Engineering, 2001,27(3):228?247.
    [11] Barbosa R, Silva N, Duraes J, Madeira H. Verification and validation of (real time) COTS products using fault injection techniques. In: Lewis G, Schuster S, Smith D, eds. Proc. of the 6th Int’l IEEE Conf. on Commercial-off-the-Shelf (COTS)-Based Software Systems (ICCBSS 2007). Los Alamitos: IEEE CS, 2007. 233?242.
    [12] Regina LOM, Eliane M, Naaliel Vicente M. Fault injection approach based on dependence analysis. In: Proc. of the 29th Annual Int’l Computer Software and Applications Conf. (COMPSAC 2005). Washington: IEEE Computer Society, 2005. 181?188.
    [13] Avresky D, Arlat J, Laprie JC, Crouzet Y. Fault injection for formal testing of fault tolerance. IEEE Trans. on Reliability, 1996,45(3):443?455.
    [14] Huang YW, Huang SK, Lin TP, Tsai CH. Web application security assessment by fault injection and behavior monitoring. In: Chen YFR, Kovacs L, Lawrence S, eds. Proc. of the 12th Int’l Conf. on World Wide Web (WWW 2003). New York: ACM, 2003. 148?159.
    [15] Wang JY, Sun JC, Yang XZ. An experimental strategy of fault injection for the whole validation of fault-tolerant computer systems. Journal of Computer Research & Development, 2001,38(1):61?67 (in Chinese with English abstract).
    [16] Du W, P.Mathur A. Testing for software vulnerability using environment perturbation. Quality and Reliability Engineering Int’l, 2002,18(3):261?272.
    [17] Jean A, Yves C, Johan K, Peter F, Emmerich FG, nther HL. Comparison of physical and software-implemented fault injection techniques. IEEE Trans. on Computers, 2003,52(9):1115?1133.
    [18] Lopez-Ongil C, Entrena L, Garcia-Valderas M, Portela MAPM, Aguirre MAAAMA, Tombs JATJ, Baena VABV, Munoz FAMF. A unified environment for fault injection at any design level based on emulation. IEEE Trans. on Nuclear Science, 2007,54(4): 946?950.
    [19] Eliane M, Cec, lia MFR, Nelson GML. Jaca: A reflective fault injection tool based on patterns. In: Proc. of the 2002 Int’l Conf. on Dependable Systems and Networks (DSN2002). Los Alamitos: IEEE Computer Society, 2002. 483?487.
    [20] Arlat J, Crouzet Y, Laprie J. Fault injection for dependability validation of fault-tolerant computing systems. In: Proc. of the 25th Int’l Symp. on Fault-Tolerant Computing, Highlights from 25 Years. Los Alamitos: IEEE Computer Society, 1995. 400?407.
    [21] Gunneflo U, Karlsson J, Torin J. Evaluation of error detection schemes using fault injection by heavy-ion radiation. In: Proc. of the 19th Int’l Symp. on Fault-Tolerant Computing (FTCS-19). Washington: IEEE Computer Society, 1989. 340?347.
    [22] Karlsson J, Arlat J, Leber G. Application of three physical fault injection techniques to the experimental assessment of the MARS architecture. In: Proc. of the 5th Ann. IEEE Int’l Working Conf. on Dependable Computing for Critical Applications. Los Alamitos: IEEE CS, 1995. 150?161.
    [23] Voas JM, Miller KW, Payne JE. PISCES: A tool for predicting software testability. In: Miller KW, ed. Proc. of the 2nd Symp. on Assessment of Quality Software Development Tools. Washington: IEEE CS, 1992. 297?309.
    [24] Han S, Shin KG, Rosenberg HA. DOCTOR: An integrated software fault injection environment for distributed real-time systems. In: Proc. of the 1995 IEEE Int’l Computer Performance and Dependability Symp. (IPDS’95). Los Alamitos: IEEE CS, 1995. 204?213.
    [25] Liu MH, Gao YF, Shan JH, Liu JH, Zhang L, Sun JS. An approach to test data generation for killing multiple mutants. In: You-Feng G, ed. Proc. of the 22nd IEEE Int’l Conf. on Software Maintenance (ICSM 2006). Los Alamitos: IEEE CS, 2006. 113?122.
    [26] Andrews JH, Brand LC, Labiche Y. Is mutation an appropriate tool for testing experiments? In: Proc. of the 27th Int’l Conf. on Software Engineering (ICSE 2005). Los Alamitos: IEEE Computer Society Press, 2005. 402?411.
    [27] Eliane M, Amanda CAR. A fault injection approach based on reflective programming. In: Proc. of the 2000 Int’l Conf. on Dependable Systems and Networks (DSN 2000). Los Alamitos: IEEE Computer Society, 2000. 407?416.
    [28] Ghosh AK, Schmid M, Shah V. Testing the robustness of windows NT software. In: Proc. of the 9th Int’l Symp. on Software Reliability Engineering (ISSRE’98). Los Alamitos: IEEE CS, 1998. 231?235.
    [29] Schmid M, Ghosh A, Hill F. Techniques for evaluating the robustness of Windows NT software. In: Proc. of the 2000 DARPA Information Survivability Conf. and Exposition (DISCEX 2000). Los Alamitos: IEEE CS, 2000. 347?360.
    [30] Thompson HH, Whittaker JA, Mottay FE. Software security vulnerability testing in hostile environments. In: Haddad H, Papadopoulos G, eds. Proc. of the 2002 ACM Symp. on Applied computing. New York: ACM, 2002. 260?264.
    [31] Barton PM, Louis F, Bryan S. An empirical study of the reliability of UNIX utilities. Communications of the ACM, 1990,33(12):32?44.
    [32] Miller BP, Koski D, Lee CP, Maganty V, Murthy R, Natarajan A, Steidl J. Fuzz Revisited: A Re-Examination of the Reliability of UNIX Utilities and Services, Vol.1, Wisconsin: Computer Sciences Department, University of Wisconsin, 2000. 1?23.
    [33] Jiang Y, Hou SS, Shan JH, Zhang L, Xie B. Contract-based mutation for testing components. In: Proc. of the 21st Int’l Conf. on Software Maintenance (ICSM 2005). Los Alamitos: IEEE Computer Society, 2005. 483?492.
    [34] Jian-jun Y, Wei-dong C, Cheng-qing Y, Yun-he P. Interface mutation for component. Journal of Zhejiang University (Engineering Science), 2003,37(2):129?133 (in Chinese with English abstract).
    [35] Voas JM. PIE: A dynamic failure-based technique. IEEE Trans. on Software Engineering, 1992,18(8):717?727.
    [36] Voas J, Ghosh A, McGraw G, Charron FACF, Miller KAMK. Defining an adaptive software security metric from a dynamic software failure tolerance measure. In: Ghosh A, ed. Proc. of the 11th Annual Conf. on Computer Assurance, Systems Integrity. Software Safety. Process Security (COMPASS’96). Piscataway: IEEE, 1996. 250?263.
    [37] Ghosh AK, Schmid M, Hill F. Wrapping windows NT software for robustness. In: Pomeranz I, Sanders WH, eds. Proc. of the 29th Annual Int’l Symp. on Fault-Tolerant Computing. Los Alamitos: IEEE Computer Society, 1999. 344?347.
    [38] Koopman P, Sung J, Dingman C, Siewiorek D, Marz T. Comparing operating systems using robustness benchmarks. In: Proc. of the 16th Symp. on Reliable Distributed Systems. Los Alamitos: IEEE Computer Society, 1997. 72?79.
    [39] Ai-Guo L, Bing-Rong H, Si W. An approach for identifying software vulnerabilities based on error propagation analysis. Chinese Journal of Computers, 2007,30(11):1910?1921 (in Chinese with English abstract).
    [40] Arkin B, Stender S, McGraw G. Software Penetration Testing. IEEE Security & Privacy, 2005,532?535.
    [41] Howden WE. Weak mutation testing and completeness of test sets. IEEE Trans. on Software Engineering, 1982,8(4):371?379.
    [42] Offutt AJ, Jie P. Detecting equivalent mutants and the feasible path problem. In: Ghosh A, ed. Proc. of the 11th Annual Conf. on Computer Assurance ‘Systems Integrity. Software Safety. Process Security’(COMPASS 1996). Piscataway: IEEE, 1996. 224?236.
    [43] Offutt AJ, Lee S D. An empirical evaluation of weak mutation. IEEE Trans. on Software Engineering, 1994,20(5):337?344.
    [44] Offutt J, Lee A, Rothermel G, Untch R, Zapf C. An experimental determination of sufficient mutation operators. ACM Trans. on Software Engineering and Methodology, 1996,5(2):99?118.
    [45] DeMillo RA, Guindi DS, McCracken WM, Offutt AJ, King KN. An extended overview of the Mothra software testing environment. In: Proc. of the 2nd Workshop on Software Testing, Verification, and Analysis. Los Alamitos: IEEE Computer Society, 1988. 142?151.
    [46] DeMilli RA, Offutt AJ. Constraint-Based automatic test data generation. IEEE Trans. on Software Engineering, 1991,17(9): 900?910.
    [47] Offutt AJ, Zhenyi J, Jie P. The dynamic domain reduction procedure for test data generation. Software Practice and Experience, 1999,29(2):167?193.
    [48] Ghosh S, Kelly JL. Bytecode fault injection for Java software. The Journal of Systems and Software, 2008,47(2):1?10.
    [49] Whittaker JA, Mottay FE, EI-Far LK. Testing exception and error cases using runtime fault injection. Florida Computer Science Technology Laboratory, 2001. 1?9.
    [50] Tsai TK, Iyer RK, Jewitt D. An approach towards benchmarking of fault-tolerant commercial systems. In: Proc. of the Annual Symp. on Fault Tolerant Computing. Los Alamitos: IEEE Computer Society, 1996. 314?323.
    [51] Barton JH, Czeck EW, Segall ZZ, Siewiorek DP. Fault injection experiments using FIAT. IEEE Trans. on Computers, 1990,39(4): 575?582.
    [52] Kropp NP. Automatic robustness testing of off-the-shelf software components. Pennsylvania: Institute for Complex Engineered Systems, Carnegie Mellon University, 1998. 1?34.
    [53] Ghosh AK, Schmid M. An approach to testing COTS software for robustness to operating system exceptions and errors. In: Schmid M, ed. Proc. of the 10th Int’l Symp. on Software Reliability Engineering (ISSRE’99). Los Alamitos: IEEE Computer Society, 1999. 166?174.
    [54] Chen J, Lu Y, Xie X, Zhang W. Testing approach of component security based on dynamic monitoring. In: J Ni, ed. Proc. of the 2nd Int’l Multi-Symp. on Computer and Computational Sciences (IMSCCS 2007). IEEE Computer Society, 2007. 381?386.
    [55] Chen J, Lu Y, Xie X. Testing approach of component security based on fault injection. In: Niu XM, Xu XF, eds. Proc. of the 2007 Int’l Conf. on Computational Intelligence and Security (CIS 2007): IEEE Computer Society, 2007. 763?767.
    [56] Martin S, Christof F. Robustness and Security Hardening of COTS Software Libraries. In: Anderson T, Kaaniche M, eds. Proc. of the 37th Annual IEEE/IFIP Int’l Conf. on Dependable Systems and Networks (DSN 2007). Los Alamitos: IEEE Computer Society, 2007. 61?71.
    [57] Chen J, Lu Y, Xie X, You L, Wen X. Design and implementation of an automatic testing platform for component security. Computer Science, 2008,35(12):229?233 (in Chinese with English abstract).
    [58] Martin H, Arshad J, Neeraj S. EPIC: Profiling the propagation and effect of data errors in software. IEEE Trans. on Computers, 2004,53(5):512?530.
    [59] Voas J, Charron F, McGraw G, Miller K, Friedman M. Predicting how badly “Good” software can behave. IEEE Software, 1997 14(4):73?83.
    [60] Dawson S, Jahanian F, Mitton T. ORCHESTRA: A probing and fault injection environment for testing protocol implementations. In: Proc. of the 2nd Int’l Computer Performance and Dependability Symp. (IPDS 1996). Los Alamitos: IEEE Computer Society, 1996. 56?56.
    [61] Looker N, Munro M, Xu J. A comparison of network level fault injection with code insertion. In: Proc. of the 29th IEEE Int’l Computer Software and Applications Conf. Washington: IEEE Computer Society, 2005. 479?484.
    [62] Duraes JA, Madeira HS. Emulation of Software Faults: A field data study and a practical approach. IEEE Trans. on Software Engineering, 2006,32(11):849?867.
    [63] Aidemark J, Vinter J, Folkesson P, Karlsson JAKJ. GOOFI: Generic object-oriented fault injection tool. In: Vinter J, ed. Proc. of the Int’l Conf. on Dependable Systems and Networks. DSN 2001. Los Alamitos: IEEE Computer Society, 2001. 83?88.
    [64] Wei-lun K, Ravishankar K I, Dong T. FINE: A fault injection and monitoring environment for tracing the UNIX system behavior under faults. IEEE Trans. on Software Engineering, 1993,19(11):1105?1118.
    [65] Antonio D, Jos FM, Lourdes L, Ana BG, Luis R. Exhaustif: A fault injection tool for distributed heterogeneous embedded systems. In: Berqia A, ed. Proc. of the 2007 Euro American Conf. on Telematics and information systems. New York: ACM, 2007. 1?8.
    [66] Baldini A, Benso A, Chiusano S, Prinetto P. “BOND”: An interposition agents based fault injectorfor Windows NT. In: Proc. of the 2000 IEEE Int’l Symp. on Defect and Fault Tolerance in VLSI Systems. Los Alamitos: IEEE CS, 2000. 387?395.
    [67] Voas J, McGraw G. Software fault Injection: Inoculating Programs Against Errors. New York: John Wiley and Sons, 1997. 191?197.
    [68] Acantilado NJP, Acantilado CP. SIMPLE: A prototype software fault-injection tool [MS. Thesis]. Monterey: Naval Postgraduate School, 2002.
    [69] Qing W, Shu-Jian W, Ming-Shu L. Research on software defect prediction. Journal of Software, 2008,19(7):1565?1580 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/19/1565.htm 附中文参考文献:
    [15] 王建莹,孙峻朝,扬孝宗.一种用于容错计算机系统整体验证的故障注入试验策略.计算机研究与发展,2001,38(1):61?67.
    [34] 杨建军,陈卫东,叶澄清,潘云鹤.面向组件的接口变异测试方法.浙江大学学报(工学版),2003,37(2):129?133.
    [39] 李爱国,洪炳镕,王司.基于错误传播分析的软件脆弱点识别方法研究.计算机学报,2007,30(11):1910?1921.
    [57] 陈锦富,卢炎生,谢晓东,游亮,温贤馨.一个组件安全自动化测试平台的设计与实现.计算机科学,2008,35(12):229?233.
    [69] 王青,伍书剑,李明树.软件缺陷预测技术研究.软件学报,2008,19(7):1565?1580. http://www.jos.org.cn/1000-9825/19/1565.htm
    相似文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

陈锦富,卢炎生,谢晓东.软件错误注入测试技术研究.软件学报,2009,20(6):1425-1443

复制
分享
文章指标
  • 点击次数:10319
  • 下载次数: 19774
  • HTML阅读次数: 0
  • 引用次数: 0
历史
  • 收稿日期:2008-06-22
  • 最后修改日期:2008-11-17
文章二维码
友情链接:中国科学院软件研究所中国计算机学会中国科学院国家自然科学基金委员会CCF数字图书馆Int'l J of Softw Info计算机系统应用
您是第19733020位访问者
版权所有：中国科学院软件研究所 京ICP备05046678号-3
地址：北京市海淀区中关村南四街4号,邮政编码：100190
电话：010-62562563 传真：010-62562533 Email：jos@iscas.ac.cn
技术支持：北京勤云科技发展有限公司

京公网安备 11040202500063号