基于规则与学习的变异技术对比研究
CSTR:
作者:
作者单位:

作者简介:

贡志豪(2000—),男,博士生,CCF学生会员,主要研究领域为软件工程;陈逸洲(1996—),男,博士生,CCF学生会员,主要研究领域为软件测试,区块链安全和深度学习;陈俊洁(1992—),男,博士,副教授,CCF高级会员,主要研究领域为软件分析与测试;郝丹(1979—),女,博士,教授,博士生导师,CCF杰出会员,主要研究领域为软件测试.

通讯作者:

郝丹,E-mail:haodan@pku.edu.cn

中图分类号:

基金项目:

国家重点研发计划(2022YFB4501902);国家自然科学基金(62232001)


Comparison Research on Rule-based and Learning-based Mutation Techniques
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    变异测试是一种有效的软件测试技术, 通过生成变异体来模拟软件缺陷, 帮助提升现有测试套件的缺陷检测能力. 变异体的质量对于变异测试的有效性具有显著影响. 传统的变异测试方法通常采用人工设计的基于语法规则的变异算子生成变异体, 并已取得一定的研究成果. 近年来, 许多研究开始结合深度学习技术, 通过学习开源项目历史代码生成变异体. 目前, 该新方法在变异体生成方面取得了初步的成果. 基于语法规则和基于学习的两种变异技术, 其机理不同, 但其目标均是通过生成变异体来提高测试套件的缺陷检测能力, 因此, 全面比较这两种变异技术对于变异测试及其下游任务至关重要. 针对这一问题, 设计实现了一项针对基于语法规则和基于学习的变异技术的实证研究, 旨在了解不同机理的变异技术在变异测试任务上的性能以及生成的变异体在程序语义上的差异性. 具体地, 以Defect4J v1.2.0数据集为实验对象, 比较以MAJOR和PIT为代表的基于语法规则的变异技术和以DeepMutation、μBERT和LEAM为代表的基于深度学习的变异技术. 实验结果表明: 基于规则与学习的变异技术均可有效支持变异测试实践, 但MAJOR的测试效果最优, 能够检测85.4%的真实缺陷. 在语义表示上, MAJOR具有最强的语义代表能力, 基于其构造的测试套件能够杀死其余变异技术生成的超过95%占比的变异体. 在缺陷表征上, 两类技术均具有独特性.

    Abstract:

    Mutation testing is an effective software testing technique. It helps improve the defect detection capability of an existing test suite by generating mutants that simulate software defects. The quality of mutants has a significant impact on the effectiveness of mutation testing. The traditional mutation testing approach typically employs manually designed syntactic rule-based mutation operators to generate mutants, and has achieved some academic success. In recent years, many studies have started to incorporate deep learning techniques to generate mutants by learning historical code from open source projects. This new approach has achieved preliminary progress in mutant generation. A comprehensive comparison of the two mutation techniques, i.e. rule-based and learning-based, which have different mechanisms but both aim to improve the defect detection capability of the test suite by mutation, is crucial for mutation testing and its downstream tasks. To handle the problem, this study designs and implements an empirical study of rule-based and learning-based mutation techniques, aiming to understand the performance of mutation techniques with different mechanisms on the task of mutation testing, as well as the variability of the generated mutants in terms of program semantics. Specifically, this study uses the Defect4J v1.2.0 dataset to compare the syntactic rule-based mutation techniques represented by MAJOR and PIT with the deep learning-based mutation techniques represented by DeepMutation, μBERT, and LEAM. The experimental results show that both rule-based and learning-based mutation techniques can effectively support mutation testing practices, but MAJOR has the best testing performance and is able to detect 85.4% of real defects. In terms of semantic representation, MAJOR has the strongest semantic representation capability, and its constructed test suite is able to kill more than 95% of the mutants generated by other mutation techniques. In terms of defect representation, both types of techniques are unique.

    参考文献
    相似文献
    引证文献
引用本文

贡志豪,陈逸洲,陈俊洁,郝丹.基于规则与学习的变异技术对比研究.软件学报,2024,35(7):3093-3114

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2023-09-11
  • 最后修改日期:2023-10-30
  • 录用日期:
  • 在线发布日期: 2024-01-05
  • 出版日期: 2024-07-06
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号