语义可感知的灰盒编译器模糊测试
作者:
通讯作者:

蒋炎岩,E-mail:jyy@nju.edu.cn

中图分类号:

TP311

基金项目:

国家重点研发计划(2022YFB4501801); 国家自然科学基金(62025202, 62272218); 江苏省前沿引领技术基础研究专项(BK20202001)


Semantic Aware Greybox Compiler Fuzz Testing
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [57]
  • |
  • 相似文献
  • |
  • 引证文献
  • | |
  • 文章评论
    摘要:

    模糊测试技术在软件质量保障、软件安全测试等领域起到重要作用. 然而, 在面对编译器这样输入语义复杂的系统时, 现有的模糊测试工具由于其变异策略中缺乏对语义的感知能力, 导致生成的程序难以通过编译器前端检查. 提出了一种语义可感知的灰盒模糊测试方法, 旨在提高模糊测试工具在编译器测试领域的效能. 设计并实现了一系列可保持输入语义合法性并探索上下文多样性的变异操作符, 并针对这些操作符的特点开发了高效的选择策略. 将这些策略与传统的灰盒模糊测试工具相结合, 实现了灰盒模糊测试工具SemaAFL. 实验结果表明, 通过应用这些变异操作符, SemaAFL在GCC和Clang编译器上的代码覆盖率相比AFL++和同类工具GrayC提高了约14.5%和11.2%. 在为期一周的实验期间, SemaAFL发现并报告了6个以前未被发现的GCC和Clang缺陷.

    Abstract:

    Fuzz testing techniques play a significant role in software quality assurance and software security testing. However, when dealing with systems like compilers that have complex input semantics, existing fuzz testing tools often struggle as a lack of semantic awareness in their mutation strategies leads to the generated programs failing to pass compiler frontend checks. This study proposes a semantically-aware greybox fuzz testing method, aiming at enhancing the efficiency of fuzz testing tools in the domain of compiler testing. It designs and implements a series of mutation operators that can maintain input semantic validity and explore contextual diversity, and develops efficient selection strategies according to the characteristics of these operators. The greybox fuzz testing tool SemaAFL is developed by integrating these strategies with traditional greybox fuzz testing tools. Experimental results indicate that by applying these mutation operators, SemaAFL achieves approximately 14.5% and 11.2% higher code coverage on GCC and Clang compilers compared to AFL++ and similar tools like GrayC. During a week-long experimental period, six previously unknown bugs in GCC and Clang are discovered and reported by SemaAFL.

    参考文献
    [1] Wang JJ, Chen BH, Wei L, Liu Y. Superion: Grammar-aware greybox fuzzing. In: Proc. of the 41st IEEE/ACM Int’l Conf. on Software Engineering. Montreal: IEEE, 2019. 724–735. [doi: 10.1109/ICSE.2019.00081]
    [2] Zalewski M. American fuzzy lop. 2024. http://lcamtuf.coredump.cx/afl/
    [3] Li YK, Chen BH, Chandramohan M, Lin SW, Liu Y, Tiu A. Steelix: Program-state based binary fuzzing. In: Proc. of the 11th Joint Meeting on Foundations of Software Engineering. Paderborn: ACM, 2017. 627–637. [doi: 10.1145/3106237.3106295]
    [4] Lemieux C, Sen K. FairFuzz: A targeted mutation strategy for increasing greybox fuzz testing coverage. In: Proc. of the 33rd ACM/IEEE Int’l Conf. on Automated Software Engineering. Montpellier: ACM, 2018. 475–485. [doi: 10.1145/3238147.3238176]
    [5] Gan ST, Zhang C, Qin XJ, Tu XW, Li K, Pei ZY, Chen ZN. CollAFL: Path sensitive fuzzing. In: Proc. of the 2018 IEEE Symp. on Security and Privacy. San Francisco: IEEE, 2018. 679–696. [doi: 10.1109/SP.2018.00040]
    [6] Petsios T, Zhao J, Keromytis AD, Jana S. SlowFuzz: Automated domain-independent detection of algorithmic complexity vulnerabilities. In: Proc. of the 2017 ACM SIGSAC Conf. on Computer and Communications Security. Dallas: ACM, 2017. 2155–2168. [doi: 10.1145/3133956.3134073]
    [7] Lemieux C, Padhye R, Sen K, Song D. PerfFuzz: Automatically generating pathological inputs. In: Proc. of the 27th ACM SIGSOFT Int’l Symp. on Software Testing and Analysis. Amsterdam: ACM, 2018. 254–265. [doi: 10.1145/3213846.3213874]
    [8] Ganesh V, Leek T, Rinard M. Taint-based directed whitebox fuzzing. In: Proc. of the 31st IEEE Int’l Conf. on Software Engineering. Vancouver: IEEE, 2009. 474–484. [doi: 10.1109/ICSE.2009.5070546]
    [9] Rawat S, Jain V, Kumar A, Cojocar L, Giuffrida C, Bos H. VUzzer: Application-aware evolutionary fuzzing. In: Proc. of the 24th Network and Distributed System Security Symp. San Diego: NDSS, 2017. 1–14. [doi: 10.14722/ndss.2017.23404]
    [10] Chen P, Chen H. Angora: Efficient fuzzing by principled search. In: Proc. of the 2018 IEEE Symp. on Security and Privacy (SP). San Francisco: IEEE, 2018. 711–725. [doi: 10.1109/SP.2018.00046]
    [11] Microsoft. Security development lifecycle (SDL) practices. 2024. https://www.microsoft.com/en-us/sdl/process/verification.aspx
    [12] Bounimova E, Godefroid P, Molnar D. Billions and billions of constraints: Whitebox fuzz testing in production. In: Proc. of the 35th Int’l Conf. on Software Engineering. San Francisco: IEEE, 2013. 122–131. [doi: 10.1109/ICSE.2013.6606558]
    [13] The chromium projects. 2024. https://www.chromium.org/Home/chromium-security/bugs
    [14] Aizatsky M, Serebryany K, Chang O, Arya A, Whittaker M. OSS-Fuzz: Continuous fuzzing for open source software. 2024. https://github.com/google/oss-fuzz
    [15] Chrome Security Team. ClusterFuzz. 2024. https://google.github.io/clusterfuzz/
    [16] Manès VJM, Han H, Han C, Cha SK, Egele M, Schwartz EJ, Woo M. The art, science, and engineering of fuzzing: A survey. IEEE Trans. on Software Engineering, 2021, 47(11): 2312–2331.
    [17] Li J, Zhao BD, Zhang C. Fuzzing: A survey. Cybersecurity, 2018, 1(1): 6.
    [18] Zhu XG, Wen S, Camtepe S, Xiang Y. Fuzzing: A survey for roadmap. ACM Computing Surveys, 2022, 54(11s): 230.
    [19] Zhao XQ, Qu HP, Xu JL, Li XH, Lv WJ, Wang GG. A systematic review of fuzzing. Soft Computing, 2024, 28(6): 5493–5522.
    [20] Fioraldi A, Maier D, Eißfeldt H, Heuse M. AFL++: Combining incremental steps of fuzzing research. In: Proc. of the 14th USENIX Conf. on Offensive Technologies. USENIX Association, 2020. 10.
    [21] 梁杰, 吴志镛, 符景洲, 朱娟, 姜宇, 孙家广. 数据库管理系统模糊测试技术研究综述. 软件学报, 2025, 36(1): 399–423. http://www.jos.org.cn/1000-9825/7048.htm
    Liang J, Wu ZY, Fu JZ, Zhu J, Jiang Y, Sun JG. Survey on database management system fuzzing techniques. Ruan Jian Xue Bao/Journal of Software, 2025, 36(1): 399–423 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/7048.htm
    [22] Even-Mendoza K, Sharma A, Donaldson AF, Cadar C. GrayC: Greybox fuzzing of compilers and analysers for C. In: Proc. of the 32nd ACM SIGSOFT Int’l Symp. on Software Testing and Analysis. Seattle: ACM, 2023. 1219–1231. [doi: 10.1145/3597926.3598130]
    [23] Srivastava P, Payer M. Gramatron: Effective grammar-aware fuzzing. In: Proc. of the 30th ACM SIGSOFT Int’l Symp. on Software Testing and Analysis. ACM, 2021. 244–256. [doi: 10.1145/3460319.3464814]
    [24] Parr TJ, Quong RW. ANTLR: A predicated-LL(k) parser generator. Software: Practice and Experience, 1995, 25(7): 789–810.
    [25] Max Brunsfeld. Tree-sitter. 2024. https://tree-sitter.github.io/tree-sitter/
    [26] Bünder H. Decoupling language and editor-the impact of the language server protocol on textual domain-specific languages. In: Proc. of the 7th Int’l Conf. on Model-Driven Engineering and Software Development. Prague: ScitePress, 2019. 129–140. [doi: 10.5220/0007556301290140]
    [27] Miller BP, Fredriksen L, So B. An empirical study of the reliability of UNIX utilities. Communications of the ACM, 1990, 33(12): 32–44.
    [28] Yun I, Lee S, Xu M, Jang Y, Kim T. QSYM: A practical concolic execution engine tailored for hybrid fuzzing. In: Proc. of the 27th USENIX Conf. on Security Symp. Baltimore: USENIX Association, 2018. 745–761.
    [29] Chen P, Liu JZ, Chen H. Matryoshka: Fuzzing deeply nested branches. In: Proc. of the 2019 ACM SIGSAC Conf. on Computer and Communications Security. London: ACM, 2019. 499–513. [doi: 10.1145/3319535.3363225]
    [30] Aschermann C, Schumilo S, Blazytko T, Gawlik R, Holz T. Redqueen: Fuzzing with input-to-state correspondence. In: Proc. of the 26th Network and Distributed System Security Symp. 2019. [doi: 10.14722/ndss.2019.23371]
    [31] Lyu CY, Ji SL, Zhang C, Li YW, Lee WH, Song Y, Beyah R. MOPT: Optimized mutation scheduling for fuzzers. In: Proc. of the 28th USENIX Conf. on Security Symp. Santa Clara: USENIX Association, 2019. 1949–1966.
    [32] Chen YQ, Schwahn O, Natella R, Bradbury M, Suri N. SlowCoach: Mutating code to simulate performance bugs. In: Proc. of the 33rd IEEE Int’l Symp. on Software Reliability Engineering. Charlotte: IEEE, 2022. 274–285. [doi: 10.1109/ISSRE55969.2022.00035]
    [33] Blair W, Mambretti A, Arshad S, Weissbacher M, Robertson W, Kirda E, Egele M. HotFuzz: Discovering algorithmic denial-of-service vulnerabilities through guided micro-fuzzing. In: Proc. of the 27th Network and Distributed System Security Symp. San Diego: NDSS, 2020. 1–18. [doi: 10.14722/ndss.2020.24415]
    [34] Wen C, Wang HJ, Li YK, Qin SC, Liu Y, Xu ZW, Chen HX, Xie XF, Pu GG, Liu T. MemLock: Memory usage guided fuzzing. In: Proc. of the 42nd Int’l Conf. on Software Engineering. Seoul: ACM, 2020. 765–777. [doi: 10.1145/3377811.3380396]
    [35] Atlidakis V, Geambasu R, Godefroid P, Polishchuk M, Ray B. Pythia: Grammar-based fuzzing of REST APIs with coverage-guided feedback and learning-based mutations. arXiv:2005.11498, 2020.
    [36] Wei JY, Chen J, Feng Y, Ferles K, Dillig I. Singularity: Pattern fuzzing for worst case complexity. In: Proc. of the 26th ACM Joint Meeting on European Software Engineering Conf. and Symp. on the Foundations of Software Engineering. Lake Buena Vista: ACM, 2018. 213–223. [doi: 10.1145/3236024.3236039]
    [37] She DD, Pei KX, Epstein D, Yang JF, Ray B, Jana S. Neuzz: Efficient fuzzing with neural program smoothing. In: Proc. of the 2019 IEEE Symp. on Security and Privacy. San Francisco: IEEE, 2019. 803–817. [doi: 10.1109/SP.2019.00052]
    [38] She DD, Krishna R, Yan L, Jana S, Ray B. MTFuzz: Fuzzing with a multi-task neural network. In: Proc. of the 28th ACM Joint Meeting on European Software Engineering Conf. and Symp. on the Foundations of Software Engineering. ACM, 2020. 737–749. [doi: 10.1145/3368089.3409723]
    [39] Godefroid P, Peleg H, Singh R. Learn&Fuzz: Machine learning for input fuzzing. In: Proc. of the 32nd IEEE/ACM Int’l Conf. on Automated Software Engineering. Urbana-Champaign: IEEE, 2017. 50–59.
    [40] Liu X, Li XT, Prajapati R, Wu DH. DeepFuzz: Automatic generation of syntax valid C programs for fuzz testing. In: Proc. of the 33rd AAAI Conf. on Artificial Intelligence. Honolulu: AAAI, 2019. 1044–1051. [doi: 10.1609/aaai.v33i01.33011044]
    [41] Zong PY, Lv T, Wang DW, Deng ZZ, Liang RG, Chen K. FuzzGuard: Filtering out unreachable inputs in directed grey-box fuzzing through deep learning. In: Proc. of the 29th USENIX Conf. on Security Symp. USENIX Association, 2020. 127.
    [42] Li YK, Xue YX, Chen HX, Wu XH, Zhang C, Xie XF, Wang HJ, Liu Y. Cerebro: Context-aware adaptive fuzzing for effective vulnerability detection. In: Proc. of the 27th ACM Joint Meeting on European Software Engineering Conf. and Symp. on the Foundations of Software Engineering. Tallinn: ACM, 2019. 533–544. [doi: 10.1145/3338906.3338975]
    [43] Guo R. MongoDB’s JavaScript Fuzzer: The fuzzer is for those edge cases that your testing didn’t catch. Queue, 2017, 15(1): 38–56.
    [44] Holler C, Herzig K, Zeller A. Fuzzing with code fragments. In: Proc. of the 21st USENIX Security Symp. Bellevue: USENIX Association, 2012. 38.
    [45] Veggalam S, Rawat S, Haller I, Bos H. IFuzzer: An evolutionary interpreter fuzzer using genetic programming. In: Proc. of the 21st European Sym. on Research in Computer Security. Heraklion: Springer, 2016. 581–601. [doi: 10.1007/978-3-319-45744-4_29]
    [46] Appelt D, Nguyen CD, Briand LC, Alshahwan N. Automated testing for SQL injection vulnerabilities: An input mutation approach. In: Proc. of the 2014 ACM Int’l Symp. on Software Testing and Analysis. San Jose: ACM, 2014. 259–269. [doi: 10.1145/2610384.2610403]
    [47] Wang JJ, Chen BH, Wei L, Liu Y. Skyfire: Data-driven seed generation for fuzzing. In: Proc. of IEEE Symp. on Security and Privacy. San Jose: IEEE, 2017. 579–594. [doi: 10.1109/SP.2017.23]
    [48] Aschermann C, Frassetto T, Holz T, Jauernig P, Sadeghi AR, Teuchert D. Nautilus: Fishing for deep bugs with grammars. In: Proc. of the 26th Network and Distributed System Security Symp. 2019. [doi: 10.14722/ndss.2019.23412]
    [49] Groß S, Koch S, Bernhard L, Holz T, Johns M. Fuzzilli: Fuzzing for JavaScript JIT Compiler Vulnerabilities. In: Proc. of the 30th Network and Distributed System Security Symp. San Diego: NDSS, 2023. 1–17. [doi: 10.14722/ndss.2023.24290]
    [50] Blazytko T, Aschermann C, Schlögel M, Abbasi A, Schumilo S, Wörner S, Holz T. Grimoire: Synthesizing structure while fuzzing. In: Proc. of the 28th USENIX Conf. on Security Symp. Santa Clara: USENIX Association, 2019. 1985–2002.
    [51] Padhye R, Lemieux C, Sen K, Papadakis M, Le Traon Y. Semantic fuzzing with zest. In: Proc. of the 28th ACM SIGSOFT Int’l Symp. on Software Testing and Analysis. Beijing: ACM, 2019. 329–340. [doi: 10.1145/3293882.3330576]
    [52] Han H, Oh D, Cha SK. CodeAlchemist: Semantics-aware code generation to find vulnerabilities in JavaScript engines. In: Proc. of the 27th Network and Distributed System Security Symp. San Diego: NDSS, 2019. 1–15. [doi: 10.14722/ndss.2019.23263]
    [53] Zhang QR, Sun CN, Su ZD. Skeletal program enumeration for rigorous compiler testing. In: Proc. of the 38th ACM SIGPLAN Conf. on Programming Language Design and Implementation. Barcelona: ACM, 2017. 347–361. [doi: 10.1145/3062341.3062379]
    [54] Xia XM, Feng Y. Detecting interpreter bugs via filling function calls in skeletal program enumeration. In: Proc. of the 34th IEEE Int’l Symp. on Software Reliability Engineering. Florence: IEEE, 2023. 612–622. [doi: 10.1109/ISSRE59848.2023.00066]
    [55] Zang ZQ, Wiatrek N, Gligoric M, Shi A. Compiler testing using template java programs. In: Proc. of the 37th IEEE/ACM Int’l Conf. on Automated Software Engineering. Rochester: ACM, 2022. 23. [doi: 10.1145/3551349.3556958]
    [56] Chen YT, Su T, Sun CN, Su ZD, Zhao JJ. Coverage-directed differential testing of JVM implementations. In: Proc. of the 37th ACM SIGPLAN Conf. on Programming Language Design and Implementation. Santa Barbara: ACM, 2016. 85–99. [doi: 10.1145/2908080.2908095]
    相似文献
    引证文献
引用本文

欧先飞,蒋炎岩,许畅.语义可感知的灰盒编译器模糊测试.软件学报,2025,36(7):1-17

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-08-18
  • 最后修改日期:2024-10-15
  • 在线发布日期: 2024-12-10
文章二维码
您是第19921548位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号