微信服务号

微信订阅号

2025-6-5- 11
Home > Archive>Volume 32, Issue 7, 2021 >2118-2141. DOI:10.13328/j.cnki.jos.006258
PDF HTML XML Export Cite reminder
Survey of State-of-the-art Automatic Code Comment Generation
Author:
Affiliation:

Fund Project:

National Key R&D Program of China (2019AAA0104301); National Natural Science Foundation of China (61702041, 61872263, 61902395, 61202006); Open Program of the State Key Laboratory of Information Security (Institute of Information Engineering, Chinese Academy of Sciences) (2020-MS-07); Open Program of the Key Laboratory of Safety-critical Software (Nanjing University of Aeronautics and Astronautics) (NJ2020022); Leading-edge Technology Program of Jiangsu Natural Science Foundation (BK20202001); Intelligent Manufacturing Special Fund of Tianjin (20193155)

  • Article
    • | |
  • Metrics
    • |
  • Reference [104]
    • |
  • Related [20]
    • | | |
  • Comments
    Abstract:

    During software development and maintenance, code comments often have some problems, such as missing, insufficient, or mismatching with code content. Writing high-quality code comments takes time and effort for developers, and the quality can not be guaranteed, therefore, it is urgent for researchers to design effective automatic code comment generation methods. The automatic code comment generation issue is an active research topic in the program comprehension domain. This study conducts a systematic review of this research topic. The existing methods are divided into three categories:Template-based generation methods, information retrieval-based methods, and deep learning-based methods. Related studies are analyzed and summarizedfor each category. Then, the corpora and comment quality evaluation methods that are often used in previous studiesare analyzed, which can facilitate the experimental study for future studies. Finally, the potential research directions in the future aresummarized and discussed.

    Reference
    [1] Xia X, Bao L, Lo D, Xing Z, Hassan AE, Li S. Measuring program comprehension:A large-scale field study with professionals. IEEE Trans. on Software Engineering, 2018,44(10):951-976.
    [2] He H. Understanding source code comments at large-scale. In:Proc. of the 27th ACM Joint Meeting on European Software Engineering Conf. and Symp. on the Foundations of Software Engineering. 2019. 1217-1219.
    [3] Kramer D. API documentation from source code comments:A case study of Javadoc. In:Proc. of the 17th Annual Int'l Conf. on Computer Documentation. 1999. 147-153.
    [4] Fluri B, Wursch M, Gall HC. Do code and comments co-evolve? On the relation between source code and comment changes. In:Proc. of the 14th Working Conf. on Reverse Engineering. 2007. 70-79.
    [5] Jin Z, Liu F, Li G. Program comprehension:Present and future. Ruan Jian Xue Bao/Journal of Software, 2019,30(1):110-126(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5643.htm[doi:10.13328/ki.jos.005643]
    [6] Stapleton S, Gambhir Y, LeClair A, Eberhart Z, Weimer W, Leach K, Huang Y. A human study of comprehension and code summarization. In:Proc. of the Int'l Conf. on Program Comprehension. 2020. 2-13.
    [7] Haiduc S, Aponte J, Moreno L, Marcus A. On the use of automated text summarization techniques for summarizing source code. In:Proc. of the 17th Working Conf. on Reverse Engineering. 2010. 35-44.
    [8] Haiduc S, Aponte J, Marcus A. Supporting program comprehension with source code summarization. In:Proc. of the 32nd ACM/IEEE Int'l Conf. on Software Engineering. 2010. 223-226.
    [9] Liu F, Li G, Hu X, Jin Z. Program comprehension based on deep learning. Journal of Computer Research and Development, 2019,56(8):1605-1620(in Chinese with English abstract).
    [10] Iyer S, Konstas I, Cheung A, Zettlemoyer L. Summarizing source code using a neural attention model. In:Proc. of the 54th Annual Meeting of the Association for Computational Linguistics. 2016. 2073-2083.
    [11] Rodeghero P, McMillan C, McBurney PW, Bosch N, D'Mello S. Improving automated source code summarization via an eye-tracking study of programmers. In:Proc. of the 36th Int'l Conf. on Software Engineering. 2014. 390-401.
    [12] McBurney PW, McMillan C. Automatic documentation generation via source code summarization of method context. In:Proc. of the 22nd Int'l Conf. on Program Comprehension. 2014. 279-290.
    [13] Hu X, Li G, Xia X, Lo D, Jin Z. Deep code comment generation. In:Proc. of the 26th IEEE/ACM Int'l Conf. on Program Comprehension. 2018. 200-210.
    [14] Liu Z, Xia X, Hassan AE, Lo D, Xing Z, Wang X. Neural-machine-translation-based commit message generation:How far are we? In:Proc. of the 33rd ACM/IEEE Int'l Conf. on Automated Software Engineering. 2018. 373-384.
    [15] Liu Z, Xia X, Treude C, Lo D, Li S. Automatic generation of pull request descriptions. In:Proc. of the 34th IEEE/ACM Int'l Conf. on Automated Software Engineering. 2019. 176-188.
    [16] Nazar N, Hu Y, Jiang H. Summarizing software artifacts:A literature review. Journal of Computer Science and Technology, 2016, 31(5):883-909.
    [17] Song X, Sun H, Wang X, Yan J. A survey of automatic generation of source code comments:Algorithms and techniques. IEEE Access, 2019,7:111411-111428.
    [18] Hill E, Pollock L, Vijay-Shanker K. Automatically capturing source code context of NL-queries for software maintenance and reuse. In:Proc. of the 31st IEEE Int'l Conf. on Software Engineering. 2009. 232-242.
    [19] Sridhara G, Hill E., Muppaneni D, Pollock L, Vijay-Shanker K. Towards automatically generating summary comments for Java methods. In:Proc. of the IEEE/ACM Int'l Conf. on Automated Software Engineering, 2010. 43-52
    [20] Sridhara G, Pollock L, Vijay-Shanker K. Generating parameter comments and integrating with method summaries. In:Proc. of the 19th IEEE Int'l Conf. on Program Comprehension. 2011. 71-80.
    [21] Sridhara G, Pollock L, Vijay-Shanker K. Automatically detecting and describing high level actions within methods. In:Proc. of the 33rd Int'l Conf. on Software Engineering. 2011. 101-110.
    [22] Wang X, Pollock L, Vijay-Shanker K. Automatically generating natural language descriptions for object-related statement sequences. In:Proc. of the 24th IEEE Int'l Conf. on Software Analysis, Evolution and Reengineering. 2017. 205-216.
    [23] McBurney PW, McMillan C. Automatic source code summarization of context for Java methods. IEEE Trans. on Software Engineering, 2015,42(2):103-119.
    [24] Moreno L, Aponte J, Sridhara G, Marcus A, Pollock L, Vijay-Shanker K. Automatic generation of natural language summaries for Java classes. In:Proc. of the 21st Int'l Conf. on Program Comprehension. 2013. 23-32.
    [25] Abid NJ, Dragan N, Collard ML, Maletic JI. Using stereotypes in the automatic generation of natural language summaries for C++ methods. In:Proc. of the 2015 IEEE Int'l Conf. on Software Maintenance and Evolution. 2015. 561-565.
    [26] Maalej W, Happel HJ. Can development work describe itself? In:Proc. of the 7th IEEE Working Conf. on Mining Software Repositories. 2010. 191-200.
    [27] Dyer R, Nguyen HA, Rajan H, Nguyen TN. BOA:A language and infrastructure for analyzing ultra-large-scale software repositories. In:Proc. of the 35th Int'l Conf. on Software Engineering. 2013. 422-431.
    [28] Buse RP, Weimer WR. Automatically documenting program changes. In:Proc. of the IEEE/ACM Int'l Conf. on Automated Software Engineering. 2010. 33-42.
    [29] Cortés-Coy LF, Linares-Vásquez M, Aponte J, Poshyvanyk D. On automatically generating commit messages via summarization of source code changes. In:Proc. of the 14th IEEE Int'l Working Conf. on Source Code Analysis and Manipulation. 2014. 275-284.
    [30] Shen J, Sun X, Li B, Yang H, Hu J. On automatic summarization of what and why information in source code changes. In:Proc. of the 40th IEEE Annual Computer Software and Applications Conf. 2016. 103-112.
    [31] Rai S, Gaikwad T, Jain S, Gupta A. Method level text summarization for Java code using nano-patterns. In:Proc. of the 24th Asia-Pacific Software Engineering Conf. 2017. 199-208.
    [32] Malhotra M, Chhabra JK. Class level code summarization based on dependencies and micro patterns. In:Proc. of the 2nd Int'l Conf. on Inventive Communication and Computational Technologies. 2018. 1011-1016.
    [33] Nazar N, Jiang H, Gao G, Zhang T, Li X, Ren Z. Source code fragment summarization with small-scale crowdsourcing based features. Frontiers of Computer Science, 2016,10(3):504-517.
    [34] Rastkar S, Murphy GC. Why did this code change? In:Proc. of the 35th Int'l Conf. on Software Engineering. 2013. 1193-1196.
    [35] Eddy BP, Robinson JA, Kraft NA, Carver JC. Evaluating source code summarization techniques:Replication and expansion. In:Proc. of the 21st Int'l Conf. on Program Comprehension. 2011. 13-22.
    [36] McBurney PW, Liu C, McMillan C, Weninger T. Improving topic model source code summarization. In:Proc. of the 22nd Int'l Conf. on Program Comprehension. 2014. 291-294.
    [37] Rodeghero P, Liu C, McBurney PW, McMillan C. An eye-tracking study of Java programmers and application to source code summarization. IEEE Trans. on Software Engineering, 2015,41(11):1038-1054.
    [38] Fowkes J, Chanthirasegaran P, Ranca R, Allamanis M, Lapata M, Sutton C. Autofolding for source code summarization. IEEE Trans. on Software Engineering, 2017,43(12):1095-1109.
    [39] Panichella S, Aponte J, Di Penta M, Marcus A, Canfora G. Mining source code descriptions from developer communications. In:Proc. of the 20th IEEE Int'l Conf. on Program Comprehension. 2012. 63-72.
    [40] Rahman MM, Roy CK, Keivanloo I. Recommending insightful comments for source code using crowdsourced knowledge. In:Proc. of the 15th IEEE Int'l Working Conf. on Source Code Analysis and Manipulation. 2015. 81-90.
    [41] Wong E, Yang J, Tan L. Autocomment:Mining question and answer sites for automatic comment generation. In:Proc. of the 28th IEEE/ACM Int'l Conf. on Automated Software Engineering. 2013. 562-567.
    [42] Vassallo C, Panichella S, Di Penta M, Canfora G. Codes:Mining source code descriptions from developers discussions. In:Proc. of the 22nd Int'l Conf. on Program Comprehension. 2014. 106-109.
    [43] Wong E, Liu T, Tan L. Clocom:Mining existing source code for automatic comment generation. In:Proc. of the 22nd IEEE Int'l Conf. on Software Analysis, Evolution, and Reengineering. 2015. 380-389.
    [44] Badihi S, Heydarnoori A. Crowdsummarizer:Automated generation of code summaries for Java programs through crowdsourcing. IEEE Software, 2017,34(2):71-80.
    [45] Huang Y, Zheng Q, Chen X, Xiong Y, Liu Z, Luo X. Mining version control system for automatically generating commit comment. In:Proc. of the 2017 ACM/IEEE Int'l Symp. on Empirical Software Engineering and Measurement. 2017. 414-423.
    [46] Jiang S, Armaly A, McMillan C. Automatically generating commit messages from diffs using neural machine translation. In:Proc. of the 32nd IEEE/ACM Int'l Conf. on Automated Software Engineering. 2017. 135-146.
    [47] Alon U, Zilberstein M, Levy O, Yahav E. code2vec:Learning distributed representations of code. In:Proc. of the ACM on Programming Languages. 2019. 1-29.
    [48] Alon U, Brody S, Levy O, Yahav E. code2seq:Generating sequences from structured representations of code. arXiv Preprint arXiv:1808.01400, 2019.
    [49] Lozoya RC, Baumann A, Sabetta A, Bezzi M. Commit2Vec:Learning distributed representations of code changes. arXiv Preprint arXiv:1911.07605, 2019.
    [50] Hoang T, Kang HJ, Lawall J, Lo D. CC2Vec:Distributed representations of code changes. arXiv Preprint arXiv:2003.05620, 2020.
    [51] Allamanis M, Barr ET, Devanbu P, Sutton C. A survey of machine learning for big code and naturalness. ACM Computing Surveys, 2018,51(4):1-37.
    [52] Hindle A, Barr ET, Su Z, Gabel M, Devanbu P. On the naturalness of software. In:Proc. of the 34th Int'l Conf. on Software Engineering. 2012. 837-847.
    [53] Li YC, Xiong DY, Zhang M. A survey of neural machine translation. Chinese Journal of Computers. 2018, 41(12):2734-2755(in Chinese with English abstract).
    [54] Allamanis M, Peng H, Sutton C. A convolutional attention network for extreme summarization of source code. In:Proc. of the Int'l Conf. on Machine Learning. 2016. 2091-2100.
    [55] Zheng W, Zhou HY, Li M, Wu J. Code attention:Translating code to comments by exploiting domain features. arXiv Preprint arXiv:1709.07642, 2017.
    [56] Liang Y, Zhu K. Q. Automatic generation of text descriptive comments for code blocks. In:Proc. of the 32nd AAAI Conf. on Artificial Intelligence. 2018.
    [57] Hu X, Li G, Xia X, Lo D, Jin Z. Deep code comment generation with hybrid lexical and syntactical information. Empirical Software Engineering, 2020,25(3):2179-2217.
    [58] Kang HJ, Bissyandé TF, Lo D. Assessing the generalizability of code2vec token embeddings. In:Proc. of the 34th IEEE/ACM Int'l Conf. on Automated Software Engineering. 2019. 1-12.
    [59] Pennington J, Socher R, Manning CD. Glove:Global vectors for word representation. In:Proc. of the 2014 Conf. on Empirical Methods in Natural Language Processing. 2014. 1532-1543.
    [60] LeClair A, Jiang S, McMillan C. A neural model for generating natural language summaries of program subroutines. In:Proc. of the 41st IEEE/ACM Int'l Conf. on Software Engineering. 2019. 795-806.
    [61] Ahmad WU, Chakraborty S, Ray B, Chang KW. A transformer-based approach for source code summarization. arXiv Preprint arXiv:2005.00653, 2020.
    [62] Loyola P, Marrese-Taylor E, Matsuo Y. A neural architecture for generating natural language descriptions from source code changes. arXiv Preprint arXiv:1704.04856, 2017.
    [63] Jiang S, McMillan C. Towards automatic generation of short summaries of commits. In:Proc. of the 25th IEEE/ACM Int'l Conf. on Program Comprehension. 2017. 320-323.
    [64] Xu S, Yao Y, Xu F, Gu T, Tong H, Lu J. Commit message generation for source code changes. In:Proc. of the 27th Int'l Joint Conf. on Artificial Intelligence. 2019. 3975-3981.
    [65] Jiang S. Boosting neural commit message generation with code semantic analysis. In:Proc. of the 34th IEEE/ACM Int'l Conf. on Automated Software Engineering. 2019. 1280-1282.
    [66] Liu S, Gao C, Chen S, Nie LY, Liu Y. ATOM:Commit message generation based on abstract syntax tree and hybrid ranking. IEEE Trans. on Software Engineering, 2020.
    [67] Shido Y, Kobayashi Y, Yamamoto A, Miyamoto A, Matsumura T. Automatic source code summarization with extended tree-LSTM. In:Proc. of the 2019 Int'l Joint Conf. on Neural Networks. 2019. 1-8.
    [68] LeClair A, Haque S, Wu L, McMillan C. Improved code summarization via a graph neural network. In:Proc. of the Int'l Conf. on Program Comprehension. 2020.
    [69] Liu S, Chen Y, Xie X, Siow JK, Liu Y. Automatic code summarization via multi-dimensional semantic fusing in GNN. arXiv Preprint arXiv:2006.05405, 2020.
    [70] Wan Y, Zhao Z, Yang M, Xu G, Ying H, Wu J, Yu PS. Improving automatic source code summarization via deep reinforcement learning. In:Proc. of the 33rd ACM/IEEE Int'l Conf. on Automated Software Engineering. 2018. 397-407.
    [71] Wang W, Zhang Y, Sui Y, Wan Y, Zhao Z, WuJ, Xu G. Reinforcement-learning-guided source code summarization via hierarchical attention. IEEE Trans. on Software Engineering, 2020.
    [72] Chen Q, Zhou M. A neural framework for retrieval and summarization of source code. In:Proc. of the 33rd IEEE/ACM Int'l Conf. on Automated Software Engineering. 2018. 826-831.
    [73] Ye W, Xie R, Zhang J, Hu T, Wang X, Zhang S. Leveraging code generation to improve code retrieval and summarization via dual learning. In:Proc. of the Web Conf. 2020. 2309-2319.
    [74] Wang W, Zhang, Y, Zeng Z, Xu G. Trans. A transformer-based framework for unifying code summarization and code search. arXiv Preprint arXiv:2003.03238, 2020.
    [75] Wei B, Li G, Xia X, Fu Z, Jin Z. Code generation as a dual task of code summarization. In:Advances in Neural Information Processing Systems. 2019. 6563-6573.
    [76] Hu X, Li G, Xia X, Lo D, Lu S, Jin Z. Summarizing source code with transferred API knowledge. In:Proc. of the 27th Int'l Joint Conf. on Artificial Intelligence. 2018. 2269-2275.
    [77] Zhang J, Wang X, Zhang H, Sun H, Liu X. Retrieval-based neural source code summarization. In:Proc. of the 42nd Int'l Conf. on Software Engineering. 2020.
    [78] Liu B, Wang T, Zhang X, Fan Q, Yin G, Deng J. A neural-network based code summarization approach by using source code and its call dependencies. In:Proc. of the 11th Asia-Pacific Symp. on Internetware. 2019. 1-10.
    [79] Zhou Y, Yan X, Yang W, Chen T, Huang Z. Augmenting Java method comments generation with context information based on neural networks. Journal of Systems and Software, 2019,156:328-340.
    [80] Haque S, LeClair A, Wu L, McMillan C. Improved automatic summarization of subroutines via attention to file context. In:Proc. of the 17th Int'l Conf. on Mining Software Repositories. 2020.
    [81] Barone AVM, Sennrich R. A parallel corpus of Python functions and documentation strings for automated code documentation and code generation. In:Proc. of the 8th Int'l Joint Conf. on Natural Language Processing. 2017. 314-319.
    [82] LeClair A, McMillan C. Recommendations for datasets for source code summarization. In:Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2019. 3931-3937.
    [83] Yao Z, Weld DS, Chen WP, Sun H. Staqc:A systematically mined question-code dataset from stack overflow. In:Proc. of the 2018 World Wide Web Conf. 2018. 1693-1703.
    [84] Eberhart Z, LeClair A, McMillan C. Automatically extracting subroutine summary descriptions from unstructured comments. In:Proc. of the 27th IEEE Int'l Conf. on Software Analysis, Evolution and Reengineering. 2020. 35-46.
    [85] Khamis N, Witte R, Rilling J. Automatic quality assessment of source code comments:The JavadocMiner. In:Proc. of the Int'l Conf. on Application of Natural Language to Information Systems. 2010. 68-79.
    [86] Steidl D, Hummel B, Juergens E. Quality analysis of source code comments. In:Proc. of the 21st Int'l Conf. on Program Comprehension. 2013. 83-92.
    [87] Nenkova A, Passonneau RJ. Evaluating content selection in summarization:The Pyramid method. In:Proc. of the Human Language Technology Conf. of the North American. 2004. 145-152.
    [88] Papineni K, Roukos S, Ward T, Zhu WJ. BLEU:A method for automatic evaluation of machine translation. In:Proc. of the 40th Annual Meeting of the Association for Computational Linguistics. 2002. 311-318.
    [89] Banerjee S, Lavie A. METEOR:An automatic metric for MT evaluation with improved correlation with human judgments. In:Proc. of the ACL WORKSHop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization. 2005. 65-72.
    [90] Lin CY. Rouge:A package for automatic evaluation of summaries. In:Proc. of the Text Summarization Branches Out. 2004. 74-81.
    [91] Vedantam R, Lawrence Zitnick C, Parikh D. Cider:Consensus-based image description evaluation. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. 4566-4575.
    [92] Kamimura M, Murphy GC. Towards generating human-oriented summaries of unit test cases. In:Proc. of the 21st Int'l Conf. on Program Comprehension. 2013. 215-218.
    [93] Li B, Vendome C, Linares-Vásquez M, Poshyvanyk D, Kraft NA. Automatically documenting unit test cases. In:Proc. of the 2016 IEEE Int'l Conf. on Software Testing, Verification and Validation. 2016. 341-352.
    [94] Moreno L, Bavota G, Di Penta M, Oliveto R, Marcus A, Canfora G. Automatic generation of release notes. In:Proc. of the 22nd ACM SIGSOFT Int'l Symp. on Foundations of Software Engineering. 2014. 484-495.
    [95] Gao C, Zeng J, Xia X, Lo D, Lyu MR, King I. Automating app review response generation. In:Proc. of the 34th IEEE/ACM Int'l Conf. on Automated Software Engineering. 2019. 163-175.
    [96] Allamanis M, Barr ET, Bird C, Sutton C. Suggesting accurate method and class names. In:Proc. of the 10th Joint Meeting on Foundations of Software Engineering. 2015. 38-49.
    [97] Jiang L, Liu H, Jiang H. Machine learning based recommendation of method names:How far are we. In:Proc. of the 34th IEEE/ACM Int'l Conf. on Automated Software Engineering. 2019. 602-614.
    [98] Oda Y, Fudaba H, Neubig G, Hata H, Sakti S, Toda T, Nakamura S. Learning to generate pseudo-code from source code using statistical machine translation. In:Proc. of the 30th IEEE/ACM Int'l Conf. on Automated Software Engineering. 2015. 574-584.
    [99] Movshovitz-Attias D, Cohen W. Natural language models for predicting programming comments. In:Proc. of the 51st Annual Meeting of the Association for Computational Linguistics. 2013. 35-40.
    [100] Ciurumelea A, Proksch S, Gall HC. Suggesting comment completions for Python using neural language models. In:Proc. of the 27th IEEE Int'l Conf. on Software Analysis, Evolution and Reengineering. 2020. 456-467.
    附中文参考文献:
    [5] 金芝,刘芳,李戈.程序理解:现状与未来.软件学报.2019;30(1):110-26. http://www.jos.org.cn/1000-9825/5643.htm[doi:10.13328/ki.jos.005643]
    [9] 刘芳,李戈,胡星,等.基于深度学习的程序理解研究进展.计算机研究与发展,2019,56(8):1605-1620.
    [53] 李亚超,熊德意,张民.神经机器翻译综述.计算机学报.2018,41(12):2734-2755.
    Cited by
Get Citation

陈翔,杨光,崔展齐,孟国柱,王赞.代码注释自动生成方法综述.软件学报,2021,32(7):2118-2141

Copy
Share
Article Metrics
  • Abstract:4925
  • PDF: 12584
  • HTML: 6993
  • Cited by: 0
History
  • Received:September 02,2020
  • Revised:October 26,2020
  • Online: January 22,2021
  • Published: July 06,2021
Links:Institute of Software, CASChina Computer FederationCASNSFC
You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address：4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code：100190
Phone：010-62562563 Fax：010-62562533 Email：jos@iscas.ac.cn
Technical Support：Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063