微信服务号

微信订阅号

2025年3月24日 12:21 星期一
首页 > 过刊浏览>2023年第34卷第10期 >4584-4600. DOI:10.13328/j.cnki.jos.006882
PDF HTML阅读 XML下载 导出引用 引用提醒
QA-KGNet: 一种语言模型驱动的知识图谱问答模型
作者:
作者简介:

乔少杰(1981-),男,博士,教授,CCF杰出会员,主要研究领域为数据库,人工智能,知识图谱;覃晓(1973-),女,教授,主要研究领域为人工智能,知识图谱;杨国平(1997-),男,硕士生,主要研究领域为数据库查询优化;屈露露(1998-),女,硕士生,主要研究领域为人工智能,数据挖掘;于泳(1998-),男,硕士生,主要研究领域为知识图谱;冉黎琼(1998-),女,硕士生,主要研究领域为数据库,数据挖掘;韩楠(1984-),女,博士,副教授,主要研究领域为数据库,数据挖掘;李贺(1984-),男,博士,副教授,CCF专业会员,主要研究领域为人工智能,数据挖掘.

通讯作者:

韩楠,E-mail:hannan@cuit.edu.cn

基金项目:

国家自然科学基金(61962006);四川省科技计划(2021JDJQ0021,2022YFG0186);四川音乐学院数字媒体艺术四川省重点实验室资助项目(21DMAKL02);成都市技术创新研发项目(2021-YF05-00491-SN);成都市重大科技创新项目(2021-YF08-00156-GX);成都市“揭榜挂帅”科技项目(2021-JB00-00025-GX);成都市软科学研究项目(2021-RK00-00065-ZF,2021-RK00-00066-ZF);广西重大创新驱动项目(桂科AA22068057);四川省社会科学高水平团队项目(2015Z177)


QA-KGNet: Language Model-driven Knowledge Graph Question-answering Model
Author:
  • 摘要
    • | |
  • 访问统计
    • |
  • 参考文献 [53]
    • |
  • 相似文献 [20]
    • | | |
  • 文章评论
    摘要:

    基于知识图谱的问答系统可以解析用户问题,已成为一种检索知识、自动回答所询问题的有效途径.知识图谱问答系统通常是利用神经程序归纳模型,将自然语言问题转化为逻辑形式,在知识图谱上执行该逻辑形式能够得到答案.然而,使用预训练语言模型和知识图谱的知识问答系统包含两个挑战:(1)给定问答(question-answering,QA)上下文,需要从大型知识图谱(knowledge graph,KG)中识别相关知识;(2)对QA上下文和KG进行联合推理.基于此,提出一种语言模型驱动的知识图谱问答推理模型QA-KGNet,将QA上下文和KG连接起来形成一个工作图,使用语言模型计算给定QA上下文节点与KG节点的关联度,并使用多头图注意力网络更新节点表示.在CommonsenseQA、OpenBookQA和MedQA-USMLE真实数据集上进行实验来评估QA-KGNet的性能,实验结果表明:QA-KGNet优于现有的基准模型,表现出优越的结构化推理能力.

    Abstract:

    The question-answering system based on knowledge graphs can analyze user questions, and has become an effective way to retrieve relevant knowledge and automatically answer the given questions. The knowledge graph-based question-answering system usually uses a neural program induction model to convert natural language question into a logical form, and the answer can be obtained by executing the logical form on the knowledge graph. However, the knowledge question-answering system by using pre-trained language models and knowledge graphs involves two challenges: (1) given the QA (question-answering) context, relevant knowledge needs to be identified from a large KG (knowledge graph); (2) it isneeded to perform the joint reasoning on QA context and KG. Based on these challenges, a language model-driven knowledge graph question-answering model is proposed, which connects the QA context and KG to form a joint graph, and uses a language model to calculate the relevance of the given QA context nodes and KG nodes, and a multi-head graph attention network is employed to update the node representation. Extensive experiments on the CommonsenseQA, OpenBookQA and MedQA-USMLE real datasets are conducted to evaluate the performance of QA-KGNet and the experimental results show that QA-KGNet outperforms existing benchmark models and exhibits excellent structured reasoning capability.

    参考文献
    [1] Xiong F, Wang HZ. Mining simple path traversal patterns in knowledge graph. Journal of Web Engineering, 2022, 21(2): 258-271. [doi: 10.13052/jwe1540-9589.2128]
    [2] Jamehshourani M, Fatemi A, Nematbakhsh MA. Implicit relation-based question answering to answer simple questions over DBpedia. Turkish Journal of Electrical Engineering and Computer Sciences, 2020, 28(3):1474-1490. [doi: 10.3906/elk-1908-102]
    [3] Jiang K, Wu DK, Jiang H. FreebaseQA: A new factoid QA data set matching trivia-style question-answer pairs with Freebase. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New York: ACM, 2019. 318-323. [doi: 10.18653/v1/n19-1028]
    [4] Lu XY, Wang LF, Jiang ZJ, He SC, Liu SZ. MMKRL: A robust embedding approach for multi-modal knowledge graph representation learning. Applied Intelligence, 2022, 52(7): 7480-7497. [doi: 10.1007/s10489-021-02693-9]
    [5] Menin A, Torchelsen RP, Nedel LP. The effects of VR in training simulators: Exploring perception and knowledge gain. Computers & Graphics, 2022, 102(5): 402-412. [doi: 10.1016/j.cag.2021.09.015]
    [6] Li JZ, Hou L. Reviews on knowledge graph research. Journal of Shanxi University (Nat. Sci. Ed.), 2017, 40(3): 454-459(in Chinese with English abstract). [doi: 10.13451/j.cnki.shanxi.univ(nat.sci.).2017.03.008]
    [7] Nayyeri M, Cil GM, Vahdati S, Osborne F, Rahman M, Angioni S, Salatino AA, Recupero DR, Vassilyeva R, Motta E, Lehmann J. Trans4E: Link prediction on scholarly knowledge graphs. Neurocomputing, 2021, 461(3): 530-542. [doi: 10.1016/j.neucom.2021. 02.100]
    [8] Dessì D, Osborne F, Recupero DR, Buscaldi D, Motta E. Generating knowledge graphs by employing natural language processing and machine learning techniques within the scholarly domain. Future Generation Computer Systems, 2021, 116(7): 253-264. [doi: 10.1016/j.future.2020.10.026]
    [9] Lin QK, Zhu YF, Lu H, Shi K, Niu ZD. Improving university faculty evaluations via multi-view knowledge graph. Future Generation Computer Systems, 2021, 117(5): 181-192. [doi: 10.1016/j.future.2020.11.021]
    [10] Yue Q, Li X, Li D. Chinese relation extraction on forestry knowledge graph construction. Computer Systems: Science & Engineering, 2021, 37(3): 423-442. [doi: 10.32604/csse.2021.014448]
    [11] Yu CM, Wang F, Liu YH, An L. Research on knowledge graph alignment model based on deep learning. Expert Systems with Applications, 2021, 186(4): 115-129. [doi: 10.1016/j.eswa.2021.115768]
    [12] Petroni F, Rocktaschel T, Riedel S, Lewis PSH, Bakhtin A, Wu YX, Miller AH. Language models as knowledge bases? In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing. Washington: ACL, 2019. 2463-2473. [doi: 10.18653/v1/ D19-1250]
    [13] Bosselut A, Rashkin H, Sap M, Malaviya C, Celikyilmaz A, Choi Y. COMET: Commonsense transformers for automatic knowledge graph construction. In: Proc. of the 57th Conf. of the Association for Computational Linguistics. Washington: ACL, 2019. 4762-4779. [doi: 10.18653/v1/p19-1470]
    [14] Speer R, Chin J, Havasi C. ConceptNet 5.5: An open multilingual graph of general knowledge. In: Proc. of the 31st AAAI Conf. on Artificial Intelligence. San Francisco: AAAI, 2017. 4444-4451.
    [15] Huang SS, Zhu KQ, Liao QZ, Shen LB, Zhao YG. Enhanced story representation by ConceptNet for predicting story endings. In: Proc. of the 29th ACM Int’l Conf. on Information and Knowledge Management. New York: ACM, 2020. 3277-3280. [doi: 10.1145/3340531.3417466]
    [16] Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou YQ, Li W, Liu PJ. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 2020, 21(2): 1-14.
    [17] Liao WX, Zeng B, Yin XW, Wei PF. An improved aspect-category sentiment analysis model for text sentiment analysis based on RoBERTa. Applied Intelligence, 2021, 51(6): 3522-3533.
    [18] Lin YB, Chen XY, Chen JM, Ren X. KagNet: Knowledge-aware graph networks for commonsense reasoning. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing. New York: ACM, 2019. 2829-2839. [doi: 10.18653/v1/D19-1282]
    [19] Bao JW, Duan N, Yan Z, Zhou M, Zhao TJ. Constraint-based question answering with knowledge graph. In: Proc. of the 26th Int’l Conf. on Computational Linguistics. Washington: ACL, 2016. 2503-2514.
    [20] Sun HT, Dhingra B, Zaheer M, Mazaitis K, Salakhutdinov R, Cohen WW. Open domain question answering using early fusion of knowledge bases and text. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. New York: ACM, 2018. 4231-4242. [doi: 10.18653/v1/d18-1455]
    [21] Wang XY, Kapanipathi P, Musa R, Yu M, Talamadupula K, Abdelaziz I, Chang M, Fokoue A, Makni B, Mattei N, Witbrock M. Improving natural language inference using external knowledge in the science questions domain. In: Proc. of the 33rd AAAI Conf. on Artificial Intelligence. San Francisco: AAAI, 2019. 7208-7215. [doi: 10.1609/aaai.v33i01.33017208]
    [22] Feng YL, Chen XY, Lin YB, Wang PF, Yan J, Ren X. Scalable multi-hop relational reasoning for knowledge-aware question answering. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing. New York: ACM, 2020. 1295-1306. [doi: 10.18653/v1/2020.emnlp-main.99]
    [23] Yasunaga M, Ren HY, Bosselut A, Liang P, Leskovec J. QA-GNN: Reasoning with language models and knowledge graphs for question answering. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Washington: ACL, 2021. 535-546. [doi: 10.18653/v1/2021.naacl-main.45]
    [24] Pan XM, Sun K, Yu D, Chen JS, Ji H, Cardie C, Yu D. Improving question answering with external knowledge. In: Proc. of the 2nd Workshop on Machine Reading for Question Answering. Washington: ACL, 2019. 27-37. [doi: 10.18653/v1/D19-5804]
    [25] Mihaylov T, Frank A. Knowledgeable reader: Enhancing cloze-style reading comprehension with external commonsense knowledge. In: Proc. of the 56th Annual Meeting of the Association for Computational Linguistics. Washington: ACL, 2018. 821-832. [doi: 10.18653/v1/P18-1076]
    [26] Bosselut A, Bras RL, Choi YJ. Dynamic neuro-symbolic knowledge graph construction for zero-shot commonsense question answering. In: Proc. of 35th AAAI Conf. on Artificial Intelligence. San Francisco: AAAI, 2021. 4923-4931.
    [27] Paul D, Frank A. Ranking and selecting multi-hop knowledge paths to better predict human needs. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Washington: ACL, 2019. 3671-3681. [doi: 10.18653/v1/n19-1368]
    [28] Yang A, Wang Q, Liu J, Liu K, Lyu YJ, Wu H, She QQ, Li SJ. Enhancing pre-trained language representations with rich knowledge for machine reading comprehension. In: Proc. of the 57th Conf. of the Association for Computational Linguistics. Washington: ACL, 2019. 2346-2357. [doi: 10.18653/v1/p19-1226]
    [29] Yih WT, Richardson M, Meek C, Chang MW, Suh J. The value of semantic parse labeling for knowledge base question answering. In: Proc. of the 54th Annual Meeting of the Association for Computational Linguistics. Washington: ACL, 2019. 546-555. [doi: 10.18653/v1/p16-2033]
    [30] Bevilacqua M, Navigli R. Quasi bidirectional encoder representations from transformers for word sense disambiguation. In: Proc. of the Int’l Conf. on Recent Advances in Natural Language Processing. Varna: INCOMA Ltd, 2019. 122-131. [doi: 10.26615/978- 954-452-056-4\_015]
    [31] Sun HT, Weiss TB, Cohen WW. PullNet: Open domain question answering with iterative retrieval on knowledge bases and text. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing. Washington: ACL, 2019. 2380-2390. [doi: 10. 18653/v1/D19-1242]
    [32] Xiong WH, Yu M, Chang SY, Guo XX, Wang YW. Improving question answering over incomplete kbs with knowledge-aware reader. In: Proc. of the 57th Conf. of the Association for Computational Linguistics. Washington: ACL, 2019. 4258-4264. [doi: 10.18653/v1/p19-1417]
    [33] Zhang YH, Qi P, Manning CD. Graph convolution over pruned dependency trees improves relation extraction. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Washington: ACL, 2018. 2205-2215. [doi: 10.18653/v1/d18-1244]
    [34] Yasunaga M, Liang P. Graph-based, self-supervised program repair from diagnostic feedback. In: Proc. of the 37th Int’l Conf. on Machine Learning. Berlin: PMLR, 2020. 10799-10808.
    [35] Lee J, Lee I, Kang J. Self-attention graph pooling. In: Proc. of the 36th Int’l Conf. on Machine Learning. PMLR, 2019. 3734-3743.
    [36] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I. Attention is all you need. In: Proc. of the Advances in Neural Information Processing Systems 30: Annual Conf. on Neural Information Processing Systems 2017. New York: ACM, 2017. 5998-6008.
    [37] Velickovic P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y. Graph attention networks. In: Proc. of the 6th Int’l Conf. on Learning Representations. New York: ACM, 2018. 1-12.
    [38] Talmor A, Herzig J, Lourie N, Berant J. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Washington: ACL, 2019. 4149-4158. [doi: 10.18653/v1/n19-1421]
    [39] Mihaylov T, Clark P, Khot T, Sabharwal A. Can a suit of armor conduct electricity? A new dataset for open book question answering. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Washington: ACL, 2018. 2381-2391. [doi: 10.18653/v1/d18-1260]
    [40] Jin D, Pan B, Oufattole N, Weng WH, Fang HY, Szolovits P. What disease does this patient have? A large-scale open domain question answering dataset from medical exams. Applied Sciences, 2021, 11(4): 1-17. [doi: 10.3390/app11146421]
    [41] Bodenreider O. The unified medical language system (UMLS): Integrating biomedical terminology. Nucleic Acids Research, 2004, 32(4): 267-270. [doi: 10.1093/nar/gkh061]
    [42] Wishart DS, Feunang YD, Guo AC, Lo EJ, Marcu A, Grant JR, Sajed T, Johnson D, Li C, Sayeeda Z, Assempour N, Iynkkaran I, Liu YF, Maciejewski A, Gale N, Wilson A, Chin L, Cummings R, Le D, Pon A, Knox C, Wilson M. DrugBank 5.0: A major update to the DrugBank database for 2018. Nucleic Acids Research, 2018, 46(8): 1074-1082. [doi: 10.1093/nar/gkx1037]
    [43] Liu FY, Shareghi E, Meng ZQ, Basaldella M, Collier N. Self-alignment pretraining for biomedical entity representations. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Washington: ACL, 2021. 4228-4238. [doi: 10.18653/v1/2021.naacl-main.334]
    [44] Liu YH, Ott M, Goyal M, Du JF, Joshi M, Chen DQ, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692, 2019.
    [45] Clark P, Etzioni O, Khashabi D, Khot T, Mishra BD, Richardson K, Sabharwal A, Schoenick C, Tafjord O, Tandon N, Bhakthavatsalam S, Groeneveld D, Guerquin M, Schmitz M. From ‘f’ to ‘a’ on the NY regent science exams: An overview of the aristo project. arXiv:1909.01958, 2019.
    [46] Santoro A, Raposo D, Barrett DGT, Malinowski M, Pascanu R, Battaglia PW, Lillicrap T. A simple neural network module for relational reasoning. In: Proc. of the Advances in Neural Information Processing Systems 30: Annual Conf. on Neural Information Processing Systems. Washington: ACL, 2017. 4967-4976.
    [47] Schlichtkrull MS, Kipf TN, Bloem P, Berg RVD, Titov I, Welling M. Modeling relational data with graph convolutional networks. In: Proc. of the 15th Int’l Conf. on Semantic Web. Berlin: Springer, 2018. 593-607. [doi: 10.1007/978-3-319-93417-4\_38]
    [48] Wang XY, Kapanipathi P, Musa R, Yu M, Talamadupula K, Abdelaziz I, Chang M, Fokoue A, Makni B, Mattei N, Witbrock M. Improving natural language inference using external knowledge in the science questions domain. In: Proc. of the 33rd AAAI Conf. on Artificial Intelligence. San Francisco: AAAI, 2019. 7208-7215. [doi: 10.1609/aaai.v33i01.33017208]
    [49] Kingma DP, Ba J. Adam: A method for stochastic optimization. In: Proc. of the 3rd Int’l Conf. on Learning Representations. Washington: ACL, 2015. 1-15.
    [50] Devlin J, Chang MW, Lee KT, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Washington: ACL, 2019. 4171-4186. [doi: 10.18653/v1/n19-1423]
    [51] Lee J, Yoon W, Kim SD, Kim D, Kim SY, So CH, Kang J. BioBERT: A pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 2020, 36(4): 1234-1240. [doi: 10.1093/bioinformatics/btz682]
    附中文参考文献:
    [6] 李涓子, 侯磊. 知识图谱研究综述. 山西大学学报(自然科学版), 2017, 40(3): 454-459. [doi: 10.13451/j.cnki.shanxi.univ(nat. sci.).2017.03.008]
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

乔少杰,杨国平,于泳,韩楠,覃晓,屈露露,冉黎琼,李贺. QA-KGNet: 一种语言模型驱动的知识图谱问答模型.软件学报,2023,34(10):4584-4600

复制
分享
文章指标
  • 点击次数:2127
  • 下载次数: 4320
  • HTML阅读次数: 2985
  • 引用次数: 0
历史
  • 收稿日期:2022-07-02
  • 最后修改日期:2022-08-18
  • 在线发布日期: 2023-01-13
  • 出版日期: 2023-10-06
文章二维码
友情链接:中国科学院软件研究所中国计算机学会中国科学院国家自然科学基金委员会CCF数字图书馆Int'l J of Softw Info计算机系统应用
您是第19727817位访问者
版权所有：中国科学院软件研究所 京ICP备05046678号-3
地址：北京市海淀区中关村南四街4号,邮政编码：100190
电话：010-62562563 传真：010-62562533 Email：jos@iscas.ac.cn
技术支持：北京勤云科技发展有限公司

京公网安备 11040202500063号