面向知识图谱的图嵌入学习研究进展
作者:
作者简介:

杨东华(1976-), 男, 博士, 副教授, 博士生导师, 主要研究领域为数据库系统, 大数据管理与分析;何涛(1997-), 男, 硕士, 主要研究领域为知识图谱;王宏志(1978-), 男, 博士, 教授, 博士生导师, CCF杰出会员, 主要研究领域为数据库系统, 大数据管理与分析;王金宝(1985-), 男, 博士, 副教授, CCF专业会员, 主要研究领域为大数据分析

通讯作者:

王金宝, E-mail: wangjinbao@hit.edu.cn

基金项目:

国家自然科学基金(61772157, 61832003, U1866602)


Survey on Knowledge Graph Embedding Learning
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [82]
  • |
  • 相似文献 [20]
  • |
  • 引证文献
  • | |
  • 文章评论
    摘要:

    知识图谱是一种用网络结构存储知识的知识库, 在知识图谱中, 单条知识被表示成三元组的形式, 即(头实体, 关系, 尾实体). 得力于知识图谱在各个领域的广泛应用, 面向知识图谱的图嵌入学习也得到越来越多研究人员的关注. 面向知识图谱的图嵌入学习任务旨在为图谱中的实体与关系学习低维且稠密的向量, 通过图嵌入向量表达实体与关系的语义信息以及度量实体之间、关系之间、实体与关系之间的语义联系, 已有许多研究证明图嵌入模型在下游任务中的有效性. 近年来, 越来越多研究人员开始关注知识图谱的图嵌入学习, 并取得大量的研究成果, 尝试将图嵌入算法分成了基于转移思想、基于张量分解、基于传统深度学习模型、基于图神经网络以及融入额外信息的图嵌入学习共5大类, 梳理、介绍各类图嵌入算法的设计思路、算法特征以及优缺点, 以帮助指导初步接触该领域的研究人员快速学习了解该研究领域的相关模型和算法.

    Abstract:

    Knowledge graphs (KGs) serve as a kind of knowledge base by storing facts with network structure, representing each piece of fact as a triple, i.e. (head, relation, tail). Thanks to the general applications of KGs in various of fields, the embedding learning of knowledge graph has also quickly gained massive attention. This study tries to classify the existing embedding algorithms as five types: translation-based models, tensor factorization-based models, traditional deep learning-based models, graph neural network-based models, and models by fusing extra information. Then, the key ideas, algorithm features, advantages and disadvantages of different embedding models are introduced and analyzed to give the first-time researchers a guideline that can be referenced to help researchers quickly get started.

    参考文献
    [1] Xu K, Reddy S, Feng YS, Huang SF, Zhao DY. Question answering on freebase via relation extraction and textual evidence. In: Proc. of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin: ACL, 2016. 2326–2336.
    [2] Hoffmann R, Zhang CL, Ling X, Zettlemoyer L, Weld DS. Knowledge-based weak supervision for information extraction of overlapping relations. In: Proc. of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Portland: ACM, 2011. 541–550.
    [3] Carlson A, Betteridge J, Kisiel B, Settles B, Hruschka ER, Mitchell TM. Toward an architecture for never-ending language learning. In: Proc. of the 24th AAAI Conf. on Artificial Intelligence. Atlanta: AAAI, 2010. 1306–1313.
    [4] Wang WY, Mazaitis K, Cohen WW. Programming with personalized pagerank: A locally groundable first-order probabilistic logic. In: Proc. of the 22nd ACM Int’l Conf. on Information & Knowledge Management. San Francisco: ACM Press, 2013. 2129–2138.
    [5] Jiang SP, Lowd D, Dou DJ. Learning to refine an automatically extracted knowledge base using markov logic. In: Proc. of the 12th IEEE Int’l Conf. on Data Mining. Brussels: IEEE, 2012. 912–917.
    [6] Bühmann L, Lehmann J. Pattern based knowledge base enrichment. In: Proc. of the 12th Int’l Semantic Web Conf. on the Semantic Web. Sydney: Springer, 2013. 33–48.
    [7] Wang X, Bo DY, Shi C, Fan SH, Ye YF, Yu PS. A survey on heterogeneous graph embedding: Methods, techniques, applications and sources. arXiv: 2011.14867, 2020.
    [8] Toutanova K, Chen DQ, Pantel P, Poon H, Choudhury P, Gamon M. Representing text for joint embedding of text and knowledge bases. In: Proc. of the 2015 Conf. on Empirical Methods in Natural Language Processing. Lisbon: ACL, 2015. 1499–1509.
    [9] Wang X, Ji HY, Shi C, Wang B, Ye YF, Cui P, Yu PS. Heterogeneous graph attention network. In: Proc. of the 2019 World Wide Web Conf. San Francisco: ACM, 2019. 2022–2032.
    [10] Wang Q, Mao ZD, Wang B, Guo L. Knowledge Graph embedding: A survey of approaches and applications. IEEE Trans. on Knowledge and Data Engineering, 2017, 29(12): 2724–2743. [doi: 10.1109/TKDE.2017.2754499
    [11] Dai YF, Wang SP, Xiong NN, Guo WZ. A survey on knowledge graph embedding: Approaches, applications and benchmarks. Electronics, 2020, 9(5): 750. [doi: 10.3390/electronics9050750
    [12] Bordes A, Usunier N, Garcia-Durán A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. In: Proc. of the 26th Int’l Conf. on Neural Information Processing Systems. Lake Tahoe: Curran Associates Inc., 2013. 2787–2795.
    [13] Bordes A, Glorot X, Weston J, Bengio Y. Joint learning of words and meaning representations for open-text semantic parsing. In: Proc. of the 15th Int’l Conf. on Artificial Intelligence and Statistics. La Palma: JMLR, 2012. 127–135.
    [14] Bordes A, Weston J, Collobert R, Bengio Y. Learning structured embeddings of knowledge bases. In: Proc. of the 25th AAAI Confe. on Artificial Intelligence. San Francisco: AAAI, 2011. 301–306.
    [15] Wang Z, Zhang JW, Feng JL, Chen Z. Knowledge graph embedding by translating on hyperplanes. In: Proc. of the 28th AAAI Conf. on Artificial Intelligence. Québec City: AAAI, 2014. 1112–1119.
    [16] Lin YK, Liu ZY, Sun MS, Liu Y, Zhu X. Learning entity and relation embeddings for knowledge graph completion. In: Proc. of the 29th Conf. on Artificial Intelligence. Austin: AAAI, 2015. 2181–2187.
    [17] Ji GL, He SZ, Xu LH, Liu K, Zhao J. Knowledge graph embedding via dynamic mapping matrix. In: Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Int’l Joint Conf. on Natural Language Processing. Beijing: ACL, 2015. 687–696.
    [18] Ji GL, Liu K, He SZ, Zhao J. Knowledge graph completion with adaptive sparse transfer matrix. In: Proc. of the 30th AAAI Conf. on Artificial Intelligence. Phoenix: AAAI, 2016. 985–991.
    [19] Nguyen DQ, Sirts K, Qu LZ, Johnson M. STransE: A novel embedding model of entities and relationships in knowledge bases. In: Proc. of the 2016 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego: ACL, 2016. 460–466.
    [20] Zhang W, Paudel B, Zhang W, Bernstein A, Chen HJ. Interaction embeddings for prediction and explanation in knowledge graphs. In: Proc. of the 12th ACM Int’l Conf. on Web Search and Data Mining. Melbourne: ACM, 2019. 96–104.
    [21] Fan M, Zhou Q, Chang E, Zheng TF. Transition-based knowledge graph embedding with relational mapping properties. In: Proc. of the 28th Pacific Asia Conf. on Language, Information and Computing. Phuket: Department of Linguistics, Chulalongkorn University, 2014. 328–337.
    [22] Feng J, Zhou ML, Hao Y, Huang ML, Zhu XY. Knowledge graph embedding by flexible translation. arXiv: 1505.05253, 2015.
    [23] Xiao H, Huang ML, Zhu XY. From one point to a manifold: Knowledge graph embedding for precise link prediction. In: Proc. of the 25th Int’l Joint Conf. on Artificial Intelligence. New York: IJCAI/AAAI Press, 2016. 1315–1321.
    [24] Sun ZQ, Deng ZH, Nie JY, Tang J. Rotate: Knowledge graph embedding by relational rotation in complex space. In: Proc. of the 7th Int’l Conf. on Learning Representations. New Orleans: OpenReview, 2019.
    [25] Sadeghi A, Graux D, Yazdi HS, Lehmann J. MDE: Multiple distance embeddings for link prediction in knowledge graphs. 2019.
    [26] Chami I, Wolf A, Juan DC, Sala F, Ravi S, Ré C. Low-dimensional hyperbolic knowledge graph embeddings. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 2020. 6901–6914.
    [27] Zhang S, Tay Y, Yao LN, Liu Q. Quaternion knowledge graph embeddings. In: Proc. of the 33rd Int’l Conf. on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2019. 246.
    [28] Lu H, Hu H. DensE?: An enhanced non-commutative representation for knowledge graph embedding with adaptive semantic hierarchy. arXiv: 2008.04548, 2020.
    [29] Dettmers T, Minervini P, Stenetorp P, Riedel S. Convolutional 2D knowledge graph embeddings. In: Proc. of the 32nd AAAI Conf. on Artificial Intelligence. New Orleans: AAAI, 2018. 1811–1818.
    [30] Zhang ZQ, Cai JY, Zhang YD, Wang J. Learning hierarchy-aware knowledge graph embeddings for link prediction. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. New York: AAAI, 2020. 3065–3072.
    [31] Hu ZT, Huang PY, Deng YT, Gao YK, Xing E. Entity hierarchy embedding. In: Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Int’l Joint Conf. on Natural Language Proc. Beijing: ACL, 2015. 1292–1300.
    [32] Balažević I, Allen C, Hospedales T. Multi-relational poincaré graph embeddings. In: Proc. of the 33rd Conf. on Neural Information Processing Systems. Vancouver, 2019. 4465–4475.
    [33] He SZ, Liu K, Ji GL, Zhao J. Learning to represent knowledge graphs with Gaussian embedding. In: Proc. of the 24th ACM Int’l Conf. on Information and Knowledge Management. Melbourne: ACM, 2015. 623–632.
    [34] Xiao H, Huang ML, Hao Y, Zhu XY. TransA: An adaptive approach for knowledge graph embedding. arXiv: 1509.05490, 2015.
    [35] Nickel M, Tresp V, Kriegel HP. A three-way model for collective learning on multi-relational data. In: Proc. of the 28th Int’l Conf. on Machine Learning. Bellevue, ICML, 2011. 809–816.
    [36] Yang BS, Yih WT, He XD, Gao JF, Deng L. Embedding entities and relations for learning and inference in knowledge bases. In: Proc. of the 3rd Int’l Conf. on Learning Representations. San Diego: ICLR, 2015.
    [37] Kadlec R, Bajgar O, Kleindienst J. Knowledge base completion: Baselines strike back. In: Proc. of the 2nd Workshop on Representation Learning for NLP. Vancouver: ACL, 2017. 69–74.
    [38] Trouillon T, Welbl J, Riedel S, Gaussier É, Bouchard G. Complex embeddings for simple link prediction. In: Proc. of the 33rd Int’l Conf. on Machine Learning. New York: JMLR, 2016. 2071–2080.
    [39] Nickel M, Rosasco L, Poggio TA. Holographic embeddings of knowledge graphs. In: Proc. of the 30th AAAI Conf. on Artificial Intelligence. Phoenix: AAAI, 2016. 1955–1961.
    [40] Kazemi SM, Poole D. SimplE embedding for link prediction in knowledge graphs. In: Proc. of the 32nd Int’l Conf. on Neural Information Processing Systems. Montréal: Curran Associates Inc., 2018. 4289–4300.
    [41] Liu HX, Wu YX, Yang YM. Analogical inference for multi-relational embeddings. In: Proc. of the 34th Int’l Conf. on Machine Learning. Sydney: PMLR, 2017. 2168–2178.
    [42] Balažević I, Allen C, Hospedales TM. Tucker: Tensor factorization for knowledge graph completion. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int’l Joint Conf. on Natural Language Processing. Hong Kong: ACL, 2019. 5185–5194.
    [43] Xu WT, Zheng S, He L, Shao B, Yin J, Liu TY. SEEK: Segmented embedding of knowledge graphs. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 2020. 3888–3897.
    [44] Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc. of the IEEE, 1998, 86(11): 2278–2324. [doi: 10.1109/5.726791
    [45] Sabour S, Frosst N, Hinton GE. Dynamic routing between capsules. In: Proc. of the 31st Int’l Conf. on Neural Information Processing Systems. Long Beach: Curran Associates Inc, 2017. 3859–3869.
    [46] Nguyen DQ, Nguyen TD, Nguyen DQ, Phung D. A novel embedding model for knowledge base completion based on convolutional neural network. In: Proc. of the 2018 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans: ACL, 2018. 327–333.
    [47] Guo LB, Sun ZQ, Hu W. Learning to exploit long-term relational dependencies in knowledge graphs. In: Proc. of the 36th Int’l Conf. on Machine Learning. Long Beach: PMLR, 2019. 2505–2514.
    [48] Jiang XT, Wang Q, Wang B. Adaptive convolution for multi-relational learning. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis: ACL, 2018. 978–987.
    [49] Vashishth S, Sanyal S, Nitin V, Agrawal N, Talukdar P. InteractE: Improving convolution-based knowledge graph embeddings by increasing feature interactions. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. New York: AAAI, 2020. 3009–3016.
    [50] Nguyen DQ, Vu T, Nguyen TD, Nguyen DQ, Phung D. A capsule network-based embedding model for knowledge graph completion and search personalization. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis: ACL, 2019. 2180–2189.
    [51] Bruna J, Zaremba W, Szlam A, LeCun Y. Spectral networks and locally connected networks on graphs. In: Proc. of the 2nd Int’l Conf. on Learning Representations. Banff: ICLR, 2014.
    [52] Defferrard M, Bresson X, Vandergheynst P. Convolutional neural networks on graphs with fast localized spectral filtering. In: Proc. of the 30th Int’l Conf. on Neural Information Processing Systems. Barcelona: Curran Associates Inc., 2016. 3844–3852.
    [53] Hamilton WL, Ying R, Leskovec J. Inductive representation learning on large graphs. In: Proc. of the 31st Int’l Conf. on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 1025–1035.
    [54] Sanyal S, Balachandran J, Yadati N, Kumar A, Rajagopalan P, Sanyal S, Talukdar P. MT-CGCNN: Integrating crystal graph convolutional neural network with multitask learning for material property prediction. arXiv: 1811.05660, 2018.
    [55] Schlichtkrull M, Kipf TN, Bloem P, van den Berg R, Titov I, Welling M. Modeling relational data with graph convolutional networks. In: Proc. of the 15th Int’l Conf. on the Semantic Web. Heraklion: Springer, 2018. 593–607.
    [56] Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE. Neural message passing for quantum chemistry. In: Proc. of the 34th Int’l Conf. on Machine Learning. Sydney: PMLR, 2017. 1263–1272.
    [57] Kipf TN, Welling M. Semi-supervised classification with graph convolutional networks. In: Proc. of the 5th Int’l Conf. on Learning Representations. Toulon: OpenReview.net, 2017. 1–14.
    [58] Ye R, Li X, Fang YJ, Zang HY, Wang MZ. A vectorized relational graph convolutional network for multi-relational network alignment. In: Proc. of the 28th Int’l Joint Conf. on Artificial Intelligence. Macao: IJCAI.org, 2019. 4135–4141.
    [59] Cai L, Yan B, Mai GC, Janowicz K, Zhu R. TransGCN: Coupling transformation assumptions with graph convolutional networks for link prediction. In: Proc. of the 10th Int’l Conf. on Knowledge Capture. Marina Del Rey: ACM, 2019. 131–138.
    [60] Vashishth S, Sanyal S, Nitin V, Talukdar PP. Composition-based multi-relational graph convolutional networks. In: Proc. of the 8th Int’l Conf. on Learning Representations. Addis Ababa: OpenReview.net, 2020.
    [61] Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. In: Proc. of the 6th Int’l Conf. on Learning Representations. Vancouver: OpenReview.net, 2018.
    [62] Shang C, Tang Y, Huang J, Bi JB, He XD, Zhou BW. End-to-end structure-aware convolutional networks for knowledge base completion. In: Proc. of the 33rd AAAI Conf. on Artificial Intelligence. Honolulu: AAAI, 2019. 3060–3067.
    [63] Nathani D, Chauhan J, Sharma C, Kaul M. Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 4710–4723.
    [64] Liu XY, Tan HB, Chen QH, Lin GY. RAGAT: Relation aware graph attention network for knowledge graph completion. IEEE Access, 2021, 9: 20840–20849. [doi: 10.1109/ACCESS.2021.3055529
    [65] Sun ZQ, Wang CM, Hu W, Chen MH, Dai J, Zhang W, Qu YZ. Knowledge graph alignment network with gated multi-hop neighborhood aggregation. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. New York: AAAI, 2020. 222–229.
    [66] Socher R, Chen DQ, Manning CD, Ng AY. Reasoning with neural tensor networks for knowledge base completion. In: Proc. of the 26th Int’l Conf. on Neural Information Processing Systems. Lake Tahoe: Curran Associates Inc, 2013. 926–934.
    [67] Zhang DX, Yuan B, Wang D, Liu R. Joint semantic relevance learning with text data and graph knowledge. In: Proc. of the 3rd Workshop on Continuous Vector Space Models. Beijing: ACL, 2015. 32–40.
    [68] Xie RB, Liu ZY, Jia J, Luan HB, Sun MS. Representation learning of knowledge graphs with entity descriptions. In: Proc. of the 30th AAAI Conf. on Artificial Intelligence. Phoenix: AAAI, 2016. 2659–2665.
    [69] Xu JC, Qiu XP, Chen K, Huang XJ. Knowledge graph representation with jointly structural and textual encoding. In: Proc. of the 26th Int’l Joint Conf. on Artificial Intelligence. Melbourne: IJCAI, 2017. 1318–1324.
    [70] Wang YS, Zhang HH, Shi G, Liu ZR, Zhou Q. A model of text-enhanced knowledge graph representation learning with mutual attention. IEEE Access, 2020, 8: 52895–52905. [doi: 10.1109/ACCESS.2020.2981212
    [71] Daza D, Cochez M, Groth P. Inductive entity representations from text via link prediction. In: Proc. of the Web Conf. 2021 (WWW2021). Ljubljana: ACM, 2021. 798–808.
    [72] Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis: ACL, 2019. 4171–4186.
    [73] Xie RB, Liu ZY, Sun MS. Representation learning of knowledge graphs with hierarchical types. In: Proc. of the 25th Int’l Joint Conf. on Artificial Intelligence. New York: IJCAI/AAAI Press, 2016. 2965–2971.
    [74] Tang X, Chen L, Cui J, Wei BG. Knowledge representation learning with entity descriptions, hierarchical types, and textual relations. Information Processing & Management, 2019, 56(3): 809–822. [doi: 10.1016/j.ipm.2019.01.005
    [75] Rossi A, Barbosa D, Firmani D, Matinata A, Merialdo P. Knowledge graph embedding for link prediction: A comparative analysis. ACM Trans. on Knowledge Discovery from Data, 2021, 15(2): 14. [doi: 10.1145/3424672
    [76] Sun ZQ, Vashishth S, Sanyal S, Talukdar P, Yang YM. A re-evaluation of knowledge graph completion methods. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 2020. 5516–5522.
    [77] Zhang Z, Zhuang FZ, Qu M, Lin F, He Q. Knowledge graph embedding with hierarchical relation structure. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Brussels: ACL, 2018. 3198–3207.
    [78] Wang ZH, Lai KP, Li PJ, Bing LD, Lam W. Tackling long-tailed relations and uncommon entities in knowledge graph completion. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int’l Joint Conf. on Natural Language Processing. Hong Kong: ACL, 2020. 250–260.
    [79] Zhang W, Paudel B, Wang L, Chen JY, Zhu H, Zhang W, Bernstein A, Chen HJ. Iteratively learning embeddings and rules for knowledge graph reasoning. In: Proc. of the 2019 World Wide Web Conf. San Francisco: ACM, 2366–2377.
    [80] Sheng JW, Guo S, Chen ZY, Yue JW, Wang LH, Liu TW, Xu HB. Adaptive attentional network for few-shot knowledge graph completion. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing. ACL, 2020. 1681–1691.
    [81] Zhang CX, Yao HX, Huang C, Jiang M, Li ZH, Chawla NV. Few-shot knowledge graph completion. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. New York: AAAI, 2020. 3041–3048.
    [82] Kazemi SM, Goel R, Jain K, Kobyzev I, Sethi A, Forsyth P, Poupart P. Representation learning for dynamic graphs: A survey. arXiv: 1905.11485, 2019.
    引证文献
引用本文

杨东华,何涛,王宏志,王金宝.面向知识图谱的图嵌入学习研究进展.软件学报,2022,33(9):3370-3390

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2021-05-13
  • 最后修改日期:2021-06-29
  • 在线发布日期: 2021-10-20
  • 出版日期: 2022-09-06
文章二维码
您是第19811425位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号