FS-Net: Frequency Statistical Network for Temporal Knowledge Graph Reasoning
Author:
Affiliation:

  • Article
  • | |
  • Metrics
  • |
  • Reference [48]
  • |
  • Related
  • |
  • Cited by
  • | |
  • Comments
    Abstract:

    Temporal knowledge graph (TKG) reasoning has attracted significant attention of researchers. Existing TKG reasoning methods have made great progress through modeling historical information. However, the time-variability problem and unseen entity (relation) problem are still two major challenges that hinder the further improvement of this field. Moreover, since the structural information and temporal dependencies of the historical subgraph sequence have to be modeled, the traditional embedding-based methods often have high time consumption in the training and predicting processes, which greatly limits the application of the reasoning model in real-world scenarios. To address these issues, this study proposes a frequency statistical network for TKG reasoning, namely FS-Net. On the one hand, FS-Net continuously generates time-varying scores for the predictions at the changing timestamps based on the latest short-term historical fact frequency statistics. On the other hand, based on the fact frequency statistics at the current timestamp, FS-Net supplements the historical unseen entities (relations) for the predictions; specially, FS-Net does not need training, and has a very high time efficiency. The experiments on two TKG benchmark datasets demonstrate that FS-Net has a great improvement compared with the baseline models.

    Reference
    [1] Wang F, Liu JP, Liu B, Qian TY, Xiao YH, Peng ZY. Survey on construction of code knowledge graph and intelligent software development. Ruan Jian Xue Bao/Journal of Software, 2020, 31(1): 47-66(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5893.htm [doi: 10.13328/j.cnki.jos.005893]
    [2] Liu BZ, Wang X, Liu PK, Li SZ, Zhang XW, Yang YJ. KGDB: Knowledge graph database system with unified model and query language. Ruan Jian Xue Bao/Journal of Software, 2021, 32(3): 781-804(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6181.htm [doi: 10.13328/j.cnki.jos.006181]
    [3] Dasgupta SS, Ray SN, Talukdar P. Hyte: Hyperplane-based temporally aware knowledge graph embedding. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. 2018. 2001-2011.
    [4] Garcia-Duran A, Dumančić S, Niepert M. Learning sequence encoders for temporal knowledge graph completion. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. 2018. 4816-4821.
    [5] Jiang TS, Liu TY, Ge T, Sha L, Chang BB, Li SJ, Sui ZF. Towards time-aware knowledge graph completion. In: Proc. of the 26th Int’l Conf. on Computational Linguistics: Technical Papers (COLING 2016). 2016. 1715-1724.
    [6] Han Z, Chen P, Ma YP, Tresp V. Explainable subgraph reasoning for forecasting on temporal knowledge graphs. In: Proc. of the Int’l Conf. on Learning Representations. 2020.
    [7] Jin WJ, Qu M, Jin XS, Ren X. Recurrent event network: Autoregressive structure inferenceover temporal knowledge graphs. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing. 2020. 6669-6683.
    [8] Li ZX, Jin XL, Guan SP, Li W, Guo JF, Wang YZ, Cheng XQ. Search from history and reason for future: Two-stage reasoning on temporal knowledge graphs. In: Proc. of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int’l Joint Conf. on Natural Language Processing (Vol.1: Long Papers). 2021. 4732-4743.
    [9] Li ZX, Jin XL, Li W, Guan SP, Guo JF, Shen HW, Wang YZ, Cheng XQ. Temporal knowledge graph reasoning based on evolutional representation learning. In: Proc. of the 44th Int’l ACM SIGIR Conf. on Research and Development in Information Retrieval. 2021. 408-417.
    [10] Liu YS, Ma YP, Hildebrandt M, Joblin M, Tresp V. Tlogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs. In: Proc. of the AAAI Conf. on Artificial Intelligence. 2022, 36(4): 4120-4127.
    [11] Sun HH, Zhong JL, Ma YP, Han Z, He K. Timetraveler: Reinforcement learning for temporal knowledge graph forecasting. In: Proc. of the 2021 Conf. on Empirical Methods in Natural Language Processing. 2021. 8306-8319.
    [12] Zhu CC, Chen MH, Fan CJ, Cheng GQ, Zhang Y. Learning from history: Modeling temporal knowledge graphs with sequential copy-generation networks. In: Proc. of the AAAI Conf. on Artificial Intelligence, Vol.35. 2021. 4732-4740.
    [13] Li ZX, Guan SP, Jin XL, Peng WH, Lyu YJ, Zhu Y, Bai L, Li W, Guo JF, Cheng XQ. Complex evolutional pattern learning for temporal knowledge graph reasoning. In: Proc. of the 60th Annual Meeting of the Association for Computational Linguistics (Vol.2: Short Papers). 2022. 290-296.
    [14] Liu KZ, Zhao F, Chen HX, Li YC, Xu GD, Jin H. DA-Net: Distributed attention network for temporal knowledge graph reasoning. In: Proc. of the 31st ACM Int’l Conf. on Information & Knowledge Management. 2022. 1289-1298.
    [15] Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems. 2013. 26.
    [16] Wang Z, Zhang JW, Feng JL, Chen Z. Knowledge graph embedding by translating on hyperplanes. In: Proc. of the AAAI Conf. on Artificial Intelligence, Vol.28. 2014.
    [17] Lin Y, Liu Z, Sun M, Liu Y, Zhu X. Learning entity and relation embeddings for knowledge graph completion. In: Proc. of the 29th AAAI Conf. on Artificial Intelligence. 2015.
    [18] Ji G, He S, Xu L, Liu K, Zhao J. Knowledge graph embedding via dynamic mapping matrix. In: Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Int’l Joint Conf. on Natural Language Processing (Vol.1: Long Papers). 2015. 687-696.
    [19] Sun ZQ, Deng ZH, Nie JY, Tang J. Rotate: Knowledge graph embedding by relational rotation in complex space. In: Proc. of the Int’l Conf. on Learning Representations. 2018.
    [20] Trouillon T, Welbl J, Riedel S, Gaussier É, Bouchard G. Complex embeddings for simple link prediction. In: Proc. of the Int’l Conf. on Machine Learning. 2016. 2071-2080.
    [21] Yang BS, Wen-tau Yih S, He XD, Gao JF, Deng L. Embedding entities and relations for learning and inference in knowledge bases. In: Proc. of the Int’l Conf. on Learning Representations 2015. 2015.
    [22] Dettmers T, Minervini P, Stenetorp P, Riedel S. Convolutional 2D knowledge graph embeddings. In: Proc. of the AAAI Conf. on Artificial Intelligence, Vol.32. 2018.
    [23] Shang C, Tang Y, Huang J, Bi JB, He XD, Zhou BW. End-to-end structure-aware convolutional networks for knowledge base completion. In: Proc. of the AAAI Conf. on Artificial Intelligence, Vol.33. 2019. 3060-3067.
    [24] Xiong W, Yu M, Chang S, Guo X, Wang WY. One-shot relational learning for knowledge graphs. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. 2018. 1980-1990.
    [25] Chen M, Zhang W, Zhang W, Chen Q, Chen H. Meta relational learning for few-shot link prediction in knowledge graphs. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int’l Joint Conf. on Natural Language Processing. 2019. 4217-4226.
    [26] Zhang NY, Xie X, Chen X, Deng SM, Ye HB, Chen HJ. Knowledge collaborative fine-tuning for low-resource knowledge graph completion. Ruan Jian Xue Bao/Journal of Software, 2022, 33(10): 3531-3545(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6628.htm [doi: 10.13328/j.cnki.jos.006628]
    [27] Chen M, Zhang W, Yao Z, Chen X., Ding M, Huang F, Chen H. Meta-learning based knowledge extrapolation for knowledge graphs in the federated setting. arXiv:2205.04692, 2022.
    [28] Schlichtkrull M, Kipf TN, Bloem P, Van Den Berg R, Titov I, Welling M. Modeling relational data with graph convolutional networks. In: Proc. of the European Semantic Web Conf. 2018. 593-607.
    [29] Vashishth S, Sanyal S, Nitin V, Talukdar P. Composition-based multi-relational graph convolutional networks. In: Proc. of the Int’l Conf. on Learning Representations. 2019.
    [30] Corbett AT, Anderson JR. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User- Adapted Interaction, 1994, 4(4): 253-278.
    [31] Rabiner L, Juang B. An introduction to hidden Markov models. IEEE ASSP Magazine, 1986, 3(1): 4-16.
    [32] Pardos ZA, Heffernan NT. KT-IDEM: Introducing item difficulty to the knowledge tracing model. In: Proc. of the Int’l Conf. on User Modeling, Adaptation, and Personalization. Berlin, Heidelberg: Springer, 2011. 243-254.
    [33] Yudelson MV, Koedinger KR, Gordon GJ. Individualized Bayesian knowledge tracing models. In: Proc. of the Int’l Conf. on Artificial Intelligence in Education. Berlin, Heidelberg: Springer, 2013. 171-180.
    [34] Käser T, Klingler S, Schwing AG, Gross M. Dynamic Bayesian networks for student modeling. IEEE Trans. on Learning Technologies, 2017, 10(4): 450-462.
    [35] Minn S, Yu Y, Desmarais MC, Zhu F, Vie JJ. Deep knowledge tracing and dynamic student classification for knowledge tracing. In: Proc. of the 2018 IEEE Int’l Conf. on Data Mining (ICDM). IEEE, 2018. 1182-1187.
    [36] Abdelrahman G, Wang Q. Knowledge tracing with sequential key-value memory networks. In: Proc. of the 42nd Int’l ACM SIGIR Conf. on Research and Development in Information Retrieval. 2019. 175-184.
    [37] Zhang J, Shi X, King I, Yeung DY. Dynamic key-value memory networks for knowledge tracing. In: Proc. of the 26th Int’l Conf. on World Wide Web. 2017. 765-774.
    [38] Pandey S, Karypis G. A self-attentive model for knowledge tracing. In: Proc. of the 12th Int’l Conf. on Educational Data Mining, EDM 2019. Int’l Educational Data Mining Society, 2019. 384-389.
    [39] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L. Attention is all you need. In: Advances in Neural Information Processing Systems. 2017. 30.
    [40] Nakagawa H, Iwasawa Y, Matsuo Y. Graph-based knowledge tracing: modeling student proficiency using graph neural network. In: Proc. of the 2019 IEEE/WIC/ACM Int’l Conf. on Web Intelligence (WI). IEEE, 2019. 156-163.
    [41] Yang Y, Shen J, Qu Y, Liu Y, Wang K, Zhu Y, Zhang W, Yu Y. GIKT: A graph-based interaction model for knowledge tracing. In: Proc. of the Joint European Conf. on Machine Learning and Knowledge Discovery in Databases. Cham: Springer, 2020. 299-315.
    [42] Tong S, Liu Q, Huang W, Huang Z, Chen E, Liu C, Ma H, Wang S. Structure-based knowledge tracing: An influence propagation view. In: Proc. of the 2020 IEEE Int’l Conf. on Data Mining (ICDM). IEEE, 2020. 541-550.
    [43] Mahdisoltani F, Biega J, Suchanek F. YAGO3: A knowledge base from multilingual wikipedias. In: Proc. of the 7th Biennial Conf. on Innovative Data Systems Research. 2014.
    [44] Seo YJ, Defferrard M, Vandergheynst P, Bresson X. Structured sequence modeling with graph convolutional recurrent networks. In: Proc. of the Int’l Conf. on Neural Information Processing. 2018. 362-373.
    附中文参考文献:
    [1] 王飞, 刘井平, 刘斌, 钱铁云, 肖仰华, 彭智勇. 代码知识图谱构建及智能化软件开发方法研究. 软件学报, 2020, 31(1): 47-66. http://www.jos.org.cn/1000-9825/5893.htm [doi: 10.13328/j.cnki.jos.005893]
    [2] 刘宝珠, 王鑫, 柳鹏凯, 李思卓, 张小旺, 杨雅君. KGDB: 统一模型和语言的知识图谱数据库管理系统. 软件学报, 2021, 32(3): 781-804. http://www.jos.org.cn/1000-9825/6181.htm [doi: 10.13328/j.cnki.jos.006181]
    [26] 张宁豫, 谢辛, 陈想, 邓淑敏, 叶宏彬, 陈华钧. 基于知识协同微调的低资源知识图谱补全方法. 软件学报, 2022, 33(10): 3531-3545. http://www.jos.org.cn/1000-9825/6628.htm [doi: 10.13328/j.cnki.jos.006628]
    Related
    Cited by
Get Citation

刘康正,赵峰,金海. FS-Net: 面向时序知识图谱推理的频次统计网络.软件学报,2023,34(10):4518-4532

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:July 05,2022
  • Revised:August 18,2022
  • Online: January 13,2023
  • Published: October 06,2023
You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063