Complex Entity Recognition Based on Prior Semantic Knowledge and Type Embedding
Author:
Affiliation:

Clc Number:

TP18

  • Article
  • | |
  • Metrics
  • |
  • Reference [47]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    Entity recognition is a key task of information extraction. With the development of information extraction technology, researchers turn the research direction from the recognition of simple entities to the recognition of complex ones. Complex entities usually have no explicit features, and they are more complicated in syntactic constructions and parts of speech, which makes the recognition of complex entities a great challenge. In addition, existing models widely use span-based methods to identify nested entities. As a result, they always have an ambiguity in the detection of entity boundaries, which affects recognition performance. In response to the above challenge and problem, this study proposes an entity recognition model GIA-2DPE based on prior semantic knowledge and type embedding. The model uses keyword sequences of entity categories as prior semantic knowledge to improve the cognition of entities, utilizes type embedding to capture potential features of different entity types, and then combines prior knowledge with entity-type features through the gated interactive attention mechanism to assist in the recognition of complex entities. Moreover, the model uses 2D probability encoding to predict entity boundaries and combines boundary features and contextual features to enhance accurate boundary detection, thereby improving the performance of nested entity recognition. This study conducts extensive experiments on seven English datasets and two Chinese datasets. The results show that GIA-2DPE outperforms state-of-the-art models and achieves a 10.4% F1 boost compared with the baseline in entity recognition tasks on the ScienceIE dataset.

    Reference
    [1] Eberts M, Ulges A. Span-based joint entity and relation extraction with transformer pre-training. In: Proc. of the 24th European Conf. on Artificial Intelligence. Santiago de Compostela: IOS Press, 2020. 2006–2013.
    [2] Friedrich A, Adel H, Tomazic F, Hingerl J, Benteau R, Marusczyk A, Lange L. The SOFC-Exp corpus and neural approaches to information extraction in the materials science domain. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 2020. 1255–1268.
    [3] Li XY, Yin F, Sun ZJ, Li XY, Yuan A, Chai D, Zhou MX, Li JW. Entity-relation extraction as multi-turn question answering. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 1340–1350.
    [4] Sahrawat D, Mahata D, Zhang HM, Kulkarni M, Sharma A, Gosangi R, Stent A, Kumar Y, Shah RR, Zimmermann R. Keyphrase extraction as sequence labeling using contextualized embeddings. In: Proc. of the 42nd European Conf. on Information Retrieval. Lisbon: Springer, 2020. 328–335.
    [5] Luan Y, He LH, Ostendorf M, Hajishirzi H. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Brussels: ACL, 2018. 3219–3232.
    [6] Gurulingappa H, Rajput AM, Roberts A, Fluck J, Hofmann-Apitius M, Toldo L. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. Journal of Biomedical Informatics, 2012, 45(5): 885–892. [doi: 10.1016/j.jbi.2012.04.008
    [7] Walker C, Strassel S, Medero J, Maeda K. ACE 2005 Multilingual Training Corpus. Philadelphia: Linguistic Data Consortium. 2006.
    [8] Augenstein I, Das M, Riedel S, Vikraman L, McCallum A. SemEval 2017 task 10: ScienceIE—Extracting keyphrases and relations from scientific publications. In: Proc. of the 11th Int’l Workshop on Semantic Evaluation. Vancouver: ACL, 2017. 546–555.
    [9] Doddington GR, Mitchell A, Przybocki MA, Ramshaw LA, Strassel SM, Weischedel RM. The automatic content extraction (ACE) program-tasks, data, and evaluation. In: Proc. of the 4th Int’l Conf. on Language Resources and Evaluation. Lisbon: European Language Resources Association, 2004.
    [10] Ohta T, Tateisi Y, Kim JD. The GENIA corpus: An annotated research abstract corpus in molecular biology domain. In: Proc. of the 2nd Int’l Conf. on Human Language Technology Research. San Diego: Morgan Kaufmann Publishers Inc., 2002. 82–86.
    [11] Tan CQ, Qiu W, Chen MS, Wang R, Huang F. Boundary enhanced neural span classification for nested named entity recognition. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. Palo Alto: AAAI Press, 2020. 9016–9023.
    [12] Li J, Sun AX, Han JL, Li CL. A survey on deep learning for named entity recognition. IEEE Trans. on Knowledge and Data Engineering, 2022, 34(1): 50–70. [doi: 10.1109/TKDE.2020.2981314
    [13] Beltagy I, Lo K, Cohan A. SciBERT: A pretrained language model for scientific text. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int’l Joint Conf. on Natural Language Processing. Hong Kong: ACL, 2019. 3615–3620.
    [14] Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis: ACL, 2019. 4171–4186.
    [15] Lai T, Bui T, Kim DS, Tran QH. A joint learning approach based on self-distillation for keyphrase extraction from scientific documents. In: Proc. of the 28th Int’l Conf. on Computational Linguistics. Barcelona: ACL, 2020. 649–656.
    [16] Jain S, van Zuylen M, Hajishirzi H, Beltagy I. SciREX: A challenge dataset for document-level information extraction. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 2020. 7506–7516.
    [17] Shibuya T, Hovy E. Nested named entity recognition via second-best sequence learning and decoding. Trans. of the Association for Computational Linguistics, 2020, 8: 605–620. [doi: 10.1162/tacl_a_00334
    [18] Huang HY, Lei M, Feng C. Hypergraph network model for nested entity mention recognition. Neurocomputing, 2021, 423: 200–206. [doi: 10.1016/j.neucom.2020.09.077
    [19] Li F, Wang Z, Hui SC, Liao LJ, Zhu XH, Huang HY. A segment enhanced span-based model for nested named entity recognition. Neurocomputing, 2021, 465: 26–37. [doi: 10.1016/j.neucom.2021.08.094
    [20] Wang YJ, Sun CZ, Wu YB, Yan JC, Gao P, Xie GT. Pre-training entity relation encoder with intra-span and inter-span information. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing. ACL, 2020. 1692–1705.
    [21] Shen YL, Ma XY, Tan ZQ, Zhang S, Wang W, Lu WM. Locate and label: A two-stage identifier for nested named entity recognition. In: Proc. of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int’l Joint Conf. on Natural Language Processing. ACL, 2021. 2782–2794.
    [22] Sennrich R, Haddow B, Birch A. Neural machine translation of rare words with subword units. In: Proc. of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin: ACL, 2016. 1715–1725.
    [23] He PC, Liu XD, Gao JF, Chen WZ. DeBERTa: Decoding-enhanced BERT with disentangled attention. In: Proc. of the 9th Int’l Conf. on Learning Representations. OpenReview.net, 2021.
    [24] Hendrycks D, Gimpel K. Gaussian error linear units (GELUs). arXiv:1606.08415, 2016.
    [25] Li XY, Feng JR, Meng YX, Han QH, Wu F, Li JW. A unified MRC framework for named entity recognition. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 2020. 5849–5859.
    [26] Wang YJ, Sun CZ, Wu YB, Zhou H, Li L, Yan JC. ENPAR: Enhancing entity and entity pair representations for joint entity relation extraction. In: Proc. of the 16th Conf. of the European Chapter of the Association for Computational Linguistics. ACL, 2021. 2877–2887.
    [27] Zhong ZX, Chen DQ. A frustratingly easy approach for entity and relation extraction. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL, 2021. 50–61.
    [28] Wan Q, Wei LN, Chen XH, Liu J. A region-based hypergraph network for joint entity-relation extraction. Knowledge-based Systems, 2021, 228: 107298. [doi: 10.1016/j.knosys.2021.107298
    [29] Garg A, Kagi SS, Singh M. SEAL: Scientific keyphrase extraction and classification. In: Proc. of the 2020 ACM/IEEE Joint Conf. on Digital Libraries. ACM, 2020. 527–528.
    [30] Chernyavskiy A, Ilvovsky D, Nakov P. Transformers: “The end of history” for Natural Language Processing? In: Proc. of the 2021 Joint European Conf. on Machine Learning and Knowledge Discovery in Databases. Bilbao: Springer, 2021. 677–693.
    [31] Jiang D, Ren HP, Cai Y, Xu JY, Liu YX, Leung HF. Candidate region aware nested named entity recognition. Neural Networks, 2021, 142: 340–350. [doi: 10.1016/j.neunet.2021.02.019
    [32] Fei H, Ren YF, Ji DH. Dispatched attention with multi-task learning for nested mention recognition. Information Sciences, 2020, 513: 241–251. [doi: 10.1016/j.ins.2019.10.065
    [33] Li CG, Wang GH, Cao J, Cai Y. A multi-agent communication based model for nested named entity recognition. IEEE/ACM Trans. on Audio, Speech, and Language Processing, 2021, 29: 2123–2136. [doi: 10.1109/TASLP.2021.3086978
    [34] Straková J, Straka M, Hajic J. Neural architectures for nested NER through linearization. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 5326–5331.
    [35] Fu Y, Tan CQ, Chen MS, Huang SF, Huang F. Nested named entity recognition with partially-observed TreeCRFs. In: Proc. of the 35th AAAI Conf. on Artificial Intelligence. AAAI Press, 2021. 12839–12847.
    [36] Tan ZQ, Shen YL, Zhang S, Lu WM, Zhuang YT. A sequence-to-set network for nested named entity recognition. In: Proc. of the 30th Int’l Joint Conf. on Artificial Intelligence. Montreal: ijcai.org, 2021. 3936–3942.
    [37] Zhao TY, Yan Z, Cao YB, Li ZJ. Asking effective and diverse questions: A machine reading comprehension based framework for joint entity-relation extraction. In: Proc. of the 29th Int’l Joint Conf. on Artificial Intelligence. Yokohama: ijcai.org, 2020. 3948–3954.
    [38] Kong LY, Lai QH, Liu S. End-to-end drug entity recognition and adverse effect relation extraction via principal neighbourhood aggregation network. Journal of Physics: Conference Series, 2021, 1848: 012110. [doi: 10.1088/1742-6596/1848/1/012110
    [39] Zhao S, Hu MH, Cai ZP, Liu F. Modeling dense cross-modal interactions for joint entity-relation extraction. In: Proc. of the 29th Int’l Joint Conf. on Artificial Intelligence. 2020. 4032–4038.
    [40] Giorgi J, Wang XD, Sahar N, Shin WY, Bader GD, Wang B. End-to-end named entity recognition and relation extraction using pre-trained language models. arXiv:1912.13415, 2019.
    [41] Wang J, Lu W. Two are better than one: Joint entity and relation extraction with table-sequence encoders. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing. ACL, 2020. 1706–1721.
    [42] Shen YL, Ma XY, Tang YC, Lu WM. A trigger-sense memory flow framework for joint entity and relation extraction. In: Proc. of the 2021 Web Conf. Ljubljana: ACM, 2021. 1704–1715.
    [43] Ji B, Yu J, Li SS, Ma J, Wu QB, Tan YS, Liu HJ. Span-based joint entity and relation extraction with attention-based span-specific and contextual semantic representations. In: Proc. of the 28th Int’l Conf. on Computational Linguistics. Barcelona: Int’l Committee on Computational Linguistics, 2020. 88–99.
    [44] Lai T, Ji H, Zhai CX, Tran QH. Joint biomedical entity and relation extraction with knowledge-enhanced collective inference. In: Proc. of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int’l Joint Conf. on Natural Language Processing. ACL, 2021. 6248–6260.
    [45] Zhang Y, Yang J. Chinese NER using lattice LSTM. In: Proc. of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne: ACL, 2018. 1554–1564.
    [46] Meng YX, Wu W, Wang F, Li XY, Nie P, Yin F, Li MY, Han QH, Sun XF, Li JW. Glyce: Glyph-vectors for Chinese character representations. In: Proc. of the 33rd Int’l Conf. on Neural Information Processing Systems. Vancouver, 2019. 247.
    [47] Zheng HY, Qin B, Xu M. Chinese medical named entity recognition using CRF-MT-Adapt and NER-MRC. In: Proc. of the 2nd Int’l Conf. on Computing and Data Science. Stanford: IEEE, 2021. 362–365.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

姜小波,何昆,阎广瑜.基于语义先验知识与类型嵌入的复杂实体识别.软件学报,2023,34(12):5649-5669

Copy
Share
Article Metrics
  • Abstract:763
  • PDF: 2452
  • HTML: 1035
  • Cited by: 0
History
  • Received:December 02,2021
  • Revised:February 25,2022
  • Online: February 15,2023
  • Published: December 06,2023
You are the first2032375Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063