基于特征融合动态图网络的多标签文本分类算法
作者:
中图分类号:

TP18

基金项目:

国家自然科学基金(62172226, 62272230); 江苏省“双创博士”人才计划(JSSCBS20210200)


Multi-label Text Classification Method Based on Feature-fused Dynamic Graph Network
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [56]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    多标签文本分类旨在为文本分配若干预定义的标签或类别. 为了充分发掘标签间的关联, 目前的方法通常使用标签关系图并结合图神经网络获取标签特征表示. 然而, 这类方法过度依赖初始建图策略, 忽视了当前文本中固有的标签相关性, 使得分类结果更依赖于数据集统计信息, 而容易忽视当前文本段中的标签相关信息. 因此, 提出一种基于特征融合动态图网络的多标签文本分类算法, 设计动态图来建模当前文本中的标签相关性, 并结合特征融合与图神经网络, 形成基于当前文本的标签表示, 并由此形成更为准确的多标签文本结果. 随后, 设计实验进行验证, 在3个数据集实验结果表明, 所提出的模型在多标签分类任务中取得优秀的性能, 验证其有效性和可行性.

    Abstract:

    Multi-label text classification aims to assign several predefined labels or categories to text. To fully explore the correlations among labels, current methods typically utilize a label relation graph and integrate it with graph neural networks to obtain the representations of label features. However, such methods often overly rely on the initial graph construction, overlooking the inherent label correlations in the current text. Consequently, classification results heavily depend on the statistics of datasets and may overlook label-related information within the text. Therefore, this study proposes an algorithm for multi-label text classification based on feature-fused dynamic graph networks. It designs dynamic graphs to model label correlations within the current text and integrates feature fusion with graph neural networks to form label representations based on the current text, thus achieving more accurate multi-label text classifications. Experimental results on three datasets demonstrate the effectiveness and feasibility of the proposed model as it shows excellent performance in multi-label text classifications.

    参考文献
    [1] Minaee S, Kalchbrenner N, Cambria E, Nikzad N, Chenaghlu M, Gao JF. Deep learning-based text classification: A comprehensive review. ACM Computing Surveys (CSUR), 2021, 54(3): Article No. 62. [doi: 10.1145/3439726]
    [2] Tang PJ, Jiang M, Xia B, Pitera JW, Welser J, Chawla NV. Multi-label patent categorization with non-local attention-based graph convolutional network. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. New York: AAAI, 2020. 9024–9031.
    [3] Jain H, Prabhu Y, Varma M. Extreme multi-label loss functions for recommendation, tagging, ranking & other missing label applications. In: Proc. of the 22nd ACM SIGKDD Int’l Conf. on Knowledge Discovery and Data Mining. San Francisco: ACM, 2016. 935–944. [doi: 10.1145/2939672.2939756]
    [4] Fiallos A, Jimenes K. Using reddit data for multi-label text classification of Twitter users interests. In: Proc. of the 6th Int’l Conf. on eDemocracy & eGovernment (ICEDEG). Quito: IEEE, 2019. 324–327. [doi: 10.1109/ICEDEG.2019.8734365]
    [5] Wang YQ, Feng S, Wang DL, Yu G, Zhang YF. Multi-label Chinese microblog emotion classification via convolutional neural network. In: Proc. of the 18th Asia-Pacific Web Conf.—Web Technologies and Applications. Suzhou: Springer, 2016. 567–580. [doi: 10.1007/978-3-319-45814-4_46]
    [6] Vu HT, Nguyen MT, Nguyen VC, Tien MT, Nguyen, VH. Label correlation based graph convolutional network for multi-label text classification. In: Proc. of the 2022 Int’l Joint Conf. on Neural Networks (IJCNN). Padua: IEEE, 2022. 1–8.
    [7] Liu J, Chang WC, Wu Y, Yang Y. Deep learning for extreme multi-label text classification. In: Proc. of the 40th Int’l ACM SIGIR Conf. on Research and Development in Information Retrieval. Shinjuku: ACM, 2017. 115–124. [doi: 10.1145/3077136.308083]
    [8] Wei X, Huang JB, Zhao R, Yu H, Xu Z. Multi-label text classification model based on multi-level constraint augmentation and label association attention. ACM Trans. on Asian and Low-resource Language Information Processing, 2024, 23(1): 14.
    [9] Luaces O, Díez J, Barranquero J, del Coz JJ, Bahamonde A. Binary relevance efficacy for multilabel classification. Progress in Artificial Intelligence, 2012, 1(4): 303–313.
    [10] Zhang ML, Zhou ZH. Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans. on Knowledge and Data Engineering, 2006, 18(10): 1338–1351.
    [11] Wu YJ, Li J, Wu J, Chang J. Siamese capsule networks with global and local features for text classification. Neurocomputing, 2020, 390: 88–98.
    [12] Hu LM, Yang TC, Shi C, Ji HY, Li XL. Heterogeneous graph attention networks for semi-supervised short text classification. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int’l Joint Conf. on Natural Language Processing (EMNLP-IJCNLP). Hong Kong: ACL, 2019. 4821–4830. [doi: 10.18653/v1/D19-1488]
    [13] 杜晓宇, 陈正, 项欣光. 基于解耦图神经网络的可解释标签感知推荐算法. 软件学报, 2023, 34(12): 5670–5685. http://www.jos.org.cn/1000-9825/6754.htm
    Du XY, Chen Z, Xiang XG. Explainable tag-aware recommendation based on disentangled graph neural network. Ruan Jian Xue Bao/Journal of Software, 2023, 34(12): 5670–5685 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6754.htm
    [14] Clare A, King RD. Knowledge discovery in multi-label phenotype data. In: Proc. of the 5th European Conf. on Principles of Data Mining and Knowledge Discovery. Freiburg: Springer, 2001. 42–53. [doi: 10.1007/3-540-44794-6_4]
    [15] Elisseeff A, Weston J. A kernel method for multi-labelled classification. In: Proc. of the 14th Int’l Conf. on Neural Information Processing Systems. Vancouver: MIT Press, 2001. 681–687.
    [16] Zhang ML, Zhou ZH. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognition, 2007, 40(7): 2038–2048.
    [17] Albawi S, Mohammed TA, Al-Zawi S. Understanding of a convolutional neural network. In: Proc. of the 2017 Int’l Conf. on Engineering and Technology (ICET). Antalya: IEEE, 2017. 1–6. [doi: 10.1109/ICEngTechnol.2017.8308186]
    [18] Pouyanfar S, Sadiq S, Yan YL, Tian HM, Tao YD, Reyes MP, Shyu ML, Chen SC, Iyengar SS. A survey on deep learning: Algorithms, techniques, and applications. ACM Computing Surveys (CSUR), 2018, 51(5): 92.
    [19] 李芳芳, 苏朴真, 段俊文, 张师超, 毛星亮. 多粒度信息关系增强的多标签文本分类. 软件学报, 2023, 34(12): 5686–5703. http://www.jos.org.cn/1000-9825/6802.htm
    Li FF, Su PZ, Duan JW, Zhang SC, Mao XL. Multi-label text classification with enhancing multi-granularity information relations. Ruan Jian Xue Bao/Journal of Software, 2023, 34(12): 5686–5703 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6802.htm
    [20] He JD, Xu BW, Yang Z, Han D, Yang CR, Lo D. PTM4Tag: Sharpening tag recommendation of stack overflow posts with pre-trained models. In: Proc. of the 30th IEEE/ACM Int’l Conf. on Program Comprehension. ACM, 2022. 1–11.
    [21] Du XY, Wu ZK, Feng FL, He XN, Tang JH. Invariant representation learning for multimedia recommendation. In: Proc. of the 30th ACM Int’l Conf. on Multimedia. Lisboa: ACM, 2022. 619–628. [doi: 10.1145/3503161.3548405]
    [22] Du XY, Wang X, He XN, Li ZC, Tang JH Chua TS. How to learn item representation for cold-start multimedia recommendation? In: Proc. of the 28th ACM Int’l Conf. on Multimedia. Seattle: ACM, 2020. 3469–3477. [doi: 10.1145/3394171.3413628]
    [23] Kim Y. Convolutional neural networks for sentence classification. In: Proc. of the 2014 Conf. on Empirical Methods in Natural Language Processing (EMNLP). Doha: ACL, 2014. 1746–1751. [doi: 10.3115/v1/D14-1181]
    [24] Dieng AB, Wang C, Gao JF, Paisley J. TopicRNN: A recurrent neural network with long-range semantic dependency. In: Proc. of the 5th Int’l Conf. on Learning Representations. Toulon: OpenReview.net, 2017.
    [25] Yang ZC, Yang DY, Dyer C, He XD, Smola A, Hovy E. Hierarchical attention networks for document classification. In: Proc. of the 2016 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego: ACL, 2016. 1480–1489. [doi: 10.18653/v1/N16-1174]
    [26] 肖琳, 陈博理, 黄鑫, 刘华锋, 景丽萍, 于剑. 基于标签语义注意力的多标签文本分类. 软件学报, 2020, 31(4): 1079–1089. http://www.jos.org.cn/1000-9825/5923.htm
    Xiao L, Chen BL, Huang X, Liu HF, Jing LP, Yu J. Multi-label text classification method based on label semantic information. Ruan Jian Xue Bao/Journal of Software, 2020, 31(4): 1079–1089 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5923.htm
    [27] Read J, Pfahringer B, Holmes G, Frank E. Classifier chains for multi-label classification. In: Proc. of the 2009 European Conf. on Machine Learning and Knowledge Discovery in Databases. Bled: Springer, 2009. 254–269. [doi: 10.1007/978-3-642-04174-7_17]
    [28] Nam J, Loza Mencía E, Kim HJ, Fürnkranz J. Maximizing subset accuracy with recurrent neural networks in multi-label classification. In: Proc. of the 30th Conf. on Neural Information Processing Systems. Long Beach, 2017. 30
    [29] Ozmen M, Zhang H, Wang PY, Coates M. Multi-relation message passing for multi-label text classification. In: Proc. of the 2022 IEEE Int’l Conf. on Acoustics, Speech and Signal Processing (ICASSP). Singapore: IEEE, 2022. 3583–3587. [doi: 10.1109/ICASSP43922.2022.9747225]
    [30] Vashishth S, Sanyal S, Nitin V, Talukdar P. Composition-based multi-relational graph convolutional networks. In: Proc. of the 8th Int’l Conf. on Learning Representations. Addis Ababa: OpenReview.net, 2020.
    [31] 李博涵, 向宇轩, 封顶, 何志超, 吴佳骏, 戴天伦, 李静. 融合知识感知与双重注意力的短文本分类模型. 软件学报, 2022, 33(10): 3565–3581. http://www.jos.org.cn/1000-9825/6630.htm
    Li BH, Xiang YX, Feng D, He ZC, Wu JJ, Dai TL, Li J. Short text classification model combining knowledge aware and dual attention. Ruan Jian Xue Bao/Journal of Software, 2022, 33(10): 3565–3581 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6630.htm
    [32] Guo L, Zhang DX, Wang L, Wang H, Cui B. CRAN: A hybrid CNN-RNN attention-based model for text classification. In: Proc. of the 37th Int’l Conf. on Conceptual Modeling. Xi’an: Springer, 2018. 571–585. [doi: 10.1007/978-3-030-00847-5_42]
    [33] Zhou P, Shi W, Tian J, Qi ZY, Li BC, Hao HW, Xu B. Attention-based bidirectional long short-term memory networks for relation classification. In: Proc. of the 54th Annual Meeting of the Association for Computational Linguistics (Vol. 2: Short Papers). Berlin: ACL, 2016. 207–212. [doi: 10.18653/v1/P16-2034]
    [34] Kipf TN, Welling M. Semi-supervised classification with graph convolutional networks. In: Proc. of the 5th Int’l Conf. on Learning Representations. Toulon: OpenReview.net, 2017.
    [35] Veli?kovi? P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. In: Proc. of the 6th Int’l Conf. on Learning Representations. Vancouver: OpenReview.net, 2018.
    [36] Liu B. GCN-BERT and memory network based multi-label classification for event text of the Chinese government hotline. IEEE Access, 2022, 10: 109267–109276.
    [37] Pal A, Selvakumar M, Sankarasubbu M. Multi-label text classification using attention-based graph neural network. arXiv:2003.11644, 2020.
    [38] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I. Attention is all you need. In: Proc. of the 31st Int’l Conf. on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 6000–6010.
    [39] He XN, Deng K, Wang X, Li Y, Zhang YD, Wang M. LightGCN: Simplifying and powering graph convolution network for recommendation. In: Proc. of the 43rd Int’l ACM SIGIR Conf. on Research and Development in Information Retrieval. ACM, 2020. 639–648.
    [40] Ma QW, Yuan CY, Zhou W, Hu SL. Label-specific dual graph neural network for multi-label text classification. In: Proc. of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int’l Joint Conf. on Natural Language Processing. ACL, 2021. 3855–3864.
    [41] Ye J, He JJ, Peng XJ, Wu WH, Qiao Y. Attention-driven dynamic graph convolutional network for multi-label image recognition. In: Proc. of the 16th European Conf. on Computer Vision. Glasgow: Springer, 2020. 649–665. [doi: 10.1007/978-3-030-58589-1_39]
    [42] Katakis I, Tsoumakas G, Vlahavas I. Multilabel text classification for automated tag suggestion. In: Proc. of the 2008 ECML/PKDD Discovery Challenge. Antwerp: ECML, 2008. 1–9.
    [43] Lewis DD, Yang YM, Rose T G, Li F. RCV1: A new benchmark collection for text categorization research. The Journal of Machine Learning Research, 2004, 5: 361–397.
    [44] Yang PC, Sun X, Li W, Ma SM, Wu W, Wang HF. SGM: Sequence generation model for multi-label classification. In: Proc. of the 27th Int’l Conf. on Computational Linguistics. Santa Fe: ACL, 2018. 3915–3926.
    [45] Zhang ML, Zhou ZH. A review on multi-label learning algorithms. IEEE Trans. on Knowledge and Data Engineering, 2014, 26(8): 1819–1837.
    [46] Yang PC, Luo FL, Ma SM, Lin JY, Sun X. A deep reinforced sequence-to-set model for multi-label classification. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 5252–5258. [doi: 10.18653/v1/P19-1518]
    [47] Boutell MR, Luo JB, Shen XP, Brown CM. Learning multi-label scene classification. Pattern Recognition, 2004, 37(9): 1757–1771.
    [48] Chen GB, Ye DH, Xing ZC, Chen JS, Cambria E. Ensemble application of convolutional and recurrent neural networks for multi-label text categorization. In: Proc. of the 2017 Int’l Joint Conf. on Neural Networks (IJCNN). Anchorage: IEEE, 2017. 2377–2383.
    [49] Wang R, Ridley R, Su XA, Qu WG, Dai XY. A novel reasoning mechanism for multi-label text classification. Information Processing & Management, 2021, 58(2): 102441.
    [50] Zhang XM, Zhang QW, Yan Z, Liu RF, Cao YB. Enhancing label correlation feedback in multi-label text classification via multi-task learning. In: Proc. of the 2021 Findings of the Association for Computational Linguistics (ACL-IJCNLP 2021). ACL, 2021. 1190–1200.
    [51] Yan YY, Liu F, Zhuang XQ, Ju J. An R-Transformer_BiLSTM model based on attention for multi-label text classification. Neural Processing Letters, 2023, 55(2): 1293–1316.
    [52] Kingma DP, Ba J. Adam: A method for stochastic optimization. In: Proc. of the 3rd Int’l Conf. on Learning Representations. San Diego, 2015. 6.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

黄靖,陶竹林,杜晓宇,项欣光.基于特征融合动态图网络的多标签文本分类算法.软件学报,2025,36(7):3239-3252

复制
分享
文章指标
  • 点击次数:185
  • 下载次数: 636
  • HTML阅读次数: 0
  • 引用次数: 0
历史
  • 收稿日期:2023-11-09
  • 最后修改日期:2024-01-04
  • 在线发布日期: 2024-12-11
文章二维码
您是第20045805位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号