基于预训练模型的用户评分预测
作者:
中图分类号:

TP18


Prediction of User Rating Based on Pre-trained Model
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [53]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    随着商家评论网站的快速发展, 推荐系统所带来的效率提升使得评分预测成为近年来新兴研究任务之一. 现有的评分预测方法通常局限于协同过滤算法以及各类神经网络模型, 并没有充分利用目前预训练模型提前学习的丰富的语义知识. 针对此问题, 提出一种基于预训练语言模型的个性化评分预测方法, 其通过分析用户和商家的历史评论, 为用户在消费前提供评分预测作为参考. 该方法首先设计一项预训练任务, 让模型学习捕捉文本中的关键信息. 其次, 通过细粒度情感分析方法对评论文本进行处理, 从而获取评论文本中的属性词. 接下来, 设计一个属性词嵌入层将上述外部领域知识融入模型中. 最后, 采用基于注意力机制的信息融合策略, 将输入文本的全局和局部语义信息进行融合. 实验结果表明, 该方法相较于基准模型, 在两个自动评价指标上均取得显著的提升.

    Abstract:

    As merchant review websites develop rapidly, the efficiency improvement brought by recommender systems makes rating prediction one of the emerging research tasks in recent years. Existing rating prediction methods are usually limited to collaborative filtering algorithms and various types of neural network models, and do not take full advantage of the rich semantic knowledge learned in advance by the current pre-trained models. To address this problem, this study proposes a personalized rating prediction method based on pre-trained language models. The method analyzes the historical reviews of users and merchants to provide users with rating predictions as a reference before consumption. It first designs a pre-training task for the model to learn to capture key information in the text. Next, the review text is processed by a fine-grained sentiment analysis method to obtain aspect terms in the review text. Subsequently, the method designs an aspect term embedding layer to incorporate the aforementioned external domain knowledge into the model. Finally, it utilizes an information fusion strategy based on the attention mechanism to fuse the global and local semantic information of the input text. The experimental results show that the method achieves significant improvement in both automatic evaluation metrics compared to the benchmark models.

    参考文献
    [1] Koren Y, Bell R, Volinsky C. Matrix factorization techniques for recommender systems. Computer, 2009, 42(8): 30–37.
    [2] Pazzani MJ, Billsus D. Content-based recommendation systems. In: Brusilovsky P, Kobsa A, Nejdl W, eds. The Adaptive Web. Berlin: Springer, 2007. 325–341. [doi: 10.1007/978-3-540-72079-9_10]
    [3] McAuley J, Leskovec J. Hidden factors and hidden topics: Understanding rating dimensions with review text. In: Proc. of the 7th ACM Conf. on Recommender Systems. Hong Kong: ACM, 2013. 165–172. [doi: 10.1145/2507157.2507163]
    [4] Koren Y. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In: Proc. of the 14th ACM SIGKDD Int’l Conf. on Knowledge Discovery and Data Mining. Las Vegas: ACM, 2008. 426–434. [doi: 10.1145/1401890.1401944]
    [5] Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. The Journal of Machine Learning Research, 2003, 3: 993–1022.
    [6] Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional Transformers for language understanding. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis: ACL, 2018. 4171–4186. [doi: 10.18653/v1/N19-1423]
    [7] Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou YQ, Li W, Liu PJ. Exploring the limits of transfer learning with a unified text-to-text Transformer. The Journal of Machine Learning Research, 2020, 21(1): 140.
    [8] Zhu GH, Lu W, Yuan CF, Huang YH. AdaMCL: Adaptive fusion multi-view contrastive learning for collaborative filtering. In: Proc. of the 46th Int’l ACM SIGIR Conf. on Research and Development in Information Retrieval. Taipei: ACM, 2023. 1076–1085. [doi: 10.1145/3539618.3591632]
    [9] Choi J, Hong S, Park N, Cho SB. Blurring-sharpening process models for collaborative filtering. In: Proc. of the 46th Int’l ACM SIGIR Conf. on Research and Development in Information Retrieval. Taipei: ACM, 2023. 1096–1106. [doi: 10.1145/3539618.3591645]
    [10] Ren XB, Xia LH, Zhao JS, Yin DW, Huang C. Disentangled contrastive collaborative filtering. In: Proc. of the 46th Int’l ACM SIGIR Conf. on Research and Development in Information Retrieval. Taipei: ACM, 2023. 1137–1146. [doi: 10.1145/3539618.3591665]
    [11] Ganu G, Elhadad N, Marian A. Beyond the stars: Improving rating predictions using review text content. In: Proc. of the 12th Int’l Workshop on the Web and Databases. Providence: WebDB, 2009. 1–6.
    [12] Zhang W, Yuan Q, Han JW, Wang JY. Collaborative multi-level embedding learning from reviews for rating prediction. In: Proc. of the 25th Int’l Joint Conf. on Artificial Intelligence. New York: AAAI, 2016. 2986–2992.
    [13] Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. arXiv:1301.3781, 2013.
    [14] Zheng L, Noroozi V, Yu PS. Joint deep modeling of users and items using reviews for recommendation. In: Proc. of the 10th ACM Int’l Conf. on Web Search and Data Mining. Cambridge: ACM, 2017. 425–434. [doi: 10.1145/3018661.3018665]
    [15] Catherine R, Cohen W. TransNets: Learning to transform for recommendation. In: Proc. of the 11th ACM Conf. on Recommender Systems. Como: ACM, 2017. 288–296. [doi: 10.1145/3109859.3109878]
    [16] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proc. of the 31st Int’l Conf. on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 6000–6010.
    [17] Shuai J, Zhang K, Wu L, Sun PJ, Hong RC, Wang M, Li Y. A review-aware graph contrastive learning framework for recommendation. In: Proc. of the 45th Int’l ACM SIGIR Conf. on Research and Development in Information Retrieval. Madrid: ACM, 2022. 1283–1293. [doi: 10.1145/3477495.3531927]
    [18] Yang W, Huo TF, Liu ZQ, Lu C. Review-based multi-intention contrastive learning for recommendation. In: Proc. of the 46th Int’l ACM SIGIR Conf. on Research and Development in Information Retrieval. Taipei: ACM, 2023. 2339–2343.
    [19] Pang B, Lee L, Vaithyanathan S. Thumbs up? Sentiment classification using machine learning techniques. In: Proc. of the 2002 Conf. on Empirical Methods in Natural Language Processing. ACL, 2002. 79–86.
    [20] Yang ZC, Yang DY, Dyer C, He XD, Smola A, Hovy E. Hierarchical attention networks for document classification. In: Proc. of the 2016 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego: ACL, 2016. 1480–1489. [doi: 10.18653/v1/N16-1174]
    [21] 王家乾, 龚子寒, 薛云, 庞士冠, 古东宏. 基于混合多头注意力和胶囊网络的特定目标情感分析. 中文信息学报, 2020, 34(5): 100–110.
    Wang JQ, Gong ZH, Xue Y, Pang SG, Gu DH. Aspect-based sentiment analysis based on hybrid multi-head attention and capsule networks. Journal of Chinese Information Processing, 2020, 34(5): 100–110 (in Chinese with English abstract).
    [22] 曾锋, 曾碧卿, 韩旭丽, 张敏, 商齐. 基于双层注意力循环神经网络的方面级情感分析. 中文信息学报, 2019, 33(6): 108–115.
    Zeng F, Zeng BQ, Han XL, Zhang M, Shang Q. Double attention neural network for aspect-based sentiment analysis. Journal of Chinese Information Processing, 2019, 33(6): 108–115 (in Chinese with English abstract).
    [23] 冯超, 黎海辉, 赵洪雅, 薛云, 唐婧尧. 基于层次注意力机制和门机制的属性级别情感分析. 中文信息学报, 2021, 35(10): 128–136.
    Feng C, Li HH, Zhao HY, Xue Y, Tang JY. Aspect-level sentiment analysis based on hierarchical attention and gate networks. Journal of Chinese Information Processing, 2021, 35(10): 128–136 (in Chinese with English abstract).
    [24] Tang DY, Qin B, Feng XC, Liu T. Effective LSTMs for target-dependent sentiment classification. In: Proc. of the 26th Int’l Conf. on Computational Linguistics: Technical Papers. Osaka: ACL, 2015. 3298–3307.
    [25] Song YW, Wang JH, Jiang T, Liu ZY, Rao YH. Targeted sentiment classification with attentional encoder network. In: Proc. of the 28th Int’l Conf. on Artificial Neural Networks. Munich: Springer, 2019. 93–103. [doi: 10.1007/978-3-030-30490-4_9]
    [26] Mao Y, Shen Y, Yu C, Cai LJ. A joint training dual-MRC framework for aspect based sentiment analysis. In: Proc. of the 35th AAAI Conf. on Artificial Intelligence. AAAI, 2021. 13543–13551.
    [27] Zeng BQ, Yang H, Xu RY, Zhou W, Han XL. LCF: A local context focus mechanism for aspect-based sentiment classification. Applied Sciences, 2019, 9(16): 3389.
    [28] Peng HY, Xu L, Bing LD, Huang F, Lu W, Si L. Knowing what, how and why: A near complete solution for aspect-based sentiment analysis. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. New York: AAAI, 2020. 8600–8607.
    [29] Wan H, Yang YF, Du JF, Liu YN, Qi KX, Pan JZ. Target-aspect-sentiment joint detection for aspect-based sentiment analysis. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. New York: AAAI, 2020. 9122–9129. [doi: 10.1609/aaai.v34i05.6447]
    [30] Cai HJ, Xia R, Yu JF. Aspect-category-opinion-sentiment quadruple extraction with implicit aspects and opinions. In: Proc. of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int’l Joint Conf. on Natural Language Processing (Vol. 1: Long Papers). ACL, 2021. 340–350.
    [31] Chen H, Zhai ZP, Feng FX, Li RF, Wang XJ. Enhanced multi-channel graph convolutional network for aspect sentiment triplet extraction. In: Proc. of the 60th Annual Meeting of the Association for Computational Linguistics (Vol. 1: Long Papers). Dublin: ACL, 2022. 2974–2985. [doi: 10.18653/v1/2022.acl-long.212]
    [32] Zhang WX, Deng Y, Li X, Yuan YF, Bing LD, Lam W. Aspect sentiment quad prediction as paraphrase generation. In: Proc. of the 2021 Conf. on Empirical Methods in Natural Language Processing. ACL, 2021. 9209–9219.
    [33] Bao XY, Wang ZQ, Jiang XT, Xiao R, Li SS. Aspect-based sentiment analysis with opinion tree generation. In: Proc. of the 31st Int’l Joint Conf. on Artificial Intelligence. Vienna: IJCAI, 2022. 4044–4050.
    [34] Zhao SM, Chen W, Wang TJ. Learning cooperative interactions for multi-overlap aspect sentiment triplet extraction. In: Proc. of the 2022 Findings of the Association for Computational Linguistics (EMNLP 2022). Abu Dhabi: ACL, 2022. 3337–3347.
    [35] Pang B, Lee L. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In: Proc. of the 43rd Annual Meeting of the Association for Computational Linguistics. Ann Arbor: ACL, 2005. 115–124. [doi: 10.3115/1219840.1219855]
    [36] Qu LZ, Ifrim G, Weikum G. The bag-of-opinions method for review rating prediction from sparse text patterns. In: Proc. of the 23rd Int’l Conf. on Computational Linguistics (Coling 2010). Beijing: ACL, 2010. 913–921.
    [37] Lu Y, Kong XF, Quan XJ, Liu WY, Xu YL. Exploring the sentiment strength of user reviews. In: Proc. of the 11th Int’l Conf. on Web-age Information Management. Jiuzhaigou: Springer, 2010. 471–482. [doi: 10.1007/978-3-642-14246-8_46]
    [38] Li JJ, Yang HT, Zong CQ. Document-level multi-aspect sentiment classification by jointly modeling users, aspects, and overall ratings. In: Proc. of the 27th Int’l Conf. on Computational Linguistics. Santa Fe: ACL, 2018. 925–936.
    [39] Wu CH, Wu FZ, Liu JX, Huang YF, Xie X. ARP: Aspect-aware neural review rating prediction. In: Proc. of the 28th ACM Int’l Conf. on Information and Knowledge Management. Beijing: ACM, 2019. 2169–2172. [doi: 10.1145/3357384.3358086]
    [40] Bu JH, Ren L, Zheng S, Yang Y, Wang JG, Zhang FZ, Wu W. ASAP: A Chinese review dataset towards aspect category sentiment analysis and rating prediction. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL, 2021. 2069–2079.
    [41] Hu MQ, Liu B. Mining opinion features in customer reviews. In: Proc. of the 19th National Conf. on Artificial Intelligence. San Jose: AAAI. 2004. 755–760.
    [42] Dai HL, Song YQ. Neural aspect and opinion term extraction with mined rules as weak supervision. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 5268–5277. [doi: 10.18653/v1/P19-1520]
    [43] Kingma DP, Ba J. Adam: A method for stochastic optimization. In: Proc. of the 3rd Int’l Conf. on Learning Representations. 2015.
    [44] Deng ZF, Peng H, Xia CY, Li JX, He LF, Yu P. Hierarchical bi-directional self-attention networks for paper review rating recommendation. In: Proc. of the 28th Int’l Conf. on Computational Linguistics. Barcelona: ACL, 2020. 6302–6314.
    [45] Zhou P, Shi W, Tian J, Qi ZY, Li BC, Hao HW, Xu B. Attention-based bidirectional long short-term memory networks for relation classification. In: Proc. of the 54th Annual Meeting of the Association for Computational Linguistics (Vol. 2: Short Papers). Berlin: ACL, 2016. 207–212. [doi: 10.18653/v1/P16-2034]
    [46] Lewis M, Liu YH, Goyal N, Ghazvininejad M, Mohamed A, Levy O, Stoyanov V, Zettlemoyer L. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 2020. 7871–7880.
    [47] Ke P, Ji HZ, Liu SY, Zhu XY, Huang ML. SentiLARE: Sentiment-aware language representation learning with linguistic knowledge. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing (EMNLP). ACL, 2020. 6975–6988.
    [48] Fan S, Lin C, Li HN, Lin ZH, Su JS, Zhang H, Gong YY, Guo J, Duan N. Sentiment-aware word and sentence level pre-training for sentiment analysis. In: Proc. of the 2022 Conf. on Empirical Methods in Natural Language Processing. Abu Dhabi: ACL, 2022. 4984–4994. [doi: 10.18653/v1/2022.emnlp-main.332]
    [49] Touvron H, Lavril T, Izacard G, Martinet X, Lachaux MA, Lacroix T, Rozière B, Goyal N, Hambro E, Azhar F, Rodriguez A, Joulin A, Grave E, Lample G. LLaMA: Open and efficient foundation language models. arXiv:2302.13971, 2023.
    [50] Adomavicius G, Bockstedt JC, Curley SP, Zhang JJ. Understanding effects of personalized vs. aggregate ratings on user preferences. In: Proc. of the 2016 Joint Workshop on Interfaces and Human Decision Making for Recommender Systems Co-located with ACM Conf. on Recommender Systems. Boston: IntRS@RecSys. 2016. 14–21.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

强敏杰,王中卿,周国栋.基于预训练模型的用户评分预测.软件学报,2025,36(2):608-624

复制
分享
文章指标
  • 点击次数:186
  • 下载次数: 1630
  • HTML阅读次数: 118
  • 引用次数: 0
历史
  • 收稿日期:2023-07-23
  • 最后修改日期:2023-09-05
  • 在线发布日期: 2024-07-17
文章二维码
您是第19708246位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号