面向人机对话意图分类的混合神经网络模型
作者:
作者简介:

周俊佐(1995-),男,四川安岳人,硕士,CCF学生会员,主要研究领域为自然语言处理;陈文亮(1977-),男,博士,教授,博士生导师,CCF专业会员,主要研究领域为自然语言处理;朱宗奎(1994-),男,硕士,CCF学生会员,主要研究领域为自然语言处理;张民(1970-),男,博士,教授,博士生导师,CCF高级会员,主要研究领域为自然语言处理,机器翻译,人工智能;何正球(1993-),男,硕士,CCF学生会员,主要研究领域为自然语言处理.

通讯作者:

陈文亮,E-mail:wlchen@suda.edu.cn

中图分类号:

TP18

基金项目:

国家自然科学基金(61876115,61572338,61525205);江苏高校优势学科建设工程(PAPD)


Hybrid Neural Network Models for Human-machine Dialogue Intention Classification
Author:
Fund Project:

National Natural Science Foundation of China (61876115, 61572338, 61525205); Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions

  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [30]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    随着人机对话的不断发展,让计算机能够准确地理解用户查询意图,对整个人机对话领域都有着重要意义.意图分类的主要目标是在人机对话的过程中判断用户的意图,提升人机对话系统的准确度与自然度.首先分析多个分类模型在意图分类任务上的优缺点.在此基础上,提出一种混合神经网络模型,综合利用多个深度网络模型的多样性输出.在输入特征预处理上,采用语言模型词向量,将语言模型拥有的语义挖掘能力应用到混合网络中,可以进一步提升模型的表达能力.所提出的混合神经网络模型相对于最好的基准模型在两份数据集上分别取得了2.95%和3.85%的性能提升.新模型在该数据上取得了最优的性能.

    Abstract:

    With the development of human-machine dialogue, it is of great significance for the computer to accurately understand the user's query intention in human-machine dialogue systems. Intention classification aims at judging the user's intention in human machine dialogue and improves the accuracy and naturalness of the human machine dialogue system. This study first analyzes the advantages and disadvantages of multiple classification models in the intention classification task. On this basis, this study proposes a hybrid neural network model to comprehensively utilize the diversity outputs of multiple deep network models. To further improve the perfoance, the language model embedding is used in the input feature preprocessing and the semantic mining ability possessed for the hybrid network which can effectively improve the expression ability of the model. The proposed model achieves 2.95% and 3.85% performance improvement on the two data sets respectively compared to the best benchmark model. The proposed model also achieves the top performance in a shared task.

    参考文献
    [1] Morbini F, De Vault D, Sagae K, Gerten J, Nazarian A, Traum D. FLoReS:A forward looking, reward seeking, dialogue manager. In:Natural Interaction with Robots, Knowbots and Smartphones. New York:Springer-Verlag, 2012.313-325.
    [2] Tur G, Celikyilmaz A, Hakkani-Tür D. Latent semantic modeling for slot filling in conversational understanding. In:Proc. of the 2013 IEEE Int'l Conf. on Acoustics, Speech, and Signal Processing. IEEE Computer Society Press, 2013.8307-8311.
    [3] Eyben F, Wöllmer M, Graves A, Schuller B, Douglas-Cowie E, Cowie R. On-line emotion recognition in a 3-D activation-valence-time continuum using acoustic and linguistic cues. Journal on Multimodal User Interfaces, 2010,3(1-2):7-12.
    [4] Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E. Hierarchical attention networks for document classification. In:Proc. of the 2016 Conf. of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Association for Computational Linguistics, 2016.1480-1489.
    [5] Lai S, Xu L, Liu K, Zhao J. Recurrent convolutional neural networks for text classification. In:Proc. of the 29th AAAI Conf. on Artificial Intelligence. AAAI, 2015.2267-2273.
    [6] Li C, Chai YM, Nan XF, Gao ML. Research on problem classification method based on deep learning. Computer Science, 2016, 43(12):115-119(in Chinese with English abstract).
    [7] Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015.1-9.
    [8] Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L. Deep contextualized word representations. arXiv preprint arXiv:1802.05365, 2018.
    [9] Mathew J, Radhakrishnan D. An FIR digital filter using onehot coded residue representation. In:Proc. of the 10th European Signal Processing Conf. IEEE Computer Society Press, 2000.1-4.
    [10] Irsoy O, Cardie C. Opinion mining with deep recurrent neural networks. In:Proc. of the 2014 Conf. on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2014.720-728.
    [11] Goldberg Y, Levy O. Word2vec explained:Deriving Mikolov et al.'s negative-sampling word-embedding method. arXiv preprint arXiv:1402.3722, 2014.
    [12] 12Pennington J, Socher R, Manning C. Glove:Global vectors for word representation. In:Proc. of the 2014 Conf. on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2014.1532-1543.
    [13] Howard J, Ruder S. Universal language model fine-tuning for text classification. In:Proc. of the 56th Annual Meeting of the Association for Computational Linguistics, Vol.1. Association for Computational Linguistics, 2018.328-339.
    [14] Devlin J, Chang MW, Lee K, Toutanova K. Bert:Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
    [15] Joachims T. Text categorization with support vector machines:Learning with many relevant features. In:Proc. of the European Conf. on Machine Learning. Berlin, Heidelberg:Springer-Verlag, 1998.
    [16] Kim Y. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, 2014.
    [17] Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans. on Neural Networks, 1994,5(2):157-166.
    [18] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997,9(8):1735-1780.
    [19] Chung J, Gulcehre C, Cho KH, Bengio Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
    [20] Sabour S, Frosst N, Hinton GE. Dynamic routing between capsules. In:Advances in Neural Information Processing Systems. MIT Press, 2017.3856-3866.
    [21] Mnih V, Heess N, Graves A. Recurrent models of visual attention. In:Advances in Neural Information Processing Systems. MIT Press, 2014.2204-2212.
    [22] Chen H, Sun M, Tu C, Lin Y, Liu Z. Neural sentiment classification with user and product attention. In:Proc. of the 2016 Conf. on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2016.1650-1659.
    [23] Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In:Proc. of the 3rd Int'l Conf. on Learning Representations. arXiv preprint arXiv:1409.0473, 2014.
    [24] Hermann KM, Kocisky T, Grefenstette E, Espeholt L, Kay W, Suleyman M, Blunsom P. Teaching machines to read and comprehend. In:Advances in Neural Information Processing Systems. MIT Press, 2015.1693-1701.
    [25] Johnson R, Zhang T. Deep pyramid convolutional neural networks for text categorization. In:Proc. of the 55th Annual Meeting of the Association for Computational Linguistics, Vol.1. Association for Computational Linguistics, 2017.562-570.
    [26] Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
    [27] Breiman L. Bagging predictors. Machine Learning, 1996,24(2):123-140.
    [28] 2018. https://mlwave.com/kaggle-ensembling-guide/
    附中文参考文献:
    [6] 李超,柴玉梅,南晓斐,高明磊.基于深度学习的问题分类方法研究.计算机科学,2016,43(12):115-119.
    引证文献
引用本文

周俊佐,朱宗奎,何正球,陈文亮,张民.面向人机对话意图分类的混合神经网络模型.软件学报,2019,30(11):3313-3325

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2019-01-15
  • 最后修改日期:2019-03-12
  • 在线发布日期: 2019-11-06
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号