Pobe: 一种基于生成式模型的分布外文本检测方法
作者:
作者简介:

欧阳亚文(1996-), 男, 博士生, 主要研究领域为自然语言理解, 开放环境下的机器学习;高源(1998-), 男, 硕士生, 主要研究领域为自然语言处理, 机器学习;宗石(1992-), 男, 博士, 主要研究领域为计算语言学;鲍宇(1993-), 男, 博士, 主要研究领域为自然语言处理, 科学智能;戴新宇(1979-), 男, 博士, 教授, 博士生导师, CCF专业会员, 主要研究领域为自然语言处理, 知识工程.

通讯作者:

戴新宇, E-mail: daixinyu@nju.edu.cn

中图分类号:

TP18

基金项目:

国家自然科学基金 (61936012, 61976114)


Pobe: Generative Model-based Out-of-distribution Text Detection Method
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [32]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    对于安全可靠的机器学习系统, 具备检测训练集分布外 (out-of-distribution, OOD) 样本的能力十分必要. 基于似然的生成式模型由于训练时不需要样本标签, 是一类非常受欢迎的OOD检测方法. 然而, 近期研究表明通过似然来检测OOD样本往往会失效, 并且失效原因与解决方案的探究仍较少, 尤其是对于文本数据. 从模型层面和数据层面分析文本上失效的原因: 生成式模型的泛化性不足和文本先验概率的偏差. 在此基础上, 提出一种新的OOD文本检测方法Pobe. 针对生成式模型泛化性不足的问题, 引入KNN检索的方式, 来提升模型的泛化性. 针对文本先验概率偏差的问题, 设计一种偏差校准策略, 借助预训练语言模型改善概率偏差对OOD检测的影响, 并通过贝叶斯定理证明策略的合理性. 通过在广泛的数据集上进行实验, 证明所提方法的有效性, 其中, 在8个数据集上的平均AUROC值超过99%, FPR95值低于1%.

    Abstract:

    It is essential to detect out-of-distribution (OOD) training set samples for a safe and reliable machine learning system. Likelihood-based generative models are popular methods to detect OOD samples because they do not require sample labels during training. However, recent studies show that likelihoods sometimes fail to detect OOD samples, and the failure reason and solutions are under explored, especially for text data. Therefore, this study investigates the text failure reason from the views of the model and data: insufficient generalization of the generative model and prior probability bias of the text. To tackle the above problems, the study proposes a new OOD text detection method, namely Pobe. To address insufficient generalization of the generative model, the study increases the model generalization via KNN retrieval. Next, to address the prior probability bias of the text, the study designs a strategy to calibrate the bias and improve the influence of probability bias on OOD detection by a pre-trained language model and demonstrates the effectiveness of the strategy according to Bayes’ theorem. Experimental results over a wide range of datasets show the effectiveness of the proposed method. Specifically, the average AUROC is over 99%, and FPR95 is below 1% under eight datasets.

    参考文献
    [1] Hendrycks D, Gimpel K. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: Proc. of the 5th Int’l Conf. on Learning Representations. Toulon: OpenReview.net, 2017.
    [2] Gangal V, Arora A, Einolghozati A, Gupta S. Likelihood ratios and generative classifiers for unsupervised out-of-domain detection in task oriented dialog. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. New York: AAAI Press, 2020. 7764–7771.
    [3] Ren J, Liu PJ, Fertig E, Snoek J, Poplin R, DePristo MA, Dillon JV, Lakshminarayanan B. Likelihood ratios for out-of-distribution detection. In: Proc. of the 33rd Int’l Conf. on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2019. 1317.
    [4] Nalisnick ET, Matsukawa A, Teh YW, G?rür D, Lakshminarayanan B. Do deep generative models know what they don’t know? In: Proc. of the 7th Int’l Conf. on Learning Representations. New Orleans: OpenReview.net, 2019.
    [5] Arora U, Huang W, He H. Types of out-of-distribution texts and how to detect them. In: Proc. of the 2021 Conf. on Empirical Methods in Natural Language Processing. Punta Cana: Association for Computational Linguistics, 2021. 10687–10701.
    [6] Serrà J, álvarez D, Gómez V, Slizovskaia O, Nú?ez JF, Luque J. Input complexity and out-of-distribution detection with likelihood-based generative models. In: Proc. of the 8th Int’l Conf. on Learning Representations. Addis Ababa: OpenReview.net, 2020.
    [7] Schirrmeister RT, Zhou YX, Ball T, Zhang D. Understanding anomaly detection with deep invertible networks through hierarchies of distributions and features. In: Proc. of the 34th Conf. on Neural Information Processing Systems. Vancouver: NeurIPS, 2020. 21038–21049.
    [8] Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language models are unsupervised multitask learners. OpenAI Blog, 2019, 1(8): 9.
    [9] Nalisnick E, Matsukawa A, Teh YW, et al. Detecting out-of-distribution inputs to deep generative models using typicality. In: Proc. of the 8th Int’l Conf. on Learning Representations. 2020.
    [10] Podolskiy A, Lipin D, Bout A, Artemova E, Piontkovskaya I. Revisiting Mahalanobis distance for Transformer-based out-of-domain detection. In: Proc. of the 35th AAAI Conf. on Artificial Intelligence. AAAI Press, 2021. 13675–13682.
    [11] Ouyang YW, Ye JS, Chen Y, Dai XY, Huang SJ, Chen JJ. Energy-based unknown intent detection with data manipulation. In: Proc. of the 2021 Findings of the Association for Computational Linguistics. Association for Computational Linguistics, 2021. 2852–2861.
    [12] Liu WT, Wang XY, Owens JD, Li YX. Energy-based out-of-distribution detection. In: Proc. of the 34th Conf. on Neural Information Processing Systems. Vancouver: NeurIPS, 2020. 21464–21475.
    [13] Lee K, Lee K, Lee H, Shin J. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In: Proc. of the 32nd Conf. on Neural Information Processing Systems. Montreal: NeurIPS, 2018. 7167–7177.
    [14] Khandelwal U, Levy O, Jurafsky D, Zettlemoyer L, Lewis M. Generalization through memorization: Nearest neighbor language models. In: Proc. of the 8th Int’l Conf. on Learning Representations. Addis Ababa: OpenReview.net, 2020.
    [15] Khandelwal U, Fan A, Jurafsky D, Zettlemoyer L, Lewis M. Nearest neighbor machine translation. In: Proc. of the 9th Int’l Conf. on Learning Representations. OpenReview.net, 2021.
    [16] Gu JT, Wang Y, Cho K, Li VOK. Search engine guided neural machine translation. In: Proc. of the 32nd AAAI Conf. on Artificial Intelligence. New Orleans: AAAI Press, 2018. 5133–5140.
    [17] Borgeaud S, Mensch A, Hoffmann J, et al. Improving language models by retrieving from trillions of tokens. In: Proc. of the 39th Int’l Conf on Machine Learning. Baltimore: PMLR, 2022. 2206–2240.
    [18] Jiang QN, Wang MX, Cao J, Cheng SB, Huang SJ, Li L. Learning kernel-smoothed machine translation with retrieved examples. In: Proc. of the 2021 Conf. on Empirical Methods in Natural Language Processing. Punta Cana: Association for Computational Linguistics, 2021. 7280–7290.
    [19] Feng Y, Zhang SY, Zhang AD, Wang D, Abel A. Memory-augmented neural machine translation. In: Proc. of the 2017 Conf. on Empirical Methods in Natural Language Processing. Copenhagen: Association for Computational Linguistics, 2017. 1390–1399.
    [20] Kassner N, Schütze H. BERT-KNN: Adding a KNN search component to pretrained language models for better QA. In: Proc. of the 2020 Findings of the Association for Computational Linguistics. Association for Computational Linguistics, 2020. 3424–3430.
    [21] He JX, Neubig G, Berg-Kirkpatrick T. Efficient nearest neighbor language models. In: Proc. of the 2021 Conf. on Empirical Methods in Natural Language Processing. Punta Cana: Association for Computational Linguistics, 2021. 5703–5714.
    [22] Bishop CM. Novelty detection and neural network validation. IEE Proceedings-Vision, Image and Signal Processing, 1994, 141(4): 217–222. [doi: 10.1049/ip-vis:19941330]
    [23] Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C. Learning word vectors for sentiment analysis. In: Proc. of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Portland: Association for Computational Linguistics, 2011. 142–150.
    [24] Larson S, Mahendran A, Peper JJ, Clarke C, Lee A, Hill P, Kummerfeld JK, Leach K, Laurenzano MA, Tang LJ, Mars J. An evaluation dataset for intent classification and out-of-scope prediction. In: Proc. o??桴瑨浥?戲爰?9 Conf. on Empirical Methods in Natural Language Processing and the 9th Int’l Joint Conf. on Natural Language Processing. Hong Kong: Association for Computational Linguistics, 2019. 1311–1316.
    [25] Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng AY, Potts C. Recursive deep models for semantic compositionality over a sentiment treebank. In: Proc. of the 2013 Conf. on Empirical Methods in Natural Language Processing. Seattle: Association for Computational Linguistics, 2013. 1631–1642.
    [26] Zhang X, Zhao JB, LeCun Y. Character-level convolutional networks for text classification. In: Proc. of the 28th Int’l Conf. on Neural Information Processing Systems. Montreal: NIPS, 2015. 649–657.
    [27] Kowsari K, Brown DE, Heidarysafa M, Meimandi KJ, Gerber MS, Barnes LE. HDLtex: Hierarchical deep learning for text classification. In: Proc. of the 16th IEEE Int’l Conf. on Machine Learning and Applications. Cancun: IEEE, 2017. 364–371.
    [28] Kong LK, Jiang HM, Zhuang YC, Lyu J, Zhao T, Zhang C. Calibrated language model fine-tuning for in- and out-of-distribution data. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2021. 1326–1340.
    [29] Sundermeyer M, Schlüter R, Ney H. LSTM neural networks for language modeling. In: Proc. of the 13th Annual Conf. of the Int’l Speech Communication Association. Portland: ISCA, 2012. 194–197.
    [30] Pennington J, Socher R, Manning C. GloVe: Global vectors for word representation. In: Proc. of the 2014 Conf. on Empirical Methods in Natural Language Processing. Doha: Association for Computational Linguistics, 2014. 1532–1543.
    [31] 朱鹏飞, 张琬迎, 王煜, 胡清华. 考虑多粒度类相关性的对比式开放集识别方法. 软件学报, 2022, 33(4): 1156–1169. http://www.jos.org.cn/1000-9825/6468.htm
    Zhu PF, Zhang WY, Wang Y, Hu QH. Multi-granularity inter-class correlation based contrastive learning for open set recognition. Ruan Jian Xue Bao/Journal of Software, 2022, 33(4): 1156?1169 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/646
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

欧阳亚文,高源,宗石,鲍宇,戴新宇. Pobe: 一种基于生成式模型的分布外文本检测方法.软件学报,2024,35(9):4365-4376

复制
分享
文章指标
  • 点击次数:616
  • 下载次数: 1775
  • HTML阅读次数: 492
  • 引用次数: 0
历史
  • 收稿日期:2022-06-02
  • 最后修改日期:2022-09-20
  • 在线发布日期: 2023-09-20
  • 出版日期: 2024-09-06
文章二维码
您是第19708617位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号