Slot Dependency Modeling for Cross-domain Slot Filling
Author:
Affiliation:

Clc Number:

TP18

  • Article
  • | |
  • Metrics
  • |
  • Reference [24]
  • | | | |
  • Comments
    Abstract:

    This study considers slot filling as a crucial component of task-oriented dialogue systems, which serves downstream tasks by identifying specific slot entities in utterances. However, in a specific domain, it necessitates a large amount of labeled data, which is costly to collect. In this context, cross-domain slot filling emerges and efficiently addresses the issue of data scarcity through transfer learning. However, existing methods overlook the dependencies between slot types in utterances, leading to the suboptimal performance of existing models when transferring to new domains. To address this issue, a cross-domain slot filling method based on slot dependency modeling is proposed in this study. Leveraging the prompt learning approach based on generative pre-trained models, a prompt template integrating slot dependency information is designed, establishing implicit dependency relationships between different slot types and fully exploiting the predictive performance of slot entities in the pre-trained model. Furthermore, to enhance the semantic dependencies between slot types, slot entities, and utterance texts, discourse filling subtask is introduced in this study to strengthen the inherent connections between utterances and slot entities through reverse filling. Transfer experiments across multiple domains demonstrate significant performance improvements in zero-shot and few-shot settings achieved by the proposed model. Additionally, a detailed analysis of the main structures in the model and ablation experiments are conducted in this study to further validate the necessity of each part of the model.

    Reference
    [1] Goo CW, Gao G, Hsu YK, Luo CL, Chen TC, Hsu KW, Chen YN. Slot-gated modeling for joint slot filling and intent prediction. In: Proc. of the 2018 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 2 (Short Papers). New Orleans: ACL, 2018. 753–757. [doi: 10.18653/v1/N18-2118]
    [2] Li X, Wang YY, Tür G. Multi-task learning for spoken language understanding with shared slots. In: Proc. of the 12th Annual Conf. of the Int’l Speech Communication Association. Florence: ISCA, 2011. 701–704.
    [3] Yazdani M, Henderson J. A model of zero-shot learning of spoken language understanding. In: Proc. of the 2015 Conf. on Empirical Methods in Natural Language Processing. Lisbon: ACL, 2015. 244–249. [doi: 10.18653/v1/D15-1027]
    [4] Zhang CW, Li YL, Du N, Fan W, Yu P. Joint slot filling and intent detection via capsule neural networks. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 5259–5267. [doi: 10.18653/v1/P19-1519]
    [5] Qin LB, Liu TL, Che WX, Kang BB, Zhao SD, Liu T. A co-interactive Transformer for joint slot filling and intent detection. In: Proc. of the 2021 IEEE Int’l Conf. on Acoustics, Speech and Signal Processing (ICASSP). Toronto: IEEE, 2021. 8193–8197.
    [6] Wu D, Ding L, Lu F, Xie J. SlotRefine: A fast non-autoregressive model for joint intent detection and slot filling. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing (EMNLP). ACL, 2020. 1932–1937.
    [7] Ferreira E, Jabaian B, Lefèvre F. Online adaptative zero-shot learning spoken language understanding using word-embedding. In: Proc. of the 2015 IEEE Int’l Conf. on Acoustics, Speech and Signal Processing (ICASSP). South Brisbane: IEEE, 2015. 5321–5325.
    [8] E HH, Niu PQ, Chen ZF, Song MN. A novel bi-directional interrelated model for joint intent detection and slot filling. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 5467–5471. [doi: 10.18653/v1/P19-1544]
    [9] Li XC, Wang YJ, Gan L, Zhan DC. Exploring transferability measures and domain selection in cross-domain slot filling. In: Proc. of the 2022 IEEE Int’l Conf. on Acoustics, Speech and Signal Processing (ICASSP 2022). Singapore: IEEE, 2022. 3758–3762.
    [10] Bapna A, Tür G, Hakkani-Tür D, Heck L. Towards zero-shot frame semantic parsing for domain scaling. In: Proc. of the 18th Annual Conf. of the Int’l Speech Communication Association. Stockholm: ISCA, 2017. 2476–2480.
    [11] Shah D, Gupta R, Fayazi A, Hakkani-Tur D. Robust zero-shot cross-domain slot filling with example values. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 5484–5490. [doi: 10.18653/v1/P19-1547]
    [12] Liu Z, Winata GI, Xu P, Fung P. Coach: A coarse-to-fine approach for cross-domain slot filling. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 2020. 19–25.
    [13] He KQ, Zhang JC, Yan YM, Yan YM, Xu WR, Niu C, Zhou J. Contrastive zero-shot learning for cross-domain slot filling with adversarial attack. In: Proc. of the 28th Int’l Conf. on Computational Linguistics. Barcelona: ACL, 2020. 1461–1467.
    [14] Liu SD, Huang PJ, Zhu ZB, Zhang HL, Tan JY. Cross-domain slot filling with distinct slot entity and type prediction. In: Proc. of the 10th CCF Int’l Conf. on Natural Language Processing and Chinese Computing. Qingdao: Springer, 2021. 517–528. [doi: 10.1007/978-3-030-88480-2_41]
    [15] Liu LC, Lin XX, Zhang P, Wang B. Improving cross-domain slot filling with common syntactic structure. In: Proc. of the 2021 IEEE Int’l Conf. on Acoustics, Speech and Signal Processing (ICASSP 2021). Toronto: IEEE, 2021. 7638–7642.
    [16] Li JN, Zhou P, Xiong CM, Hoi S. Prototypical contrastive learning of unsupervised representations. In: Proc. of the 9th Int’l Conf. on Learning Representations. Vienna: OpenReview.net, 2021.
    [17] Du XY, He LH, Li Q, Yu D, Pasupat P, Zhang Y. QA-driven zero-shot slot filling with weak supervision pretraining. In: Proc. of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int’l Joint Conf. on Natural Language Processing (Vol. 2: Short Papers). ACL, 2021. 654–664. [doi: 10.18653/v1/2021.acl-short.83]
    [18] Yu MS, Liu J, Chen YF, Xu JA, Zhang YJ. Cross-domain slot filling as machine reading comprehension. In: Proc. of the 30th Int’l Joint Conf. on Artificial Intelligence. Montreal: ijcai.org, 2021. 3992–3998. [doi: 10.24963/ijcai.2021/550]
    [19] Ge CJ, Huang R, Xie MX, Lai ZH, Song SJ, Li S, Huang G. Domain adaptation via prompt learning. arXiv:2202.06687, 2022.
    [20] Yan Y, Ye JD, Zhang ZB, Wang LW. AISFG: Abundant information slot filling generator. In: Proc. of the 2022 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Seattle: ACL, 2022. 4180–4187. [doi: 10.18653/v1/2022.naacl-main.308]
    [21] Luo QY, Liu LQ. Zero-shot slot filling with slot-prefix prompting and attention relationship descriptor. In: Proc. of the 37th AAAI Conf. on Artificial Intelligence. Washington: AAAI Press, 2023. 13344–13352. [doi: 10.1609/aaai.v37i11.26566]
    [22] Coucke A, Saade A, Ball A, Bluche T, Caulier A, Leroy D, Doumouro C, Gisselbrecht T, Caltagirone F, Lavril T, Primet M, Dureau J. Snips voice platform: An embedded spoken language understanding system for private-by-design voice interfaces. arXiv:1805.10190, 2018.
    [23] Gupta S, Shah R, Mohit M, Kumar A, Lewis M. Semantic parsing for task oriented dialog using hierarchical representations. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Brussels: ACL, 2018. 2787–2792. [doi: 10.18653/v1/D18-1300]
    [24] Lin ZJ, Liu B, Moon S, Crook P, Zhou ZP, Wang ZG, Yu Z, Madotto A, Cho E, Subba R. Leveraging slot descriptions for zero-shot cross-domain dialogue state tracking. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL, 2021. 5640–5648.
    Related
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

王泽,周夏冰,鞠鑫,王中卿,周国栋.基于槽依赖建模的跨领域槽填充方法.软件学报,2025,36(4):1557-1569

Copy
Share
Article Metrics
  • Abstract:179
  • PDF: 1611
  • HTML: 55
  • Cited by: 0
History
  • Received:November 09,2023
  • Revised:January 08,2024
  • Online: June 14,2024
You are the first2035264Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063