OASIcs.LDK.2021.30.pdf
- Filesize: 0.87 MB
- 11 pages
Document
Improving Intent Detection Accuracy Through Token Level Labeling
Authors
-
Part of:
Volume:
3rd Conference on Language, Data and Knowledge (LDK 2021)
Series: Open Access Series in Informatics (OASIcs)
Conference: Conference on Language, Data and Knowledge (LDK) - License:
Creative Commons Attribution 4.0 International license
- Publication Date: 2021-08-30
Cite AsGet BibTex
Michał Lew, Aleksander Obuchowski, and Monika Kutyła. Improving Intent Detection Accuracy Through Token Level Labeling. In 3rd Conference on Language, Data and Knowledge (LDK 2021). Open Access Series in Informatics (OASIcs), Volume 93, pp. 30:1-30:11, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2021)
https://doi.org/10.4230/OASIcs.LDK.2021.30
https://doi.org/10.4230/OASIcs.LDK.2021.30
Abstract
Intent detection is traditionally modeled as a sequence classification task where the role of the models is to map the users' utterances to their class. In this paper, however, we show that the classification accuracy can be improved with the use of token level intent annotations and introducing new annotation guidelines for labeling sentences in the intent detection task. What is more, we introduce a method for training the network to predict joint sentence level and token level annotations. We also test the effects of different annotation schemes (BIO, binary, sentence intent) on the model’s accuracy.
Subject Classification
ACM Subject Classification
- Computing methodologies → Natural language processing
Keywords
- Intent Detection
- Annotation
- NLP
- Chatbots
Metrics
- Access Statistics
-
Total Accesses (updated on a weekly basis)
0PDF Downloads
References
- Qian Chen, Zhu Zhuo, and Wen Wang. Bert for joint intent classification and slot filling. arXiv preprint, 2019. URL: http://arxiv.org/abs/1902.10909.
- Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint, 2018. URL: http://arxiv.org/abs/1805.10190.
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint, 2018. URL: http://arxiv.org/abs/1810.04805.
-
Rashmi Gangadharaiah and Balakrishnan Narayanaswamy. Joint multiple intent detection and slot labeling for goal-oriented dialog. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 564-569, 2019.
-
Chih-Wen Goo, Guang Gao, Yun-Kai Hsu, Chih-Li Huo, Tsung-Chieh Chen, Keng-Wei Hsu, and Yun-Nung Chen. Slot-gated modeling for joint slot filling and intent prediction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 753-757, 2018.
-
E Haihong, Peiqing Niu, Zhongfu Chen, and Meina Song. A novel bi-directional interrelated model for joint intent detection and slot filling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5467-5471, 2019.
-
Byeongchang Kim, Seonghan Ryu, and Gary Geunbae Lee. Two-stage multi-intent detection for spoken language understanding. Multimedia Tools and Applications, 76(9):11377-11390, 2017.
- Bing Liu and Ian Lane. Attention-based recurrent neural network models for joint intent detection and slot filling. arXiv preprint, 2016. URL: http://arxiv.org/abs/1609.01454.
-
Christopher Manning and Richard Socher. Natural language processing with deep learning. Lecture Notes Stanford University School of Engineering, 2017.
-
Aleksander Obuchowski and Michal Lew. Transformer-capsule model for intent detection (student abstract). In AAAI, pages 13885-13886, 2020.
-
Narodowy Korpus Języka Polskiego. Nkjp. bd). Pobrano, 31, 2017.
- Libo Qin, Wanxiang Che, Yangming Li, Haoyang Wen, and Ting Liu. A stack-propagation framework with token-level intent detection for spoken language understanding. arXiv preprint, 2019. URL: http://arxiv.org/abs/1909.02188.
-
Rachele Sprugnoli, Caselli Tommaso, Tonelli Sara, and Moretti Giovanni. The content types dataset: a new resource to explore semantic and functional characteristics of texts. In 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, volume 2, pages 260-266. Association for Computational Linguistics, 2017.
-
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015.
-
Congying Xia, Chenwei Zhang, Xiaohui Yan, Yi Chang, and S Yu Philip. Zero-shot user intent detection via capsule neural networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3090-3099, 2018.
- Chenwei Zhang, Yaliang Li, Nan Du, Wei Fan, and Philip S Yu. Joint slot filling and intent detection via capsule neural networks. arXiv preprint, 2018. URL: http://arxiv.org/abs/1812.09471.