Document Open Access Logo

Improving Intent Detection Accuracy Through Token Level Labeling

Authors Michał Lew, Aleksander Obuchowski, Monika Kutyła

PDF
Thumbnail PDF

File

OASIcs.LDK.2021.30.pdf
  • Filesize: 0.87 MB
  • 11 pages

Document Identifiers

Author Details

Michał Lew
  • SentiOne Research, Gdańsk, Poland
Aleksander Obuchowski
  • SentiOne Research, Gdańsk, Poland
  • Gdańsk University of Technology, Poland
Monika Kutyła
  • SentiOne Research, Gdańsk, Poland

Cite As Get BibTex

Michał Lew, Aleksander Obuchowski, and Monika Kutyła. Improving Intent Detection Accuracy Through Token Level Labeling. In 3rd Conference on Language, Data and Knowledge (LDK 2021). Open Access Series in Informatics (OASIcs), Volume 93, pp. 30:1-30:11, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2021) https://doi.org/10.4230/OASIcs.LDK.2021.30

Abstract

Intent detection is traditionally modeled as a sequence classification task where the role of the models is to map the users' utterances to their class. In this paper, however, we show that the classification accuracy can be improved with the use of token level intent annotations and introducing new annotation guidelines for labeling sentences in the intent detection task. What is more, we introduce a method for training the network to predict joint sentence level and token level annotations. We also test the effects of different annotation schemes (BIO, binary, sentence intent) on the model’s accuracy.

Subject Classification

ACM Subject Classification
  • Computing methodologies → Natural language processing
Keywords
  • Intent Detection
  • Annotation
  • NLP
  • Chatbots

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Qian Chen, Zhu Zhuo, and Wen Wang. Bert for joint intent classification and slot filling. arXiv preprint, 2019. URL: http://arxiv.org/abs/1902.10909.
  2. Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint, 2018. URL: http://arxiv.org/abs/1805.10190.
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint, 2018. URL: http://arxiv.org/abs/1810.04805.
  4. Rashmi Gangadharaiah and Balakrishnan Narayanaswamy. Joint multiple intent detection and slot labeling for goal-oriented dialog. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 564-569, 2019. Google Scholar
  5. Chih-Wen Goo, Guang Gao, Yun-Kai Hsu, Chih-Li Huo, Tsung-Chieh Chen, Keng-Wei Hsu, and Yun-Nung Chen. Slot-gated modeling for joint slot filling and intent prediction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 753-757, 2018. Google Scholar
  6. E Haihong, Peiqing Niu, Zhongfu Chen, and Meina Song. A novel bi-directional interrelated model for joint intent detection and slot filling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5467-5471, 2019. Google Scholar
  7. Byeongchang Kim, Seonghan Ryu, and Gary Geunbae Lee. Two-stage multi-intent detection for spoken language understanding. Multimedia Tools and Applications, 76(9):11377-11390, 2017. Google Scholar
  8. Bing Liu and Ian Lane. Attention-based recurrent neural network models for joint intent detection and slot filling. arXiv preprint, 2016. URL: http://arxiv.org/abs/1609.01454.
  9. Christopher Manning and Richard Socher. Natural language processing with deep learning. Lecture Notes Stanford University School of Engineering, 2017. Google Scholar
  10. Aleksander Obuchowski and Michal Lew. Transformer-capsule model for intent detection (student abstract). In AAAI, pages 13885-13886, 2020. Google Scholar
  11. Narodowy Korpus Języka Polskiego. Nkjp. bd). Pobrano, 31, 2017. Google Scholar
  12. Libo Qin, Wanxiang Che, Yangming Li, Haoyang Wen, and Ting Liu. A stack-propagation framework with token-level intent detection for spoken language understanding. arXiv preprint, 2019. URL: http://arxiv.org/abs/1909.02188.
  13. Rachele Sprugnoli, Caselli Tommaso, Tonelli Sara, and Moretti Giovanni. The content types dataset: a new resource to explore semantic and functional characteristics of texts. In 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, volume 2, pages 260-266. Association for Computational Linguistics, 2017. Google Scholar
  14. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015. Google Scholar
  15. Congying Xia, Chenwei Zhang, Xiaohui Yan, Yi Chang, and S Yu Philip. Zero-shot user intent detection via capsule neural networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3090-3099, 2018. Google Scholar
  16. Chenwei Zhang, Yaliang Li, Nan Du, Wei Fan, and Philip S Yu. Joint slot filling and intent detection via capsule neural networks. arXiv preprint, 2018. URL: http://arxiv.org/abs/1812.09471.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact