OASIcs.SLATE.2014.251.pdf
- Filesize: 2.18 MB
- 15 pages
Document
Language Identification: a Neural Network Approach
Authors
-
Part of:
Volume:
3rd Symposium on Languages, Applications and Technologies (SLATE 2014)
Series: Open Access Series in Informatics (OASIcs)
Conference: Symposium on Languages, Applications and Technologies (SLATE) - License:
Creative Commons Attribution 3.0 Unported license
- Publication Date: 2014-06-18
Cite AsGet BibTex
Alberto Simões, José João Almeida, and Simon D. Byers. Language Identification: a Neural Network Approach. In 3rd Symposium on Languages, Applications and Technologies. Open Access Series in Informatics (OASIcs), Volume 38, pp. 251-265, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2014)
https://doi.org/10.4230/OASIcs.SLATE.2014.251
https://doi.org/10.4230/OASIcs.SLATE.2014.251
Abstract
One of the first tasks when building a Natural Language application is the detection of the used language in order to adapt the system to that language. This task has been addressed several times. Nevertheless most of these attempts were performed a long time ago when the amount of computer data and the computational power were limited. In this article we analyze and explain the use of a neural network for language identification, where features can be extracted automatically, and therefore, easy to adapt to new languages. In our experiments we got some surprises, namely with the two Chinese variants, whose forced us for some language-dependent tweaking of the neural network. At the end, the network had a precision of 95%, only failing for the Portuguese language.
Keywords
- language identification
- neural networks
- language models
- trigrams
Metrics
- Access Statistics
-
Total Accesses (updated on a weekly basis)
0PDF Downloads