DROPS

Document

DOI: 10.4230/OASIcs.SLATE.2014.251

Language Identification: a Neural Network Approach

Authors: Alberto Simões, José João Almeida, and Simon D. Byers

Published in: OASIcs, Volume 38, 3rd Symposium on Languages, Applications and Technologies (2014)

Abstract

One of the first tasks when building a Natural Language application is the detection of the used language in order to adapt the system to that language. This task has been addressed several times. Nevertheless most of these attempts were performed a long time ago when the amount of computer data and the computational power were limited. In this article we analyze and explain the use of a neural network for language identification, where features can be extracted automatically, and therefore, easy to adapt to new languages. In our experiments we got some surprises, namely with the two Chinese variants, whose forced us for some language-dependent tweaking of the neural network. At the end, the network had a precision of 95%, only failing for the Portuguese language.

Cite as

Alberto Simões, José João Almeida, and Simon D. Byers. Language Identification: a Neural Network Approach. In 3rd Symposium on Languages, Applications and Technologies. Open Access Series in Informatics (OASIcs), Volume 38, pp. 251-265, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2014)

Copy BibTex To Clipboard

@InProceedings{simoes_et_al:OASIcs.SLATE.2014.251,
  author =	{Sim\~{o}es, Alberto and Almeida, Jos\'{e} Jo\~{a}o and Byers, Simon D.},
  title =	{{Language Identification: a Neural Network Approach}},
  booktitle =	{3rd Symposium on Languages, Applications and Technologies},
  pages =	{251--265},
  series =	{Open Access Series in Informatics (OASIcs)},
  ISBN =	{978-3-939897-68-2},
  ISSN =	{2190-6807},
  year =	{2014},
  volume =	{38},
  editor =	{Pereira, Maria Jo\~{a}o Varanda and Leal, Jos\'{e} Paulo and Sim\~{o}es, Alberto},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/OASIcs.SLATE.2014.251},
  URN =		{urn:nbn:de:0030-drops-45749},
  doi =		{10.4230/OASIcs.SLATE.2014.251},
  annote =	{Keywords: language identification, neural networks, language models, trigrams}
}

Search Results

Documents authored by Byers, Simon D.

Language Identification: a Neural Network Approach

Abstract

Cite as

Thanks for your feedback!

Could not send message