Document Open Access Logo

Time Expressions Recognition with Word Vectors and Neural Networks

Authors Mathias Etcheverry, Dina Wonsever

PDF
Thumbnail PDF

File

LIPIcs.TIME.2017.12.pdf
  • Filesize: 0.51 MB
  • 20 pages

Document Identifiers

Author Details

Mathias Etcheverry
Dina Wonsever

Cite AsGet BibTex

Mathias Etcheverry and Dina Wonsever. Time Expressions Recognition with Word Vectors and Neural Networks. In 24th International Symposium on Temporal Representation and Reasoning (TIME 2017). Leibniz International Proceedings in Informatics (LIPIcs), Volume 90, pp. 12:1-12:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2017)
https://doi.org/10.4230/LIPIcs.TIME.2017.12

Abstract

This work re-examines the widely addressed problem of the recognition and interpretation of time expressions, and suggests an approach based on distributed representations and artificial neural networks. Artificial neural networks allow us to build highly generic models, but the large variety of hyperparameters makes it difficult to determine the best configuration. In this work we study the behavior of different models by varying the number of layers, sizes and normalization techniques. We also analyze the behavior of distributed representations in the temporal domain, where we find interesting properties regarding order and granularity. The experiments were conducted mainly for Spanish, although this does not affect the approach, given its generic nature. This work aims to be a starting point towards processing temporality in texts via word vectors and neural networks, without the need of any kind of feature engineering.
Keywords
  • Natural Language Processing
  • Time Expressions
  • Word Embeddings
  • Neural Networks

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Sisay Fissaha Adafre and Maarten de Rijke. Feature engineering and post-processing for temporal expression recognition using conditional random fields. In Proceedings of the ACL Workshop on Feature Engineering for Machine Learning in Natural Language Processing, FeatureEng'05, pages 9-16, Stroudsburg, PA, USA, 2005. Association for Computational Linguistics. URL: http://www.aclweb.org/anthology/W05-0402.
  2. David Ahn, Sisay Fissaha Adafre, and Maarten de Rijke. Towards task-based temporal extraction and recognition. In Graham Katz, James Pustejovsky, and Frank Schilder, editors, Annotating, Extracting and Reasoning about Time and Events, number 05151 in Dagstuhl Seminar Proceedings, Dagstuhl, Germany, 2005. Internationales Begegnungs- und Forschungszentrum für Informatik (IBFI), Schloss Dagstuhl, Germany. Google Scholar
  3. David Ahn, Joris Rantwijk, and Maarten Rijke. A cascaded machine learning approach to interpreting temporal expressions. In in Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT 2007. Citeseer, 2007. Google Scholar
  4. Gabor Angeli, Christopher D. Manning, and Daniel Jurafsky. Parsing time: Learning to interpret time expressions. In Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 3-8, 2012, Montréal, Canada, pages 446-455, 2012. URL: http://www.aclweb.org/anthology/N12-1049.
  5. Gabor Angeli and Jakob Uszkoreit. Language-independent discriminative parsing of temporal expressions. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, ACL 2013, 4-9 August 2013, Sofia, Bulgaria, Volume 1: Long Papers, pages 83-92, 2013. URL: http://www.aclweb.org/anthology/P/P13/P13-1009.pdf.
  6. Agustín Azzinnari and Alejandro Martínez. Representación de Palabras en Espacios de Vectores. Proyecto de grado, Universidad de la República, Uruguay, 2016. Google Scholar
  7. Steven Bethard. Cleartk-timeml: A minimalist approach to tempeval 2013. Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), 2013. Google Scholar
  8. Steven Bethard. A synchronous context free grammar for time normalization. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18-21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 821-826, 2013. URL: http://www.aclweb.org/anthology/D/D13/D13-1078.pdf.
  9. Angel X. Chang and Christopher Manning. Sutime: A library for recognizing and normalizing time expressions. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis, editors, Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12), Istanbul, Turkey, may 2012. European Language Resources Association (ELRA). Google Scholar
  10. R. Collobert and J. Weston. A unified architecture for natural language processing: deep neural networks with multitask learning. In Proceedings of ICML, pages 160–167, 2008. Google Scholar
  11. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa. Natural language processing (almost) from scratch. JMLR, 12:2493–2537, 2011. Google Scholar
  12. Mathias Etcheverry and Dina Wonsever. Spanish word vectors from wikipedia. Language Resource Conference (LREC 2016), 2016. Google Scholar
  13. Michele Filannino. Temporal expression normalisation in natural language texts. CoRR, abs/1206.2010, 2012. URL: http://arxiv.org/abs/1206.2010.
  14. Michele Filannino, Gavin Brown, and Goran Nenadic. Mantime: Temporal expression identification and normalization in the tempeval-3 challenge. CoRR, abs/1304.7942, 2013. URL: http://arxiv.org/abs/1304.7942.
  15. A. Graves and J. Schmidhuber. Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures. Neural Networks, 18(5-6):602-610, 2005. Google Scholar
  16. Claire Grover, Richard Tobin, Beatrice Alex, and Kate Byrne. Edinburgh-LTG: TempEval-2 System Description. In Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval'10, pages 333-336, Stroudsburg, PA, USA, 2010. Association for Computational Linguistics. URL: http://dl.acm.org/citation.cfm?id=1859664.1859738.
  17. Naman Gupta, Aditya Joshi, and Pushpak Bhattacharyya. A temporal expression recognition system for medical documents by taking help of news domain corpora. 12th International Conference on Natural Language Processing (ICON), 2015. Google Scholar
  18. Ozan Irsoy and Claire Cardie. Opinion mining with deep recurrent neural networks. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014. Google Scholar
  19. Leif Johnson, Majid alDosari, Filip Juricek, John, Kyle Kastner, Yoav Goldberg, talbaumel, Yu Yang, mhr, Eben Olson, and Sergey Romanov. theanets: v0.6.1, July 2015. URL: http://dx.doi.org/10.5281/zenodo.19930.
  20. Hyuckchul Jung and Amanda Stent. ATT1: temporal annotation using big windows and rich syntactic and semantic features. In Proceedings of the 7th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2013, Atlanta, Georgia, USA, June 14-15, 2013, pages 20-24, 2013. Google Scholar
  21. Oleksandr Kolomiyets and Marie-Francine Moens. Kul: Recognition and normalization of temporal expressions. In Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval'10, pages 325-328, Stroudsburg, PA, USA, 2010. Association for Computational Linguistics. URL: http://www.aclweb.org/anthology/S10-1072.
  22. Kenton Lee, Yoav Artzi, Jesse Dodge, and Luke Zettlemoyer. Context-dependent semantic parsing for time expressions, volume 1, pages 1437-1447. Association for Computational Linguistics (ACL), 2014. Google Scholar
  23. H. Llorens, E. Saquete, and B. Navarro-Colorado. Timeml events recognition and classification: Learning crf models with semantic roles. In In COLING 2010, 23rd International Conference on Computational Linguistics, Proceedings of the Conference, 23-27 August 2010, Beijing, China, page 725–733, 2010. Google Scholar
  24. Inderjeet Mani and D. George Wilson. Robust temporal processing of news. In 38th Annual Meeting of the Association for Computational Linguistics, Hong Kong, China, October 1-8, 2000., 2000. URL: http://www.aclweb.org/anthology/P00-1010.
  25. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. In Proceedings of Workshop at ICLR, 2013. Google Scholar
  26. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014. Google Scholar
  27. Jordi Poveda, Mihai Surdeanu, and Jordi Turmo. An analysis of bootstrapping for the recognition of temporal expressions. In Proceedings of the NAACL HLT 2009 Workshop on Semi-Supervised Learning for Natural Language Processing, SemiSupLearn'09, pages 49-57, Stroudsburg, PA, USA, 2009. Association for Computational Linguistics. URL: http://dl.acm.org/citation.cfm?id=1621829.1621836.
  28. Georgiana Puscasu. A framework for temporal resolution. In Proceedings of the Fourth International Conference on Language Resources and Evaluation, LREC 2004, May 26-28, 2004, Lisbon, Portugal, 2004. URL: http://www.lrec-conf.org/proceedings/lrec2004/pdf/664.pdf.
  29. James Pustejovsky, José M. Castaño, Robert Ingria, Roser Sauri, Robert J. Gaizauskas, Andrea Setzer, Graham Katz, and Dragomir R. Radev. Timeml: Robust specification of event and temporal expressions in text. In New Directions in Question Answering, Papers from 2003 AAAI Spring Symposium, Stanford University, Stanford, CA, USA, pages 28-34, 2003. Google Scholar
  30. Lev Ratinov and Dan Roth. Design challenges and misconceptions in named entity recognition. Proceedings of the Thirteenth Conference on Computational Natural Language Learning , CoNLL ’09, pages 147–155, Stroudsburg, PA, USA., 2009. Google Scholar
  31. Richard Socher, John Bauer, Christopher D, Manning, and Andrew Y. Ng. Parsing with compositional vector grammars. Association for Computational Linguistics 2013 Conference (ACL 2013), 2013. Google Scholar
  32. Richard Socher, Eric H. Huang, Jeffrey Pennington, Andrew Y. Ng, and Christopher D. Manning. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. Advances in Neural Information Processing Systems (NIPS 2011), 2011. Google Scholar
  33. Richard Socher, Alex Perelygin, Jason Chuang Jean Wu, Chris Manning, Andrew Ng, and Chris Potts. Recursive deep models for semantic compositionality over a sentiment treebank. Conference on Empirical Methods in Natural Language Processing (EMNLP 2013), 2013. Google Scholar
  34. Jannik Strötgen and Michael Gertz. HeidelTime: High Quality Rule-based Extraction and Normalization of Temporal Expressions. In Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval'10, pages 321-324, Stroudsburg, PA, USA, 2010. Association for Computational Linguistics. URL: http://dl.acm.org/citation.cfm?id=1859664.1859735.
  35. Jannik Strötgen and Michael Gertz. Multilingual and cross-domain temporal tagging. Language Resources and Evaluation, 47(2):269-298, 2013. URL: http://dx.doi.org/10.1007/s10579-012-9179-y.
  36. Theano Development Team. Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints, abs/1605.02688, May 2016. URL: http://arxiv.org/abs/1605.02688.
  37. Tijmen Tieleman and Geoffrey E. Hinton. Lecture 6.5 - rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 2012. Google Scholar
  38. Naushad UzZaman and James F. Allen. TRIPS and TRIOS System for TempEval-2: Extracting Temporal Information from Text. In Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval'10, pages 276-283, Stroudsburg, PA, USA, 2010. Association for Computational Linguistics. URL: http://www.aclweb.org/anthology/S10-1062.
  39. Naushad UzZaman, Hector Llorens, James F. Allen, Leon Derczynski, Marc Verhagen, and James Pustejovsky. Tempeval-3: Evaluating events, time expressions, and temporal relations. CoRR, abs/1206.5333, 2012. URL: http://arxiv.org/abs/1206.5333.
  40. L. J. P. van der Maaten and G. E. Hinton. Visualizing high-dimensional data using t-sne. Journal of Machine Learning Research 9(Nov):2579-2605, 2008. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact