1 Search Results for "Dodge, Jesse"


Document
Efficient and Equitable Natural Language Processing in the Age of Deep Learning (Dagstuhl Seminar 22232)

Authors: Jesse Dodge, Iryna Gurevych, Roy Schwartz, Emma Strubell, and Betty van Aken

Published in: Dagstuhl Reports, Volume 12, Issue 6 (2023)


Abstract
This report documents the program and the outcomes of Dagstuhl Seminar 22232 "Efficient and Equitable Natural Language Processing in the Age of Deep Learning". Since 2012, the field of artificial intelligence (AI) has reported remarkable progress on a broad range of capabilities including object recognition, game playing, speech recognition, and machine translation. Much of this progress has been achieved by increasingly large and computationally intensive deep learning models: training costs for state-of-the-art deep learning models have increased 300,000 times between 2012 and 2018 [1]. Perhaps the epitome of this trend is the subfield of natural language processing (NLP) that over the past three years has experienced even sharper growth in model size and corresponding computational requirements in the word embedding approaches (e.g. ELMo, BERT, openGPT-2, Megatron-LM, T5, and GPT-3, one of the largest models ever trained with 175B dense parameters) that are now the basic building blocks of nearly all NLP models. Recent studies indicate that this trend is both environmentally unfriendly and prohibitively expensive, raising barriers to participation in NLP research [2,3]. The goal of this seminar was to mitigate these concerns and promote equity of access in NLP. References. [1] D. Amodei and D. Hernandez. 2018. AI and Compute. https://openai.com/blog/ai-and-compute [2] R. Schwartz, D. Dodge, N. A. Smith, and O. Etzioni. 2020. Green AI. Communications of the ACM (CACM) [3] E. Strubell, A. Ganesh, and A. McCallum. 2019. Energy and Policy Considerations for Deep Learning in NLP. In Proc. of ACL.

Cite as

Jesse Dodge, Iryna Gurevych, Roy Schwartz, Emma Strubell, and Betty van Aken. Efficient and Equitable Natural Language Processing in the Age of Deep Learning (Dagstuhl Seminar 22232). In Dagstuhl Reports, Volume 12, Issue 6, pp. 14-27, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023)


Copy BibTex To Clipboard

@Article{dodge_et_al:DagRep.12.6.14,
  author =	{Dodge, Jesse and Gurevych, Iryna and Schwartz, Roy and Strubell, Emma and van Aken, Betty},
  title =	{{Efficient and Equitable Natural Language Processing in the Age of Deep Learning (Dagstuhl Seminar 22232)}},
  pages =	{14--27},
  journal =	{Dagstuhl Reports},
  ISSN =	{2192-5283},
  year =	{2023},
  volume =	{12},
  number =	{6},
  editor =	{Dodge, Jesse and Gurevych, Iryna and Schwartz, Roy and Strubell, Emma and van Aken, Betty},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/DagRep.12.6.14},
  URN =		{urn:nbn:de:0030-drops-174549},
  doi =		{10.4230/DagRep.12.6.14},
  annote =	{Keywords: deep learning, efficiency, equity, natural language processing (nlp)}
}
  • Refine by Author
  • 1 Dodge, Jesse
  • 1 Gurevych, Iryna
  • 1 Schwartz, Roy
  • 1 Strubell, Emma
  • 1 van Aken, Betty

  • Refine by Classification
  • 1 Computing methodologies → Natural language processing
  • 1 Computing methodologies → Neural networks
  • 1 Social and professional topics → Sustainability

  • Refine by Keyword
  • 1 deep learning
  • 1 efficiency
  • 1 equity
  • 1 natural language processing (nlp)

  • Refine by Type
  • 1 document

  • Refine by Publication Year
  • 1 2023

Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail