Document Open Access Logo

Cost-Effective Hate Speech Detection in Portuguese Using Lightweight LLMs

Authors Mauro Cardoso , Eugénio Ribeiro , Fernando Batista

PDF

File

Thumbnail PDF
PDF
OASIcs.SLATE.2026.9.pdf
  • Filesize: 1.41 MB
  • 16 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Computing methodologies → Natural language processing
  • Human-centered computing → Social network analysis
  • Computing methodologies → Language resources
  • Applied computing → Sociology
Keywords
  • Hate Speech Detection
  • Zero-shot Classification
  • Lightweight Language Models

Metrics

Abstract

Addressing Online Hate Speech (OHS) in Portuguese faces significant challenges, including a lack of comprehensive, annotated data and classification frameworks, effective detection tools, and the high computational cost of Large Language Models (LLMs). This work evaluates the efficiency and competitiveness of seven smaller, more accessible LLMs (ranging from 3B to 27B parameters) through zero-shot classification with structured instructions, leveraging the expert-annotated test dataset and annotation scheme of the kNOwHATE project. Findings indicate that models such as Mistral Small 3.2 24B, Gemma 3 27B, and Phi-4 achieve competitive performance, balancing precision and recall, whilst remaining computationally affordable. While excelling in identifying direct and indirect hate speech, out-group derogation, and specific target groups, the models faced challenges in detecting subtle rhetorical devices and emotions. Analysis of the Cohen’s Kappa and F1-scores revealed moderate agreement with human annotators, highlighting the potential of optimised models to democratise OHS detection in resource-constrained settings, alongside the need for further refinement to bridge the gap in human-level nuance and improve generalisation. This study aims to broaden knowledge regarding the detection of OHS in Portuguese, by demonstrating a viable, competitive, and low-cost approach that does not rely on large-scale infrastructure or expensive proprietary models.

Cite As Get BibTex

Mauro Cardoso, Eugénio Ribeiro, and Fernando Batista. Cost-Effective Hate Speech Detection in Portuguese Using Lightweight LLMs. In 15th Symposium on Languages, Applications and Technologies (SLATE 2026). Open Access Series in Informatics (OASIcs), Volume 144, pp. 9:1-9:16, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/OASIcs.SLATE.2026.9

Author Details

Mauro Cardoso
  • Instituto Universitário de Lisboa (ISCTE-IUL), Portugal
  • INESC-ID Lisboa, Portugal
Eugénio Ribeiro
  • INESC-ID Lisboa, Portugal
  • ISTAR, Instituto Universitário de Lisboa (ISCTE-IUL), Portugal
Fernando Batista
  • Instituto Universitário de Lisboa (ISCTE-IUL), Portugal
  • INESC-ID Lisboa, Portugal

Funding

This work was supported by Portuguese national funds through Fundação para a Ciência e a Tecnologia, I.P. (FCT) under projects UID/50021/2025 (DOI: https://doi.org/10.54499/UID/50021/2025), UID/PRR/50021/2025 (DOI: https://doi.org/10.54499/UID/PRR/50021/2025), and UID/04466/2025 (DOI: https://doi.org/10.54499/UID/04466/2025).

References

  1. Aish Albladi, Minarul Islam, Amit Das, Maryam Bigonah, Zheng Zhang, Fatemeh Jamshidi, Mostafa Rahgouy, Nilanjana Raychawdhary, Daniela Marghitu, and Cheryl Seals. Hate Speech Detection Using Large Language Models: A Comprehensive Review. IEEE Access, 13:20871-20892, 2025. URL: https://doi.org/10.1109/ACCESS.2025.3532397.
  2. Gabriel Assis, Annie Amorim, Jonnathan Carvalho, Daniel de Oliveira, Daniela Vianna, and Aline Paes. Exploring Portuguese Hate Speech Detection in Low-Resource Settings: Lightly Tuning Encoder Models or In-Context Learning of Large Models? In Proceedings of the 16th International Conference on Computational Processing of Portuguese - Vol. 1, pages 301-311. Association for Computational Lingustics, 2024. URL: https://aclanthology.org/2024.propor-1.31/.
  3. Basel Barakat and Sardar Jaf. Beyond Traditional Classifiers: Evaluating Large Language Models for Robust Hate Speech Detection. Computation, 13(8), 2025. URL: https://doi.org/10.3390/computation13080196.
  4. Mauro Cardoso, Eugénio Ribeiro, and Fernando Batista. Portuguese Far-Right Discourse on Social Media: Insights from Topic Modeling. In 14th Symposium on Languages, Applications and Technologies (SLATE 2025), volume 135 of Open Access Series in Informatics (OASIcs), pages 12:1-12:16. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2025. URL: https://doi.org/10.4230/OASIcs.SLATE.2025.12.
  5. Amanda et al. da Silva Oliveira. Toxic Speech Detection in Portuguese: A Comparative Study of Large Language Models. In Proceedings of the 16th International Conference on Computational Processing of Portuguese - Vol. 1, pages 108-116. Association for Computational Lingustics, 2024. URL: https://aclanthology.org/2024.propor-1.11/.
  6. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM), pages 512-515, 2017. URL: https://doi.org/10.1609/icwsm.v11i1.14955.
  7. J. Angel Diaz-Garcia and Joao Paulo Carvalho. A Literature Review of Textual Cyber Abuse Detection Using Cutting-Edge Natural Language Processing Techniques: Language Models and Large Language Models. WIREs Data Mining and Knowledge Discovery, 15(3):e70029, 2025. URL: https://doi.org/10.1002/widm.70029.
  8. Amanda Oliveira et al. How Good Is ChatGPT For Detecting Hate Speech In Portuguese? In Anais do XIV Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 94-103, Porto Alegre, RS, Brasil, 2023. SBC. URL: https://doi.org/10.5753/stil.2023.233943.
  9. Amanda Oliveira et al. Toxic Text Classification in Portuguese: Is LLaMA 3.1 8B All You Need? In Anais do XV Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 57-66. SBC, 2024. URL: https://doi.org/10.5753/stil.2024.245416.
  10. Amit Das et al. Investigating Annotator Bias in Large Language Models for Hate Speech Detection. ArXiv, 2024. URL: https://doi.org/10.48550/arXiv.2406.11109.
  11. Gemma Team et al. Gemma 3 technical report, 2025. URL: https://doi.org/10.48550/arXiv.2503.19786.
  12. Microsoft et. al. Phi-4 technical report. arXiv, 2024. URL: https://arxiv.org/abs/2412.08905v1.
  13. Zahra Safdari Fesaghandis and Suman Kalyan Maity. Multilingual hate speech detection and counterspeech generation: A comprehensive survey and practical guide. arXiv preprint arXiv:2603.19279, 2026. URL: https://doi.org/10.48550/arXiv.2603.19279.
  14. Pedro Fialho, Ricardo Ribeiro, Fernando Batista, Gil Ramos, António Fonseca, Sérgio Moro, Rita Guerra, Paula Carvalho, Catarina Marques, and Cláudia Silva. Counter Hate Speech Detection in Youtube Conversations, pages 94-105. Springer Nature Switzerland, 2025. URL: https://doi.org/10.1007/978-3-031-73997-2_9.
  15. Faeze Ghorbanpour, Daryna Dementieva, and Alexander Fraser. Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study. In Proceedings of the The 9th Workshop on Online Abuse and Harms (WOAH), pages 413-425, Vienna, Austria, August 2025. Association for Computational Linguistics. URL: https://aclanthology.org/2025.woah-1.39/.
  16. Google DeepMind. Gemini 2.0 flash lite: Documentation and technical overview. https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/2-0-flash-lite, 2024. Access: April 2026.
  17. Rita Guerra, Fernando Batista, Paula Carvalho, and Ricardo Ribeiro. Data (Golden set/Test files), November 2025. URL: https://doi.org/10.17605/OSF.IO/MPEJ3.
  18. Rita Guerra, Paula Carvalho, Catarina Marques, Margarida Carmona, Rodrigo Sarroeira, Fernando Batista, Ricardo Ribeiro, António Fonseca, Sérgio Moro, and Cláudia Silva. Unpacking online hate speech in Portuguese social media: a social-psychological and linguistic-discursive approach. Humanities and Social Sciences Communications, 12:1709, 2025. URL: https://doi.org/10.1057/s41599-025-05392-9.
  19. Tharindu Kumarage, Amrita Bhattacharjee, and Joshua Garland. Harnessing Artificial Intelligence to Combat Online Hate: Exploring the Challenges and Opportunities of Large Language Models in Hate Speech Detection. arXiv, 2024. URL: https://doi.org/10.48550/arXiv.2403.08035.
  20. Tiago Lapa and Branco di Fátima. Hate Speech Among Security Forces in Portugal. Communication Library, 7:277-293, 2023. URL: https://doi.org/10.25768/654-916-9.
  21. Luca Manucci. Portuguese Populism: People, Parties, and Politics. Análise Social, 59(2 (251)):2-11, 2024. URL: https://doi.org/10.31447/202200.
  22. Meta. Llama 3.1: Model cards & prompt formats. https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_1/, 2024. Access: April 2026.
  23. Meta. Llama 3.2: Model cards & prompt formats. https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_2/, 2024. Access: April 2026.
  24. Mistral AI. Mistral-nemo: Architecture and performance report. https://mistral.ai/news/mistral-nemo, 2024. Access: April 2026.
  25. Mistral AI. Mistral 3: Release notes and benchmarks. https://mistral.ai/news/mistral-small-3, 2025. Access: April 2026.
  26. Flor Miriam Plaza-del arco, Debora Nozza, and Dirk Hovy. Respectful or Toxic? Using Zero-Shot Learning with Language Models to Detect Hate Speech. In The 7th Workshop on Online Abuse and Harms (WOAH), pages 60-68. Association for Computational Linguistics, 2023. URL: https://doi.org/10.18653/v1/2023.woah-1.6.
  27. Fabio Poletto, Valerio Basile, Manuela Sanguinetti, Cristina Bosco, and Viviana Patti. Resources and benchmark corpora for hate speech detection: a systematic review. Language Resources and Evaluation, 55:477-523, June 2021. URL: https://doi.org/10.1007/s10579-020-09502-8.
  28. Catarina Pontes, António Fonseca, Sérgio Moro, Fernando Batista, Ricardo Ribeiro, Catarina Marques, Paula Carvalho, Cláudia Silva, and Rita Guerra. Unveiling Patterns of Hate Speech in the Portuguese Sphere: A Social Network Analysis Approach, pages 70-81. Springer Nature Switzerland, 2025. URL: https://doi.org/10.1007/978-3-031-73997-2_7.
  29. Gil Ramos, Fernando Batista, Ricardo Ribeiro, Pedro Fialho, Sérgio Moro, António Fonseca, Rita Guerra, Paula Carvalho, Catarina Marques, and Cláudia Silva. A comprehensive review on automatic hate speech detection in the age of the transformer. Social Network Analysis and Mining, 14:204, 2024. URL: https://doi.org/10.1007/s13278-024-01361-3.
  30. Gil Ramos, Fernando Batista, Ricardo Ribeiro, Pedro Fialho, Sérgio Moro, António Fonseca, Rita Guerra, Paula Carvalho, Catarina Marques, and Cláudia Silva. Leveraging Transfer Learning for Hate Speech Detection in Portuguese Social Media Posts. IEEE Access, 12:101374-101389, 2024. URL: https://doi.org/10.1109/ACCESS.2024.3430848.
  31. Natalia Umansky, Maël Kubli, Ana Kotarcic, Laura Bronner, Selina Kurer, Philip Grech, Dominik Hangartner, Fabrizio Gilardi, and Karsten Donnay. Improving hate speech detection with large language models. European Journal of Political Research, pages 1-12, January 2026. URL: https://doi.org/10.1017/S1475676525100546.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact