Document Open Access Logo

Beyond the Score: Exploring the Intersection Between Sociodemographics and Linguistic Features in English (L1) Writing Placement

Authors Miguel Da Corte , Jorge Baptista

PDF HTML 5

Files

Thumbnail PDF
PDF
OASIcs.SLATE.2025.6.pdf
  • Filesize: 0.64 MB
  • 18 pages
HTML5 Logo HTML (experimental)
OASIcs.SLATE.2025.6.html

Document Identifiers

Subject Classification

ACM Subject Classification
  • Social and professional topics → Student assessment
  • Social and professional topics → Adult education
  • Computing methodologies → Language resources
  • Computing methodologies → Lexical semantics
  • Social and professional topics → Race and ethnicity
  • Social and professional topics → Men
  • Social and professional topics → Women
Keywords
  • Developmental Education (DevEd)
  • sociolinguistic variation
  • text classification
  • Machine Learning
  • placement equity

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

This study examines the intersection of sociodemographic characteristics, linguistic features, and writing placement outcomes at a community college in the United States of America. It focuses on 210 anonymized writing samples from native English speakers (L1) that were automatically classified by Accuplacer and independently assessed by two trained raters. Disparities across gender and race using 40 top-ranked linguistic features selected from Coh-Metrix, CTAP, and Developmental Education-Specific (DES) sets were analyzed. Three statistical tests were used: one-way ANOVA, Tukey’s HSD, and Chi-square. ANOVA results showed racial differences in nine linguistic features, especially those tied to syntactic complexity, discourse markers, and lexical precision. Gender differences were more limited, with only one feature reaching significance (Positive Connectives, p = 0.007). Tukey’s HSD pairwise tests showed no significant gender group variation but revealed sensitivity in DES features when comparing racial groups. Chi-square analysis indicated no significant association between gender and placement outcomes but suggested a possible link between race and human-assigned levels (χ² = 9.588, p = 0.048). These findings suggest that while automated systems assess general writing skills, human-devised linguistic features and demographic insights can support more equitable placement practices for all students entering college-level programs.

Cite As Get BibTex

Miguel Da Corte and Jorge Baptista. Beyond the Score: Exploring the Intersection Between Sociodemographics and Linguistic Features in English (L1) Writing Placement. In 14th Symposium on Languages, Applications and Technologies (SLATE 2025). Open Access Series in Informatics (OASIcs), Volume 135, pp. 6:1-6:18, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.SLATE.2025.6

Author Details

Miguel Da Corte
  • University of Algarve, Campus de Gambelas, Faro, Portugal
  • INESC-ID Lisboa, Portugal
Jorge Baptista
  • University of Algarve, Campus de Gambelas, Faro, Portugal
  • INESC-ID Lisboa, Portugal

Funding

This work was supported by Portuguese national funds through FCT (Reference: UIDB/50021/2020, DOI: https://doi.org/10.54499/UIDB/50021/2020) and by the European Commission (Project: iRead4Skills, Grant number: 1010094837, Topic: HORIZON-CL2-2022-TRANSFORMATIONS-01-07, DOI: https://doi.org/10.3030/101094837).

References

  1. Hervé Abdi and Lynne J Williams. Tukey’s Honestly Significant Difference (HSD) test. Encyclopedia of Research Design, 3(1):1-5, 2010. Google Scholar
  2. Lisa R Arnold, Lei Jiang, and Holly Hassel. After implementation: Assessing student self-placement in college writing programs. Journal of Writing Assessment, 17(1), 2024. Google Scholar
  3. Elisabeth A Barnett, Elizabeth Kopko, Dan Cullinan, and Clive R Belfield. Who should take college-level courses? Impact findings from an evaluation of a multiple measures assessment strategy. Center for the Analysis of Postsecondary Readiness, 2020. Google Scholar
  4. Jessie S Barrot and Joan Y Agdeppa. Complexity, accuracy, and fluency as indices of college-level L2 writers’ proficiency. Assessing Writing, 47:100510, 2021. Google Scholar
  5. Susan Bickerstaff, Katie Beal, Julia Raufman, Erika B Lewy, and Austin Slaughter. Five principles for reforming Developmental Education: A review of the evidence. Center for the Analysis of Postsecondary Readiness, pages 1-8, 2022. Google Scholar
  6. Carolina Castillo and Natalia Ávila Reyes. Students’ sociodemographic characteristics and writing performance: A systematic literature review. Reading and Writing, pages 1-37, 2025. Google Scholar
  7. Xiaobin Chen and Detmar Meurers. CTAP: A Web-Based Tool Supporting Automatic Complexity Analysis. In Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity (CL4LC), pages 113-119, Osaka, Japan, December 2016. The COLING 2016 Organizing Committee. URL: https://aclanthology.org/W16-4113.
  8. Miguel Da Corte and Jorge Baptista. Charting the linguistic landscape of developing writers: An annotation scheme for enhancing native language proficiency. In Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue, editors, Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 3046-3056, Torino, Italia, May 2024. ELRA and ICCL. URL: https://aclanthology.org/2024.lrec-main.272/.
  9. Miguel Da Corte and Jorge Baptista. Enhancing writing proficiency classification in Developmental Education: The quest for accuracy. In Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue, editors, Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 6134-6143, Torino, Italia, May 2024. ELRA and ICCL. URL: https://aclanthology.org/2024.lrec-main.542/.
  10. Miguel Da Corte and Jorge Baptista. Leveraging NLP and machine learning for English (l1) writing assessment in developmental education. In Proceedings of the 16th International Conference on Computer Supported Education (CSEDU 2024), 2-4 May, 2024, Angers, France, volume 2, pages 128-140, 2024. URL: https://doi.org/10.5220/0012740500003693.
  11. Miguel Da Corte and Jorge Baptista. Refining English writing proficiency assessment and placement in Developmental Education using NLP tools and Machine Learning. In Proceedings of the 17th International Conference on Computer Supported Education - Volume 2: CSEDU, pages 288-303. INSTICC, SciTePress, 2025. URL: https://doi.org/10.5220/0013351500003932.
  12. Miguel Da Corte and Jorge Baptista. Toward consistency in writing proficiency assessment: Mitigating classification variability in Developmental Education. In Proceedings of the 17th International Conference on Computer Supported Education - Volume 2: CSEDU, pages 139-150. INSTICC, SciTePress, 2025. URL: https://doi.org/10.5220/0013353900003932.
  13. Jane Denison-Furness, Stacey Lee Donohue, Annemarie Hamlin, and Tony Russell. Welcome/Not Welcome: From Discouragement to Empowerment in the Writing Placement Process at Central Oregon Community College. In Jassica Nastal, Mya Poe, and Christie Toth, editors, Writing Placement in Two-Year Colleges: The Pursuit of Equity in Postsceondary Education, pages 107-127. The WAC Clearinghouse/University Press of Colorado, 2022. URL: https://doi.org/10.37514/PRA-B.2022.1565.2.04.
  14. Martin East and David Slomp. The ethical turn in writing assessment: How far have we come, and where do we still need to go? Language Teaching, 57(2):262-273, 2024. Google Scholar
  15. Nikki Edgecombe and Michael Weiss. Promoting equity in Developmental Education reform: A conversation with Nikki Edgecombe and Michael Weiss. Center for the Analysis of Postsecondary Readiness, page 1, 2024. Google Scholar
  16. Elizabeth Ganga and Amy Mazzariello. Modernzing college course placement by using multiple measures. Education Commission of the States, pages 1-9, 2019. URL: https://postsecondaryreadiness.org/wp-content/uploads/2019/03/Modernizing_College_Course_Placement_by_Using_Multiple_Measures_Final.pdf.
  17. Sandra Götz and Sylviane Granger. Learner corpus research for pedagogical purposes: An overview and some research perspectives. International Journal of Learner Corpus Research, 10(1):1-38, 2024. Google Scholar
  18. Sarah Hirsch, Kenny Smith, and Madeleine Sorapure. Collaborative writing placement: Partnering with students in the placement process. Journal of Writing Assessment, 17(2), 2024. Google Scholar
  19. Darin L Jensen and Joanne Baird Giordano. Afterword. Placement, equity, and the promise of democratic open-access education. Writing Placement in Two-Year Colleges: The Pursuit of Equity in Postsecondary Education, pages 279-86, 2022. Google Scholar
  20. Young-Suk Grace Kim, Christopher Schatschneider, Jeanne Wanzek, Brandy Gatlin, and Stephanie Al Otaiba. Writing evaluation: Rater and task effects on the reliability of writing scores for children in grades 3 and 4. Reading and writing, 30:1287-1310, 2017. Google Scholar
  21. Holly Kosiewicz, Cristián Morales, and Kalena E. Cortes. The "missing English learner"' in higher education: How identification, assessment, and placement shape the educational outcomes of English learners in community colleges. In Higher Education: Handbook of Theory and Research: Volume 39, pages 1-55. Springer, 2023. Google Scholar
  22. Josh Lederman. Validity and racial justice in educational assessment. Applied Measurement in Education, 36(3):242-254, 2023. URL: https://doi.org/10.1080/08957347.2023.2214654.
  23. Bruce W Lee and Jason Hyung-Jong Lee. Prompt-based learning for text readability assessment. In In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 1-19. Toronto, Canada. Association forComputational Linguistics., 2023. Google Scholar
  24. Stephanie Link and Svetlana Koltovskaia. Automated Scoring of Writing, pages 333-345. Springer International Publishing, Cham, 2023. URL: https://doi.org/10.1007/978-3-031-36033-6_21.
  25. Susan Lyons, Maria Elena Oliveri, and Mya Poe. A framework for enacting equity aims in assessment use: A justice-oriented approach. In Culturally Responsive Assessment in Classrooms and Large-Scale Contexts, pages 88-105. Routledge, 2025. Google Scholar
  26. Ross Markle. Redesigning course placement in service of guided pathways. Educational Considerations, 50(2):8, 2025. Google Scholar
  27. Michael Matta and Narmene Hamsho and. Consequences of response formats on racial and ethnic bias and fairness in writing assessments. School Psychology Review, 0(0):1-14, 2025. URL: https://doi.org/10.1080/2372966X.2025.2462984.
  28. Danielle S McNamara, Yasuhiro Ozuru, Arthur C Graesser, and Max Louwerse. Validating CoH-Metrix. In Proceedings of the 28th annual Conference of the Cognitive Science Society, pages 573-578, 2006. Google Scholar
  29. Neil Murray and Gerard Sharpling and. What traits do academics value in student writing? Insights from a psychometric approach. Assessment & Evaluation in Higher Education, 44(3):489-500, 2019. URL: https://doi.org/10.1080/02602938.2018.1521372.
  30. Jessica Nastal. Beyond tradition: Writing placement, fairness, and success at a two-year college. Journal of Writing Assessment, 12(1), 2019. Google Scholar
  31. Jessica Nastal and Kris Messer. Afterword: Finding the right note in writing placement. Journal of Writing Assessment, 18(1), 2025. Google Scholar
  32. Mari Nygård and Anne Kathrine Hundal. Features of grammatical writing competence among early writers in a Norwegian school context. Languages, 9(1):29, 2024. Google Scholar
  33. María Elena Oliveri, René Lawless, and Robert J. Mislevy. Using evidence-centered design to support the development of culturally and linguistically sensitive collaborative problem-solving assessments. International Journal of Testing, 19(3):270-300, 2019. URL: https://doi.org/10.1080/15305058.2018.1543308.
  34. Ramona T Pittman, Lynette O’Neal, Kimberly Wright, and Brittany R White. Elevating students’ oral and written language: Empowering African American students through language. Education Sciences, 14(11):1191, 2024. Google Scholar
  35. Mya Poe, Jessica Nastal, and Norbert Elliot. Reflection. An admitted student is a qualified student: A roadmap for writing placement in the two-year college. Journal of Writing Assessment, 12(1), 2019. Google Scholar
  36. Jennifer Reid, Mahshid Ahmadian, D. Jennings, Anathea Abad Pepperl, and et.al. Saying it aloud: Inclusive teaching statements impact on sense of belonging and engagement. Journal of College Science Teaching, 0(0):1-14, 2025. URL: https://doi.org/10.1080/0047231X.2025.2487437.
  37. Rachael Ruegg. Assessment of written assignments in first-year Humanities and Social Sciences courses: Textual features of academic writing. Assessment in Education: Principles, Policy & Practice, 32(1):60-76, 2025. URL: https://doi.org/10.1080/0969594X.2025.2467672.
  38. Padam Singh. P value, statistical significance and clinical significance. Journal of Clinical and Preventive Cardiology, 2(4):202-204, 2013. Google Scholar
  39. J Suárez-Álvarez, M Oliveri, A Zenisky, and SG Sireci. Current assessment needs in adult education and workforce development: Summary report (center for educational assessment report no. 998). Center for Educational Assessment, 2023. URL: https://createadultskills.org/system/files/ASAP%20Brief%201_Nov%202023_Needs%20Assessment.pdf.
  40. Meghan Sweeney and Crystal Colombini. (Re) placing personalis: A study of placement reform and self-construction in mission-driven contexts. Journal of Writing Assessment, 17(1), 2024. Google Scholar
  41. The College Board. ACCUPLACER Program Manual. (online), 2022. URL: https://secure-media.collegeboard.org/digitalServices/pdf/accuplacer/accuplacer-program-manual.pdf.
  42. Rodrigo Wilkens, David Alfter, Xiaoou Wang, Alice Pintard, Anaïs Tack, Kevin P Yancey, and Thomas François. Fabra: French aggregator-based readability assessment toolkit. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 1217-1233, 2022. URL: https://aclanthology.org/2022.lrec-1.130.
  43. Sachiko Yasuda. What does it mean to construct an argument in academic writing? a synthesis of English for general academic purposes and English for specific academic purposes perspectives. Journal of English for Academic Purposes, 66:101307, 2023. Google Scholar
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact