2026-10-08T14:23:30Z https://drops.dagstuhl.de/oai

oai:drops-oai.dagstuhl.de:23681 2025-11-12T11:50:38Z ddc:004 open_access

From Prediction to Precision: Leveraging LLMs for Equitable and Data-Driven Writing Placement in Developmental Education Da Corte, Miguel Baptista, Jorge Large Language Models (LLMs) Developmental Education (DevEd) writing assessment text classification English writing proficiency Accurate text classification and placement remain challenges in U.S. higher education, with traditional automated systems like Accuplacer functioning as "black-box" models with limited assessment transparency. This study evaluates Large Language Models (LLMs) as complementary placement tools by comparing their classification performance against a human-rated gold standard and Accuplacer. A 450-essay corpus was classified using Claude, Gemini, GPT-3.5-turbo, and GPT-4o across four prompting strategies: Zero-shot, Few-shot, Enhanced, and Enhanced+ (definitions with examples). Two classification approaches were tested: (i) a 1-step, 3 class classification task, distinguishing DevEd Level 1, DevEd Level 2, and College-level texts in one single run; and (ii) a 2-step classification task, first separating College vs. Non-College texts before further classifying Non-College texts into DevEd sublevels. The results show that structured prompt refinement improves the precision of LLMs' classification, with Claude Enhanced + achieving 62.22% precision (1 step) and Gemini Enhanced + reaching 69.33% (2 step), both surpassing Accuplacer (58.22%). Gemini and Claude also demonstrated strong correlation with human ratings, with Claude achieving the highest Pearson scores (ρ = 0.75; 1-step, ρ = 0.73; 2-step) vs. Accuplacer (ρ = 0.67). While LLMs show promise for DevEd placement, their precision remains a work in progress, highlighting the need for further refinement and safeguards to ensure ethical and equitable placement. Schloss Dagstuhl – Leibniz-Zentrum für Informatik Miguel Da Corte and Jorge Baptista 2025 Is Part Of OASIcs, Volume 135, 14th Symposium on Languages, Applications and Technologies (SLATE 2025) InProceedings Text doc-type:ResearchArticle publishedVersion application/pdf doi:10.4230/OASIcs.SLATE.2025.1 urn:nbn:de:0030-drops-236817 https://drops.dagstuhl.de/entities/document/10.4230/OASIcs.SLATE.2025.1 eng https://creativecommons.org/licenses/by/4.0/legalcode