Search Results

Background. Evaluating automated Software Requirements Specification (SRS) generation is challenging because few datasets provide fine-grained traceability between source requirements, intermediate elicitation artifacts, and generated specifications. Aims. We study whether legacy SRS documents can serve as controlled reference evidence for constructing traceable synthetic pre-SRS artifacts that enable fine-grained evaluation of LLM-based requirements transformations. Method. We conduct an empirical study using ReqGenX, a controlled pipeline that decomposes SRS sections into atomic statements grounded in the SRS content, routes atoms to standards-inspired artifact types through multi-LLM plurality voting, and generates artifacts using constrained prompts with iterative judge-guided refinement. We evaluate ReqGenX on seven PURE SRS documents using grounding, quality, information retention, and downstream reconstruction analyses. Results. ReqGenX produces highly grounded, well-formed atoms that score strongly on our atomicity and downstream-utility criteria, with median AlignScore values typically between 0.96 and 0.99 and Prometheus scores ranging from 4.34 to 4.85. Generated artifacts remain strongly grounded in their source atoms, with AlignScore values typically between 0.80 and 0.94 and judge pass rates near 100%; stricter Prometheus evaluation yields pass rates ranging from 54.8% to 97.1%. In a downstream SRS reconstruction case study, artifact-backed atoms remain recoverable from generated SRSs, with SBERT means between 0.69 and 0.75 and AlignScore medians between 0.76 and 0.84. Conclusions. Traceable synthetic pre-SRS artifacts can support fine-grained evaluation of LLM-based SRS generation, while exposing tradeoffs among faithfulness, information retention, and artifact completeness. These artifacts are intended as controlled evaluation resources rather than substitutes for primary elicitation evidence.