2026-10-06T22:28:39Z https://drops.dagstuhl.de/oai

oai:drops-oai.dagstuhl.de:28054 2026-10-05T06:44:06Z ddc:004 open_access

CORA: Config Risk Analyzer - Predicting Risk of Configuration Changes at Scale Sun, Weiyan Mockus, Audris Ellis, Brian Ge, Jun Kim, Madigan Kumar, Sahil Singh, Gursharan Steiner, Matt Uppalapati, Siri Nagappan, Nachiappan Configuration changes risk prediction defect prediction code freeze software supply chain gradient boosting service reliability Configuration changes - modifications to feature flags and service parameters - are important to operating services, but they can cause severe outages (SEVs) with substantial service disruption. While code diff risk prediction is well studied, configuration diff risk has received little attention. The code-oriented Diff Risk Score (DRS) model provides limited discriminatory power for config diffs because config risk stems not from code complexity but from the importance, blast radius, and operational readiness of the affected services. We describe the development of CORA (COnfig Risk Analyzer), a dedicated risk model for config diffs. A key insight is that the config-path-to-service mapping bridges from what changed to which services are affected, enabling risk assessment in terms of service criticality rather than code properties. CORA evolved through three iterations. A logistic regression model with config-specific features achieved an 11% improvement in recall at 5% gating over the DRS model on config diffs. Used for a freeze period, it reduced config gating while outage impact decreased and config diff landing volume increased 91.4%. A unified LightGBM model trained on Meta-wide data achieved a 25.95% improvement in recall at 10% gating, with organization-level improvements ranging from 15% to 66%. An enhanced model with enriched service features and SEV-severity-aware training (having also explored Bayesian hyperparameter optimization) achieved a 22% improvement in recall at 15% gating, intercepting meaningful additional high-severity outage impact in backtesting. To our knowledge, CORA is the first system to apply predictive risk modeling specifically to configuration changes. Schloss Dagstuhl – Leibniz-Zentrum für Informatik Weiyan Sun and Audris Mockus and Brian Ellis and Jun Ge and Madigan Kim and Sahil Kumar and Gursharan Singh and Matt Steiner and Siri Uppalapati and Nachiappan Nagappan 2026 Is Part Of LIPIcs, Volume 394, 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026) InProceedings Text doc-type:ResearchArticle publishedVersion application/pdf doi:10.4230/LIPIcs.ESEM.2026.86 urn:nbn:de:0030-drops-280542 https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ESEM.2026.86 eng https://creativecommons.org/licenses/by/4.0/legalcode