Search Results

Documents authored by Mockus, Audris

Document
Software Engineering in Practice Track Paper
DOI: 10.4230/LIPIcs.ESEM.2026.86
CORA: Config Risk Analyzer - Predicting Risk of Configuration Changes at Scale

Authors: Weiyan Sun, Audris Mockus, Brian Ellis, Jun Ge, Madigan Kim, Sahil Kumar, Gursharan Singh, Matt Steiner, Siri Uppalapati, and Nachiappan Nagappan

Published in: LIPIcs, Volume 394, 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)

Abstract
Configuration changes - modifications to feature flags and service parameters - are important to operating services, but they can cause severe outages (SEVs) with substantial service disruption. While code diff risk prediction is well studied, configuration diff risk has received little attention. The code-oriented Diff Risk Score (DRS) model provides limited discriminatory power for config diffs because config risk stems not from code complexity but from the importance, blast radius, and operational readiness of the affected services. We describe the development of CORA (COnfig Risk Analyzer), a dedicated risk model for config diffs. A key insight is that the config-path-to-service mapping bridges from what changed to which services are affected, enabling risk assessment in terms of service criticality rather than code properties. CORA evolved through three iterations. A logistic regression model with config-specific features achieved an 11% improvement in recall at 5% gating over the DRS model on config diffs. Used for a freeze period, it reduced config gating while outage impact decreased and config diff landing volume increased 91.4%. A unified LightGBM model trained on Meta-wide data achieved a 25.95% improvement in recall at 10% gating, with organization-level improvements ranging from 15% to 66%. An enhanced model with enriched service features and SEV-severity-aware training (having also explored Bayesian hyperparameter optimization) achieved a 22% improvement in recall at 15% gating, intercepting meaningful additional high-severity outage impact in backtesting. To our knowledge, CORA is the first system to apply predictive risk modeling specifically to configuration changes.
Cite as

Weiyan Sun, Audris Mockus, Brian Ellis, Jun Ge, Madigan Kim, Sahil Kumar, Gursharan Singh, Matt Steiner, Siri Uppalapati, and Nachiappan Nagappan. CORA: Config Risk Analyzer - Predicting Risk of Configuration Changes at Scale. In 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 394, pp. 86:1-86:22, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026)

Copy BibTex To Clipboard

@InProceedings{sun_et_al:LIPIcs.ESEM.2026.86,
  author =	{Sun, Weiyan and Mockus, Audris and Ellis, Brian and Ge, Jun and Kim, Madigan and Kumar, Sahil and Singh, Gursharan and Steiner, Matt and Uppalapati, Siri and Nagappan, Nachiappan},
  title =	{{CORA: Config Risk Analyzer - Predicting Risk of Configuration Changes at Scale}},
  booktitle =	{20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)},
  pages =	{86:1--86:22},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-450-5},
  ISSN =	{1868-8969},
  year =	{2026},
  volume =	{394},
  editor =	{Feldt, Robert and Paasivaara, Maria and Mendez, Daniel and Wagner, Stefan and Bar\'{o}n, Marvin Mu\~{n}oz},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ESEM.2026.86},
  URN =		{urn:nbn:de:0030-drops-280542},
  doi =		{10.4230/LIPIcs.ESEM.2026.86},
  annote =	{Keywords: Configuration changes, risk prediction, defect prediction, code freeze, software supply chain, gradient boosting, service reliability}
}
Document
Software Engineering in Practice Track Paper
DOI: 10.4230/LIPIcs.ESEM.2026.93
A Preliminary Analysis of the Impact of AI Assisted/Generated Code on Quality, Centrality and Review Time at Scale

Authors: Audris Mockus, Peter C. Rigby, Don Stewart, Chandra Maddila, and Nachiappan Nagappan

Published in: LIPIcs, Volume 394, 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)

Abstract
AI code-generation tools are widely deployed, yet AI is not applied uniformly: its use varies systematically with the context of the task, the code being modified, and the engineer. Studies that do not account for these contextual differences risk attributing to AI what is actually driven by context. Using data from over 10K developers and 400K diffs at Meta, with character-level provenance tracing of AI-suggested code through editing, review, and landing, we model (a) which context factors predict AI code landing, (b) how landed AI code relates to review time, and (c) its association with production outages (SEVs) - while controlling for contextual confounds. We find that AI code lands more often in less central code, larger changes, test files, and for authors with higher tenure. Review time decreases and SEV rates are lower for diffs with AI code, but both associations are confounded by AI code appearing in less central contexts. These findings demonstrate that naive comparisons of AI vs. non-AI work may reach misleading conclusions, and we provide actionable guidance for controlling these confounds.
Cite as

Audris Mockus, Peter C. Rigby, Don Stewart, Chandra Maddila, and Nachiappan Nagappan. A Preliminary Analysis of the Impact of AI Assisted/Generated Code on Quality, Centrality and Review Time at Scale. In 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 394, pp. 93:1-93:23, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026)

Copy BibTex To Clipboard

@InProceedings{mockus_et_al:LIPIcs.ESEM.2026.93,
  author =	{Mockus, Audris and Rigby, Peter C. and Stewart, Don and Maddila, Chandra and Nagappan, Nachiappan},
  title =	{{A Preliminary Analysis of the Impact of AI Assisted/Generated Code on Quality, Centrality and Review Time at Scale}},
  booktitle =	{20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026)},
  pages =	{93:1--93:23},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-450-5},
  ISSN =	{1868-8969},
  year =	{2026},
  volume =	{394},
  editor =	{Feldt, Robert and Paasivaara, Maria and Mendez, Daniel and Wagner, Stefan and Bar\'{o}n, Marvin Mu\~{n}oz},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ESEM.2026.93},
  URN =		{urn:nbn:de:0030-drops-280619},
  doi =		{10.4230/LIPIcs.ESEM.2026.93},
  annote =	{Keywords: AI code generation, code review, software quality, context-dependent AI, landing rate}
}

Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact