2026-10-05T21:32:41Z https://drops.dagstuhl.de/oai

oai:drops-oai.dagstuhl.de:28011 2026-10-05T06:44:04Z ddc:004 open_access

The Overlooked Spirit of Open Source: LLM-Based Analysis of Non-Explicit Mentoring in OSS Communities Cui, Xing Wu, Jingzheng Luo, Tianyue Ling, Xiang open source software non-explicit mentoring large language models contributor retention Background. The sustainable evolution of Open Source Software (OSS) depends not only on code production but also on continuous knowledge transfer and collaborative support within communities. However, existing contribution metrics primarily center on code commits and merges, overlooking the value of non-coding activities embedded in technical discussions. Among these activities, Non-explicit Mentoring (NEM) refers to guidance-oriented interactions in Pull Request (PR) and Issue discussions, where contributors convey technical knowledge, facilitate problem diagnosis, clarify design rationale, provide normative guidance, and share feedback within natural conversational contexts. Although prior studies acknowledge the existence of such behaviors, systematic quantitative investigations of their scale, structural patterns, and impact remain limited due to reliance on qualitative methods, insufficient automated detection techniques, and the scarcity of high-quality annotated datasets. Aims. To address these challenges, this paper proposes MentoScope, a large language model (LLM)-based automated framework for fine-grained identification and classification of NEM in GitHub collaboration contexts. Method. Built upon Llama-3.1-8B, MentoScope undergoes a three-stage training pipeline. First, continual pre-training (CPT) with LoRA is applied on 92,778 PRs and 118,569 Issues for OSS domain adaptation. Second, supervised fine-tuning (SFT) on 15,214 high-quality annotated samples enables recognition of 8 predefined NEM categories. Third, Odds Ratio Preference Optimization (ORPO) with 3,280 preference pairs aligns model outputs with human judgment to enhance classification robustness. Results. MentoScope achieves F1-scores of 94.20% and 80.13% on NEM identification and classification tasks respectively, significantly outperforming all baselines, with ablation studies confirming the necessity of each training stage. Large-scale analysis of 591,154 comments from 500 OSS projects reveals that NEM is present in over 70% of collaborative interactions, with distributional patterns varying systematically across project scales and discussion contexts. Furthermore, survival analysis on 1,245 newcomer contributors shows that NEM recipients achieve a 12.6% improvement in PR merge rates and a 35% extension in average active tenure compared to the control group. Conclusion. These findings demonstrate that NEM constitutes a prevalent and impactful collaborative behavior in open source communities, with significant implications for newcomer integration and community sustainability. Schloss Dagstuhl – Leibniz-Zentrum für Informatik Xing Cui and Jingzheng Wu and Tianyue Luo and Xiang Ling 2026 Is Part Of LIPIcs, Volume 394, 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026) InProceedings Text doc-type:ResearchArticle publishedVersion application/pdf doi:10.4230/LIPIcs.ESEM.2026.43 urn:nbn:de:0030-drops-280119 https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.ESEM.2026.43 eng https://creativecommons.org/licenses/by/4.0/legalcode