Document Open Access Logo

Mining GitHub Software Repositories to Look for Programming Language Cocktails

Authors João Loureiro, Alvaro Costa Neto , Maria João Varanda Pereira , Pedro Rangel Henriques

PDF HTML 5

Files

Thumbnail PDF
PDF
OASIcs.SLATE.2025.13.pdf
  • Filesize: 0.84 MB
  • 16 pages
HTML5 Logo HTML (experimental)
OASIcs.SLATE.2025.13.html

Document Identifiers

Subject Classification

ACM Subject Classification
  • Software and its engineering → Software libraries and repositories
Keywords
  • Software Repository Mining
  • Source Version Control
  • GitHub Scraping
  • Programming Cocktails

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

In light of specific development needs, it is common to concurrently apply different technologies to build complex applications. Given that lowering risks, costs, and other negative factors, while improving their positive counterparts is paramount to a better development environment, it becomes relevant to find out what technologies work best for each intended purpose in a project. In order to reach these findings, it is necessary to analyse and study the technologies applied in these projects and how they interconnect and relate to each other. The theory behind Programming Cocktails (meaning the set of programming technologies - Ingredients - that are used to develop complex systems) can support these analysis. However, due to the sheer amount of data that is required to construct and analyse these Cocktails, it becomes unsustainable to manually obtain them. From the desire to accelerate this process comes the need for a tool that automates the data collection and its conversion into an appropriate format for analysis. As such, the project proposed in this paper revolves around the development of a web-scraping application that can generate Cocktail Identity Cards (CIC) from source code repositories hosted on GitHub. Said CICs contain the Ingredients (programming languages, libraries and frameworks) used in the corresponding GitHub repository and follow the ontology previously established in a larger research project to model each Programming Cocktail. This paper presents a survey of current Source Version Control Systems (SVCSs) and web-scrapping technologies, an overview of Programming Cocktails and its current foundations, and the design of a tool that can automate the gathering of CICs from GitHub repositories.

Cite As Get BibTex

João Loureiro, Alvaro Costa Neto, Maria João Varanda Pereira, and Pedro Rangel Henriques. Mining GitHub Software Repositories to Look for Programming Language Cocktails. In 14th Symposium on Languages, Applications and Technologies (SLATE 2025). Open Access Series in Informatics (OASIcs), Volume 135, pp. 13:1-13:16, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.SLATE.2025.13

Author Details

João Loureiro
  • ALGORITMI Research Centre/LASI - DI, University of Minho, Braga, Portugal
Alvaro Costa Neto
  • ALGORITMI Research Centre/LASI - DI, University of Minho, Braga, Portugal
  • Research Centre in Digitalization and Intelligent Robotics (CeDRI), Laboratório Associado para a Sustentabilidade e Tecnologia em Regiões de Montanha (SusTEC), Instituto Politécnico de Bragança, Portugal
  • Instituto Federal de Educação, Ciência e Tecnologia de São Paulo, Barretos, Brazil
Maria João Varanda Pereira
  • Research Centre in Digitalization and Intelligent Robotics (CeDRI), Laboratório Associado para a Sustentabilidade e Tecnologia em Regiões de Montanha (SusTEC), Instituto Politécnico de Bragança, Portugal
Pedro Rangel Henriques
  • ALGORITMI Research Centre/LASI - DI, University of Minho, Braga, Portugal

Funding

This work has been supported by FCT – Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UID/00319/2023. The work of Maria João and Alvaro was supported by national funds: UID/05757 - Research Centre in Digitalization and Intelligent Robotics (CeDRI); and SusTEC, LA/P/0007/2020 (DOI: https://doi.org/10.54499/LA/P/0007/2020).

References

  1. Adem Ait, Javier Luis Cánovas Izquierdo, and Jordi Cabot. An empirical study on the survival rate of github projects. In Proceedings of the 19th International Conference on Mining Software Repositories, MSR '22, pages 365-375, New York, NY, USA, 2022. Association for Computing Machinery. URL: https://doi.org/10.1145/3524842.3527941.
  2. John Alhson. Parallel (concurrent) code development or how to develop software in a distributed workgroup or the concurrent versions system, cvs. Nuclear Physics B - Proceedings Supplements, 61(3):674-677, 1998. Proceedings of the Fifth International Conference on Advanced Technology and Particle Physics. URL: https://doi.org/10.1016/S0920-5632(97)00636-1.
  3. Natanael Arndt, Patrick Naumann, Norman Radtke, Michael Martin, and Edgard Marx. Decentralized collaborative knowledge management using git. Journal of Web Semantics, 54:29-47, 2019. Managing the Evolution and Preservation of the Data Web. URL: https://doi.org/10.1016/j.websem.2018.08.002.
  4. Sai Sri Nandan Challapalli, Prakarsh Kaushik, Shashikant Suman, Basu Dev Shivahare, Vimal Bibhu, and Amar Deep Gupta. Web development and performance comparison of web development technologies in node.js and python. In 2021 International Conference on Technological Advancements and Innovations (ICTAI), pages 303-307, 2021. URL: https://doi.org/10.1109/ICTAI53825.2021.9673464.
  5. Alvaro Costa Neto, Maria João Varanda Pereira, and Pedro Rangel Henriques. Application of programming cocktails identity cards to development complexity analysis. In Proceedings of the 23nd Belgium-Netherlands Software Evolution Workshop (BENEVOL 2024). CEUR-SW.org, 2024. To be published. Google Scholar
  6. Alvaro Costa Neto, Maria João Varanda Pereira, and Pedro Rangel Henriques. An ontology to understand programming cocktails. In Maria Ganzha, Leszek Maciaszek, Marcin Paprzycki, and Dominik Ślezak, editors, Proceedings of the 19th Conference on Computer Science and Intelligence Systems, Annals of Computer Science and Information Systems. IEEE, 2024. To be published. Google Scholar
  7. Ozren Dabic, Emad Aghajani, and Gabriele Bavota. Sampling projects in github for msr studies. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), pages 560-564, 2021. URL: https://doi.org/10.1109/MSR52588.2021.00074.
  8. N. Deepa, B. Prabadevi, L.B. Krithika, and B. Deepa. An analysis on version control systems. In 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE), pages 1-9, 2020. URL: https://doi.org/10.1109/ic-ETITE47903.2020.39.
  9. Yılmaz Dikilitaş, Çoşkun Çakal, Ahmet Can Okumuş, Halime Nur Yalçın, Emine Yıldırım, Ömer Faruk Ulusoy, Bilal Macit, Aslı Ece Kırkaya, Özkan Yalçın, Ekin Erdoğmuş, and Ahmet Sayar. Performance analysis for web scraping tools: Case studies on beautifulsoup, scrapy, htmlunit and jsoup. In Fausto Pedro García Márquez, Akhtar Jamil, Alaa Ali Hameed, and Isaac Segovia Ramírez, editors, Emerging Trends and Applications in Artificial Intelligence, pages 471-480, Cham, 2024. Springer Nature Switzerland. Google Scholar
  10. M El Asikri, S Knit, and H Chaib. Using web scraping in a knowledge environment to build ontologies using python and scrapy. European Journal of Molecular & Clinical Medicine, 7(03):2020, 2020. Google Scholar
  11. Hamzeh Eyal Salman. Ai-based clustering of similar issues in github’s repositories. Journal of Computer Languages, 78:101257, 2024. URL: https://doi.org/10.1016/j.cola.2023.101257.
  12. Alan L. Glasser. The evolution of a source code control system. SIGSOFT Softw. Eng. Notes, 3(5):122-125, January 1978. URL: https://doi.org/10.1145/953579.811111.
  13. Youngtaek Kim, Jaeyoung Kim, Hyeon Jeon, Young-Ho Kim, Hyunjoo Song, Bohyoung Kim, and Jinwook Seo. Githru: Visual analytics for understanding software development history through git metadata analysis. IEEE Transactions on Visualization and Computer Graphics, 27(2):656-666, 2021. URL: https://doi.org/10.1109/TVCG.2020.3030414.
  14. Ali Koc and Abdullah Uz Tansel. A survey of version control systems. ICEME 2011, 2011. Google Scholar
  15. Sandeep Kaur Kuttal, Se Yeon Kim, Carlos Martos, and Alexandra Bejarano. How end-user programmers forage in online repositories? an information foraging perspective. Journal of Computer Languages, 62:101010, 2021. URL: https://doi.org/10.1016/j.cola.2020.101010.
  16. Rana Majumdar, Rachna Jain, Shivam Barthwal, and Chetna Choudhary. Source code management using version control system. In 2017 6th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), pages 278-281, 2017. URL: https://doi.org/10.1109/ICRITO.2017.8342438.
  17. Edi Muškardin, Tamim Burgstaller, Martin Tappler, and Bernhard K. Aichernig. Active model learning of git version control system. In 2024 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 78-82, 2024. URL: https://doi.org/10.1109/ICSTW60967.2024.00024.
  18. Simona Vasilica Oprea and Adela Bâra. Why is more efficient to combine beautifulsoup and selenium in scraping for data under energy crisis. Ovidius University Annals, Economic Sciences Series, 22(2):146-152, 2022. Google Scholar
  19. Stack Overflow. Stack overflow developer survey 2022. https://survey.stackoverflow.co/2022/#version-control-version-control-system, 2022. Last Accessed: 12/01/2025.
  20. Sakshi Pant, Er. Narinder Yadav, Milan, Monnie Sharma, Yash Bedi, and Anshuman Raturi. Web scraping using beautiful soup. In 2024 International Conference on Knowledge Engineering and Communication Systems (ICKECS), volume 1, pages 1-6, 2024. URL: https://doi.org/10.1109/ICKECS61492.2024.10617017.
  21. Pranit Patil, Pramila Chawan, and Prithviraj Chauhan. Parsing of html document. International Journal of Advanced Research in Computer Engineering & Technology, 1:2278-1323, July 2012. Google Scholar
  22. Ruchitaa Raj N R, Nandhakumar Raj S, and Vijayalakshmi M. Web scrapping tools and techniques: A brief survey. In 2023 4th International Conference on Innovative Trends in Information Technology (ICITIIT), pages 1-4, 2023. URL: https://doi.org/10.1109/ICITIIT57246.2023.10068666.
  23. Martina Radilova, Patrik Kamencay, Robert Hudec, Miroslav Benco, and Roman Radil. Tool for parsing important data from web pages. Applied Sciences, 12(23), 2022. URL: https://doi.org/10.3390/app122312031.
  24. Vidhi Singrodia, Anirban Mitra, and Subrata Paul. A review on web scrapping and its applications. In 2019 International Conference on Computer Communication and Informatics (ICCCI), pages 1-6, 2019. URL: https://doi.org/10.1109/ICCCI.2019.8821809.
  25. Walter F. Tichy. Design, implementation, and evaluation of a revision control system. In Proceedings of the 6th International Conference on Software Engineering, ICSE '82, pages 58-67, Washington, DC, USA, 1982. IEEE Computer Society Press. URL: http://dl.acm.org/citation.cfm?id=807748.
  26. Federico Tomassetti and Marco Torchiano. An empirical assessment of polyglot-ism in github. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, EASE '14, New York, NY, USA, 2014. Association for Computing Machinery. URL: https://doi.org/10.1145/2601248.2601269.
  27. Wikipedia. Comparison of source-code-hosting facilities. https://en.wikipedia.org/wiki/Comparison_of_source-code-hosting_facilities#Popularity, 2024. Last Accessed: 05/10/2024.
  28. Nazatul Nurlisa Zolkifli, Amir Ngah, and Aziz Deraman. Version control system: A review. Procedia Computer Science, 135:408-415, 2018. The 3rd International Conference on Computer Science and Computational Intelligence (ICCSCI 2018) : Empowering Smart Technology in Digital Era for a Better Life. URL: https://doi.org/10.1016/j.procs.2018.08.191.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact