Vulnerability Detection Across Different CI/CD Platforms

CI/CD consists of two core practices, with the first practice being Continuous Integration or CI, which consists of frequent integrations along with automated testing of the code pushed by developers to a shared repository. It ensures that the committed code changes do not include any bugs, guaranteeing the stability and functionality of the codebase. This process includes various elements such as:

• $\mathrm{■}$

  Automated Build Process: Automating the application’s build process, ensuring that the latest code changes are always in a deployable state.

• $\mathrm{■}$

  Instant Feedback: Developers get prompt feedback on their code changes, enabling them to quickly address any issues.

• $\mathrm{■}$

  Enhanced Code Quality: CI/CD pipelines often include static code analysis and security scanning to identify vulnerabilities.

• $\mathrm{■}$

  Performance Monitoring: Performance monitoring is implemented to ensure that recent code changes do not negatively impact the application’s performance.

The second practice can be approached in two distinct ways, depending on whether the process is partially or fully automated, respectively:

• $\mathrm{■}$

  Continuous Delivery: involves ensuring that the application is always in a deployable state after passing the integration stage. The final deployment to production requires manual approval.

• $\mathrm{■}$

  Continuous Deployment: builds upon this by fully automating the process. Code changes that pass automated tests are deployed directly to production without human intervention. This approach enables faster and more frequent delivery of updates, ensuring consistent and reliable deployments.

These two practices together form a CI/CD pipeline, accelerating software delivery by automating and optimizing the processes of integrating, testing, and deploying code [17].³³3Besides this section, whenever the acronym CD is mentioned, it is referring to Continuous Deployment and not Continuous Delivery, unless it is stated otherwise.

Thus, CI/CD tools allow for these two practices. This automation minimizes human intervention, reduces errors, and ensures faster delivery of high-quality software. CI/CD tools, such as GitHub Actions [4] or Jenkins [7], provide the infrastructure necessary to define and execute this kind of pipelines, allowing for the automation of these workflows [16, 17]. A CI/CD pipeline, or CI/CD workflow, is a sequence of automated steps designed to streamline the process of delivering new software versions [5].

Attackers with access to CI/CD components (SCM⁵⁵5tools and systems used to track and manage changes to source code, such as GitHub, GitLab and Bitbucket., CI, artifact repository⁶⁶6An artifact repository is a system for storing, managing, and retrieving build artifacts (like compiled code, libraries, packages or container images), which are typically generated during the CI/CD pipeline process., etc.) can push malicious code or artifacts due to weak enforcement of approvals and reviews. CI/CD pipelines prioritize speed, often relying on automation with minimal human intervention.

Without proper flow control, an attacker could:

• $\mathrm{■}$

  Push malicious code that gets automatically deployed.

• $\mathrm{■}$

  Exploit weak branch protection or auto-merge rules.

• $\mathrm{■}$

  Upload tampered artifacts that are later deployed.

• $\mathrm{■}$

  Directly modify production code or infrastructure without verification.

To mitigate these vulnerabilities, the defenders should adopt the following strategies:

• $\mathrm{■}$

  Implement branch protection rules and restrict exclusions.

• $\mathrm{■}$

  Limit auto-merge usage and ensure strict validation.

• $\mathrm{■}$

  Require additional approval before triggering production deployments.

• $\mathrm{■}$

  Prefer allowing artifacts that were created by a pre-approved CI service account to flow through the pipeline.

• $\mathrm{■}$

  Detect and correct drifts between production code and CI/CD sources.

PPE is an attack where an adversary with access to a source control system, but not the build environment, modifies CI/CD pipeline configurations to execute malicious code during the build process.

PPE exploits permissions in a source control repository to inject malicious commands into a CI/CD pipeline. Pipelines that execute unreviewed code, from pull requests or commits to arbitrary branches, are particularly vulnerable. Once executed, the attack runs within the pipeline’s context, potentially leading to severe security breaches. There are three types of PPE:

• $\mathrm{■}$

  Direct PPE (D-PPE): The attacker modifies the CI configuration file, directly in the repository, by doing a direct push to an unprotected branch or through a pull request. Once the pipeline is triggered, the malicious commands run in the build node.

• $\mathrm{■}$

  Indirect PPE (I-PPE): Occurs when direct modification of the CI configuration file is restricted. The attacker injects malicious code into referenced files (like Makefiles, scripts, test cases, or tool configurations) that the pipeline executes, and when the pipeline runs, these files execute their malicious instructions.

• $\mathrm{■}$

  Public PPE (3PE): Targets public repositories where anyone can submit pull requests. If the pipeline executes unreviewed contributions, an anonymous attacker can inject malicious code. This can expose internal assets, especially if public and private pipelines share the same CI environment.

Once the malicious code is executed, it runs within the pipeline’s context and can carry out a variety of harmful operations, such as compromising the build environment, stealing sensitive data, or deploying further malicious code into production. To mitigate PPE risks, the next strategies should be adopted:

• $\mathrm{■}$

  Restrict write access to CI/CD configuration files.

• $\mathrm{■}$

  Enforce rigorous code reviews for changes that affect pipeline configurations.

• $\mathrm{■}$

  Separate CI configuration from source code when possible.

• $\mathrm{■}$

  Limit the exposure of files that can be referenced by the pipeline to trusted sources only.

This risk associated with granting third-party services access to an organization’s CI/CD systems without proper governance. It expands the attack surface due to the ease of providing third-party services access to sensitive resources.

Organizations often integrate third-party services into their CI/CD systems to increase functionality. However, the methods for connecting these services (like OAuth, SSH keys, or access tokens) are simple to implement, often without requiring additional permissions or approvals. This can result in third parties having extensive access, from reading code in a single repository to full administrative control. These services are also easily integrated into build pipelines, giving them access to resources that could expose the system to security risks.

Lack of governance and visibility can prevent organizations from maintaining proper Role-based access control and least privilege⁷⁷7Least Privilege is a security concept that restricts users, applications, and systems to the minimum level of access necessary to perform their functions.. A single compromised third party could be used by attackers to manipulate code or trigger malicious actions in build systems, potentially affecting broader systems within the organization. To intercept those ungoverned third-party services, the following strategies should be adopted:

• $\mathrm{■}$

  Implement vetting procedures for third-party services before granting access to the environment, ensuring the principle of least privilege.

• $\mathrm{■}$

  Maintain visibility over third-party access, covering integration methods, granted permissions, and actual usage.

• $\mathrm{■}$

  Limit third-party access to only necessary resources and regularly review access controls.

• $\mathrm{■}$

  Periodically remove unused or unnecessary third-party services from the environment.

CI/CD workflows can be defined in various formats, depending on the CI/CD tool in use, including YAML, XML, and JSON, with Jenkins being the only exception, using a unique Groovy format.

All of these formats can easily be converted into a dictionary object. By converting each workflow to a dictionary object, before scanning, we ensure consistent parsing across different tools.

Several libraries already support parsing YAML, XML and JSON files to dictionary objects (e.g., Python’s yaml, json, and xmltodict libraries). While there is no third-party solution to convert Groovy based Jenkinfiles into dictionaries, it is still achievable using a simple custom parsing function.

It is also possible to detect automatically which CI/CD tool a specific workflow belongs to by scanning for certain patterns that are characteristic of that specific tool inside the given workflow.

For example, a workflow containing patterns like “jobs:”, “steps:”, “uses: actions/”, “github.”, and “runs-on” likely indicates a GitHub Actions pipeline.

By collecting multiple pattern sets for each tool and evaluating how many of them match a given workflow, we can calculate a confidence score and make an accurate tool prediction.

A workflow can be described as a dictionary, so instead of just using the simple YARA-X features for pattern matching, we can use the YARA-X’s Dictionary Module (see Section 3.2) to scan vulnerabilities more accurately.

This is made by using the YARA-X Python API to compile rules and scan the workflows. By converting a rule file into a string and passing it to the YARA-X compile() method, we create a Rules object that contains those rules. This object provides a scan() method, which we use to scan the provided workflow by first converting the already formatted dictionary into a JSON string and encoding it in utf-8.

Listing 3 shows an example of how to use the Dictionary Module.

By using the Dictionary module, we can guarantee that there will be a Pull Request Target that is trying to run a command using a checkout action [10], all while verifying if the action checkout is being made before running the command.

Since different CI/CD tools use different rules and syntaxes, we generally can not use detection rules made for a tool to another, however it is the only thing that needs to be different, meaning that we can just make a detection rule file using YARA-X rules and the Dictionary module for each CI/CD tool, easily creating new rulesets for new CI/CD tools and adding rules to existing ones.

Plugins are typically packaged as archive files (e.g. .zip, .hpi), so, to scan these plugins, we must first extract their contents to access the source code, however, this task can take a lot of time.

A good practice to circumvent this problem is to store the results of the already scanned plugins in a database, ensuring that a plugin only needs to be decompiled once.

Given that plugins can suffer changes over time due to updates or even tampering, we must store not just the plugins, but their versions (in case of an update) and hashes (in case of tampering) as well.

Plugins will also be scanned using YARA-X, though not with the Dictionary module, as plugin code does not follow a unified structure like CI/CD workflows.

With YARA-X we can make rules with additional information via metadata, which is going to be used to get the rule’s description, author and version, as well as capture patterns within a file. With the YARA-X pattern detection, the pattern that was detected gives the match content, offset and length, and with the offset we can easily get in which line the pattern was detected as well.

This is also made by using the YARA-X Python API to compile rules and scan plugin files.

The API provides the following endpoints:

POST /scan

– Scans a given CI/CD configuration file for vulnerabilities.

• $\mathrm{■}$

  Parameters:

  • –

    file – The configuration file to be scanned (uploaded via form-data).

  • –

    rule_type – (Optional) The name of the CI/CD platform the configuration belongs to (e.g., github_actions, jenkins, circleCI).

  • –

    format – (Optional) The format that the results will be displayed on. Supported values are sarif (default) or json.

  • –

    automatic – (Optional) Boolean flag to enable automatic detection of the CI/CD platform. When set to true, the rule_type parameter is not required.

• $\mathrm{■}$

  Returns: The scan results in SARIF or JSON format. Listing 4 shows an example of a result in SARIF.

POST /add_rule

– Adds a new YARA-X rule to an existing ruleset.

• $\mathrm{■}$

  Parameters:

  • –

    file – A .txt, .yar, or .yara file containing the new rule (uploaded via form-data).

  • –

    rule_type – The name of the existing ruleset to which the rule should be added (e.g., github_actions, jenkins, circleCI).

• $\mathrm{■}$

  Returns: A message confirming the successful addition of the rule, or an error if the rule is invalid or already exists, or the rule_set does not exist.

POST /add_ruleset

– Creates a new ruleset for a CI/CD platform.

• $\mathrm{■}$

  Parameters:

  • –

    file – A .txt, .yar, or .yara file containing a complete ruleset (uploaded via form-data).

  • –

    rule_type – The name for the new ruleset.

• $\mathrm{■}$

  Returns: A message confirming the creation of the ruleset or an error if the file is invalid or a ruleset with that name already exists.

The API provides the following endpoints:

POST /plugin_file

– Scans a single plugin file for potential risks and vulnerabilities.

• $\mathrm{■}$

  Parameters:

  • –

    file – The plugin file to be scanned (e.g., a .hpi file), uploaded via form-data.

  • –

    cfr – (Optional) The decompiler to use during analysis. Supported values are cfr (default) or jadx.

  • –

    format – (Optional) The format that the results will be displayed on. Supported values are sarif (default) or json.

• $\mathrm{■}$

  Returns: A JSON or SARIF file with the detected plugins and plugin files matches, with the YARA rules and their metadata. If no issues are found, a message indicating the absence of matches is returned. Listing 5 shows an example of a result in JSON.

POST /plugin_list

– Scans a list of plugins provided in a single file for batch analysis.

• $\mathrm{■}$

  Parameters:

  • –

    file – A JSON file containing multiple plugins to be scanned.

  • –

    cfr – (Optional) The decompiler to use during analysis. Supported values are cfr (default) or jadx.

• $\mathrm{■}$

  Returns: A JSON or SARIF file containing scan results for each plugin. Each entry includes rule matches or a message indicating that no matches were found.

POST /plugin_url

– Downloads and scans a plugin from a given URL.

• $\mathrm{■}$

  Parameters:

  • –

    url – The URL from which to download the plugin.

  • –

    cfr – (Optional) The decompiler to use during analysis. Supported values are cfr (default) or jadx.

• $\mathrm{■}$

  Returns: A JSON or SARIF file with the YARA rule matches if any are detected. Otherwise, a message stating that no matches were found is returned.