How to Protect Against the Trivy Supply Chain Attack: Defending CI/CD Pipelines from Infostealer Malware

Introduction

In a concerning development for the security community, the widely-used Trivy vulnerability scanner was compromised in a supply-chain attack. The attackers, a group known as TeamPCP, managed to distribute credential-stealing malicious software through official releases and GitHub Actions. This breach underscores the growing risk of supply chain attacks and highlights how even security tools can be weaponized against organizations.

For defenders, this incident is particularly significant because it demonstrates how attackers can compromise the tools we rely on for security, turning them into delivery mechanisms for malware. When a security tool like Trivy is compromised, it creates a trusted pathway into organizations that might otherwise have robust security postures.

Technical Analysis

Trivy, developed by Aqua Security, is a popular open-source vulnerability scanner that organizations use to find security issues in containers, Kubernetes, file systems, Git repositories, and other assets. In this supply chain attack, threat actors gained unauthorized access to Trivy's infrastructure and manipulated its distribution channels.

Attack Details

The attackers achieved the following:

• Compromised Trivy's official GitHub repository
• Modified the GitHub Actions workflow to execute malicious code
• Distributed credential-stealing malware through official releases

The malicious code was designed to steal credentials from victims' systems, potentially giving attackers access to sensitive data and systems within compromised organizations.

Affected Systems

Organizations that downloaded or used Trivy during the compromise window are at risk. This includes:

• Systems running vulnerable versions of Trivy
• Environments that executed affected GitHub Actions workflows
• CI/CD pipelines that integrated with the compromised Trivy repository

Severity

This incident has been classified as high severity due to:

• The widespread use of Trivy across industries
• The trusted nature of the tool in many security pipelines
• The potential for credential theft leading to further compromises

Fix Details

Aqua Security has addressed the compromise by:

• Revoking compromised credentials
• Auditing and securing all infrastructure
• Releasing clean versions of Trivy
• Implementing additional security controls

Defensive Monitoring

Security teams should actively monitor for signs of compromise related to this attack. The following detection queries and scripts can help identify potential indicators of compromise (IoCs).

Microsoft Sentinel KQL Queries

The following KQL query can help detect suspicious activity related to Trivy or similar supply chain attacks:

// Detect suspicious process execution related to Trivy compromise
DeviceProcessEvents
| where Timestamp > ago(7d)
| where ProcessName has_any ("trivy", "unknown") 
| where InitiatingProcessAccountName != "System"
| where ProcessCommandLine contains_any ("git", "curl", "wget", "powershell", "bash")
| project Timestamp, DeviceName, AccountName, ProcessName, ProcessCommandLine, InitiatingProcessAccountName
| order by Timestamp desc


To monitor for potential credential theft behavior:

// Detect credential theft patterns
DeviceProcessEvents
| where Timestamp > ago(7d)
| where ProcessName in~ ("cmd.exe", "powershell.exe", "bash", "sh")
| where ProcessCommandLine contains_any ("token", "credential", "password", "secret", "key")
| project Timestamp, DeviceName, AccountName, ProcessName, ProcessCommandLine
| order by Timestamp desc

PowerShell Script

This script can help check if potentially vulnerable versions of Trivy are installed:

# Check for potentially vulnerable Trivy installations
$trivyPaths = @(
    "$env:USERPROFILE\.local\bin\trivy",
    "$env:ProgramFiles\trivy\trivy.exe",
    "$env:ChocolateyInstall\lib\trivy\tools\trivy.exe"
)

foreach ($path in $trivyPaths) {
    if (Test-Path $path) {
        $version = & $path --version 2>&1
        Write-Host "Trivy found at $path"
        Write-Host "Version: $version"
        
        # Check file properties for modification date
        $fileInfo = Get-Item $path
        Write-Host "Last Modified: $($fileInfo.LastWriteTime)"
        
        # Additional checks can be added here
        Write-Host "---"
    }
}

Bash Script

For Linux environments, this script helps identify Trivy installations:

#!/bin/bash

# Check for Trivy installation
echo "Checking for Trivy installation..."

# Common installation paths
trivy_paths=(
    "/usr/local/bin/trivy"
    "$HOME/.local/bin/trivy"
    "/usr/bin/trivy"
)

for path in "${trivy_paths[@]}"; do
    if [ -f "$path" ]; then
        echo "Trivy found at: $path"
        version=$(trivy --version 2>&1)
        echo "Version: $version"
        
        # Check file properties
        ls -l "$path"
        echo "---"
    fi
done

# Check for running Trivy processes
echo "Checking for running Trivy processes..."
pgrep -a trivy

GitHub Actions Monitoring

For organizations using GitHub Actions, monitor workflow runs for suspicious activity:

# Example workflow to check for suspicious GitHub Actions activity
name: Security Check

on:
  schedule:
    - cron: '0 */6 * * *'  # Run every 6 hours
  workflow_dispatch:

jobs:
  check-workflows:
    runs-on: ubuntu-latest
    steps:
      - name: Check for modified workflows
        run: |
          # Check for recently modified workflow files
          echo "Checking for modified workflow files..."
          find .github/workflows -name "*.yml" -o -name "*.yaml" | xargs ls -lt | head -20

Remediation

To protect against this and similar supply chain attacks, organizations should take the following steps:

Immediate Actions

1. Update Trivy: Ensure you're using the latest, verified version of Trivy from the official Aqua Security repository.

# Update to the latest version
brew upgrade trivy  # macOS
sudo apt-get update && sudo apt-get install trivy  # Ubuntu/Debian
sudo yum update trivy  # RHEL/CentOS


2. **Verify Integrity**: Verify the integrity of downloaded files using checksums provided by the vendor.

# Example verification process
wget https://github.com/aquasecurity/trivy/releases/download/vX.Y.Z/trivy_X.Y.Z_Linux-64bit.tar.gz
wget https://github.com/aquasecurity/trivy/releases/download/vX.Y.Z/trivy_X.Y.Z_Linux-64bit.tar.gz.sha256
sha256sum -c trivy_X.Y.Z_Linux-64bit.tar.gz.sha256


3. **Rotate Credentials**: If you used potentially compromised versions of Trivy, rotate all credentials that may have been exposed, including GitHub tokens, API keys, and other secrets.

4. **Audit Logs**: Review GitHub Actions logs, authentication logs, and other relevant logs for signs of suspicious activity.

Long-term Security Measures

1. Implement Software Bill of Materials (SBOM): Create and maintain SBOMs for all software components to improve visibility and enable faster incident response.

2. Adopt Supply Chain Security Best Practices:

   • Require code signing for all releases
   • Implement multi-factor authentication for repository access
   • Use branch protection rules and require code review
   • Implement least privilege access controls

3. Monitor Third-Party Components: Implement continuous monitoring of third-party dependencies for vulnerabilities and security incidents.

4. Secure CI/CD Pipelines:

   • Use pinned dependency versions
   • Implement security scanning at multiple stages
   • Review and audit all GitHub Actions workflows regularly
   • Use environment-specific secrets with limited scope

5. Implement Zero Trust Principles: Verify every request, regardless of origin, and enforce least privilege access throughout your environment.

6. Enhance Threat Detection: Deploy security solutions that can detect anomalous behavior related to supply chain attacks, including unexpected credential access and unusual process execution.

7. Conduct Regular Security Assessments: Perform periodic penetration testing and security assessments of your supply chain and CI/CD infrastructure.

8. Establish an Incident Response Plan: Have a well-defined plan for responding to supply chain attacks, including communication procedures and recovery steps.

GitHub Actions Hardening

For organizations using GitHub Actions, implement these security measures:

# Example of a hardened workflow with security best practices
name: Secure Workflow

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

# Restrict permissions to minimum required
permissions:
  contents: read

jobs:
  security-scan:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          # Ensure fetching complete history for integrity checks
          fetch-depth: 0
      
      - name: Verify commit signatures
        run: |
          # Verify that commits are signed
          git log --show-signature
      
      - name: Run security scan
        env:
          # Never log secrets
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          # Use pinned versions of tools
          curl -sSfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin v0.50.0
          trivy fs --security-checks vuln,config .

Related Resources

Security Arsenal Incident Response Services AlertMonitor Platform Book a SOC Assessment incident-response Intel Hub