When the LiteLLM supply-chain attack broke, the initial narrative pointed at poisoned PyPI packages as the intrusion vector. New analysis has flipped that story on its head: the root cause was a compromised Trivy GitHub Action, and the data is stark — over 95% of the roughly 2,500 affected organizations were exposed before the malicious LiteLLM packages were ever published to PyPI. LiteLLM was the downstream symptom. Trivy was the delivery mechanism.
This distinction matters enormously for defenders. If your incident response scope was limited to "did we install the bad litellm version," you likely missed the actual breach. Organizations running the compromised Trivy action in their CI/CD pipelines had their build environments — and the secrets inside them — exposed at workflow execution time, independent of whether LiteLLM was ever a dependency.
If Trivy runs anywhere in your GitHub Actions estate, you are in scope for this review. Full stop.
Technical Analysis: Anatomy of the Attack Chain
What is Trivy and why is it a high-value target?
Trivy is Aqua Security's widely adopted open-source vulnerability and misconfiguration scanner. Its GitHub Action (aquasecurity/trivy-action) is embedded in tens of thousands of CI/CD pipelines to scan container images, filesystems, and repositories during builds. That placement is exactly what makes it a prized supply-chain target:
- It executes inside the build runner, with access to the runner's filesystem, environment variables, and network.
- CI runners routinely hold high-value secrets: cloud provider credentials (AWS/GCP/Azure), registry tokens, PyPI/npm publish tokens, SSH keys, and
GITHUB_TOKENwith repository write scope. - Security tooling is often implicitly trusted and excluded from egress filtering and EDR scrutiny — who suspects the vulnerability scanner?
The attack chain
- Action compromise: Attackers gained the ability to modify the Trivy GitHub Action — poisoning release tags/versions that downstream workflows referenced. Because many consumers pin actions by mutable tag (e.g.,
@masteror@0.x) rather than immutable commit SHA, the malicious code propagated silently into pipelines with no change visible in the victim's own repository. - In-pipeline execution: When victim workflows ran, the compromised action executed attacker-controlled code inside the runner. The observable payload behavior in this class of attack is consistent: enumerate environment variables, read credential files (
~/.aws/credentials,~/.pypirc,~/.npmrc, Docker config, kubeconfig), and exfiltrate them — typically via outbound HTTPS to attacker infrastructure or dead-drop services. - Downstream poisoning: Harvested credentials — including PyPI publishing tokens — were then used to push the malicious LiteLLM packages. This is the critical timeline insight from the reporting: the LiteLLM packages were built with keys stolen via Trivy. Exposure happened at the Trivy execution step, days or weeks before the PyPI artifacts appeared.
Who is affected
- Any organization whose GitHub workflows invoked the compromised Trivy action versions/tags during the exposure window — regardless of whether LiteLLM is in their stack.
- Organizations that installed the poisoned LiteLLM releases from PyPI (a second, overlapping victim set).
- The ~2,500 compromised organizations span any sector using Python-heavy CI/CD; the blast radius tracks Trivy's popularity, not any one industry.
Exploitation status
This is confirmed, in-the-wild, mass exploitation — not theoretical. Approximately 2,500 organizations are identified as compromised. This is a supply-chain incident with credential theft as the primary objective, which means the exploitation window extends until every stolen secret is rotated. Treat this as an active incident if Trivy executed in your environment during the affected period.
Detection & Response
The highest-fidelity detection surface is the CI runner itself: the GitHub Actions runner worker process executing unexpected child processes, reading credential material, or making unusual outbound connections. If you self-host runners, you have full EDR telemetry. If you use GitHub-hosted runners, your telemetry is workflow logs and artifact review — audit those aggressively.
---
title: GitHub Actions Runner Spawning Shell with Download-and-Execute Pattern
id: 3f8c1a94-7b2e-4d51-9a06-5c8e2f7b1d34
status: experimental
description: Detects GitHub Actions runner worker processes spawning shells that fetch and execute remote content, consistent with the Trivy supply-chain payload behavior of running attacker-controlled code inside CI runners.
references:
- https://www.securityweek.com/trivy-not-litellm-behind-the-2500-org-compromise/
- https://attack.mitre.org/techniques/T1195/002/
- https://attack.mitre.org/techniques/T1059/004/
author: Security Arsenal
date: 2026/04/06
tags:
- attack.execution
- attack.t1059.004
- attack.t1195.002
logsource:
category: process_creation
product: linux
detection:
selection_parent:
ParentCommandLine|contains:
- 'Runner.Worker'
- 'run.sh'
- '/actions-runner/'
selection_child:
CommandLine|contains:
- 'curl '
- 'wget '
selection_pipe:
CommandLine|contains:
- '| bash'
- '| sh'
- '|bash'
- '|sh'
condition: selection_parent and selection_child and selection_pipe
falsepositives:
- Legitimate setup actions that bootstrap tooling via install scripts (review against workflow YAML)
level: high
---
title: CI Runner Process Accessing Cloud or Package Registry Credential Files
id: 8d2e5b17-4c6a-4f83-b1d9-7a3c9e5f2b68
status: experimental
description: Detects processes within a GitHub Actions runner context accessing AWS credentials, PyPI, npm, or Docker registry credential stores — behavior consistent with secret harvesting during the Trivy action compromise.
references:
- https://www.securityweek.com/trivy-not-litellm-behind-the-2500-org-compromise/
- https://attack.mitre.org/techniques/T1552/001/
- https://attack.mitre.org/techniques/T1078/004/
author: Security Arsenal
date: 2026/04/06
tags:
- attack.credential_access
- attack.t1552.001
logsource:
category: process_creation
product: linux
detection:
selection:
CommandLine|contains:
- '.aws/credentials'
- '.pypirc'
- '.npmrc'
- '.docker/config.json'
- '.kube/config'
- 'GH_TOKEN'
- 'GITHUB_TOKEN'
- 'AWS_SECRET_ACCESS_KEY'
filter_known_steps:
CommandLine|contains:
- 'aws s3'
- 'aws ecr'
- 'docker login'
- 'npm publish'
- 'twine upload'
condition: selection and not filter_known_steps
falsepositives:
- Build steps that legitimately authenticate to registries; tune the filter list to your pipeline's known publish steps
level: high
---
title: Python Package Install of LiteLLM Outside Requirements Lockfile
id: 5b1f7c33-9d4e-4a26-8e72-2f6b8d3c1a95
status: experimental
description: Detects direct pip installs of litellm with an explicit version or from an index at runtime rather than from a pinned lockfile, a pattern seen when poisoned packages enter build environments dynamically.
references:
- https://www.securityweek.com/trivy-not-litellm-behind-the-2500-org-compromise/
- https://attack.mitre.org/techniques/T1195/001/
author: Security Arsenal
date: 2026/04/06
tags:
- attack.execution
- attack.t1195.001
logsource:
category: process_creation
product: linux
detection:
selection:
CommandLine|contains: 'litellm'
CommandLine|contains:
- 'pip install'
- 'pip3 install'
- 'uv pip install'
condition: selection
falsepositives:
- Legitimate developer and build installs of litellm; investigate version and source index in every hit
level: medium
The following KQL hunts CI runner behavior in Microsoft Sentinel/Defender. If you ingest self-hosted runner EDR telemetry into Defender, DeviceProcessEvents gives you the cleanest view; Syslog ingestion covers auditd/execve logging on Linux runners.
// Hunt: Suspicious child processes and credential access under GitHub Actions runners (last 30 days)
let Lookback = 30d;
DeviceProcessEvents
| where TimeGenerated > ago(Lookback)
| where InitiatingProcessCommandLine has_any ("Runner.Worker", "actions-runner", "run.sh")
or ProcessCommandLine has_any ("Runner.Worker", "actions-runner")
| where ProcessCommandLine has_any (".aws/credentials", ".pypirc", ".npmrc", "config.json", "GITHUB_TOKEN", "AWS_SECRET_ACCESS_KEY")
or (ProcessCommandLine has_any ("curl", "wget") and ProcessCommandLine has_any ("| bash", "| sh", "|bash"))
or ProcessCommandLine has "env" and ProcessCommandLine has "|"
| project TimeGenerated, DeviceName, InitiatingProcessCommandLine, FileName, ProcessCommandLine, AccountName, ReportId
| order by TimeGenerated desc;
// Hunt: Outbound connections from runner processes to non-standard destinations
// Baseline the KnownInfra list to your org's normal CI egress before operationalizing
DeviceNetworkEvents
| where TimeGenerated > ago(Lookback)
| where InitiatingProcessCommandLine has_any ("Runner.Worker", "actions-runner")
| where RemoteUrl !has_any ("github.com", "githubusercontent.com", "githubassets.com", "actions.githubusercontent.com", "pypi.org", "files.pythonhosted.org", "azure.com", "amazonaws.com")
| summarize ConnectionCount = count(), FirstSeen = min(TimeGenerated), LastSeen = max(TimeGenerated) by DeviceName, RemoteUrl, RemoteIP, InitiatingProcessFileName
| order by FirstSeen asc
For DFIR teams running Velociraptor across self-hosted runner fleets, this artifact hunts runner-context processes touching credential stores and identifies credential files modified during the exposure window — the latter helps you scope which secrets to prioritize for rotation.
-- Trivy Supply-Chain IR: Runner process audit and credential store access
-- Deploy across self-hosted GitHub Actions runners to scope secret exposure
-- Part 1: Running processes under runner context with suspicious command lines
SELECT Pid, Ppid, Name, Exe, CommandLine, Username, CreateTime
FROM pslist()
WHERE CommandLine =~ '(?i)(\.aws/credentials|\.pypirc|\.npmrc|GITHUB_TOKEN|AWS_SECRET|curl.*\|.*(ba)?sh|wget.*\|.*(ba)?sh)'
OR Exe =~ '(?i)actions-runner'
-- Part 2: Credential stores modified within the exposure window (adjust dates to your confirmed window)
SELECT FullPath, Size, Mtime, Atime, Ctime
FROM glob(globs=[
'/home/*/.aws/credentials',
'/home/*/.pypirc',
'/home/*/.npmrc',
'/home/*/.docker/config.json',
'/root/.aws/credentials',
'/home/*/.kube/config'
])
WHERE Mtime > '2026-01-01'
ORDER BY Mtime DESC
Remediation
1. Audit and purge compromised action references immediately
Every workflow referencing aquasecurity/trivy-action by mutable tag must be reviewed and repinned to a known-good, immutable commit SHA. Consult Aqua's official advisory and the Trivy GitHub repository security notices for the confirmed-clean versions and compromised tag list before repinning — do not assume the current tag head is safe.
The following Bash script audits a GitHub organization for Trivy action usage, flags unpinned (tag-based) references, and checks local/Python environments for litellm installs that need version verification against the known-malicious release list.
#!/bin/bash
# Trivy/LiteLLM Supply-Chain Triage — Security Arsenal
# Requires: gh CLI authenticated with org read scope
set -euo pipefail
ORG="YOUR_ORG_HERE"
REPORT="trivy_supplychain_audit_$(date +%Y%m%d).txt"
echo "=== Trivy Action Usage Audit: $ORG ===" | tee "$REPORT"
# 1. Find all workflow files referencing trivy-action across the org
gh search code "aquasecurity/trivy-action" --owner "$ORG" \
--filename ".yml" --limit 200 \
--json repository,path --jq '.[] | "\(.repository.nameWithOwner): \(.path)"' \
| tee -a "$REPORT"
echo "" | tee -a "$REPORT"
echo "=== Checking for mutable (non-SHA) pins ===" | tee -a "$REPORT"
# 2. Flag any reference pinned by tag instead of 40-char commit SHA
gh api "orgs/$ORG/repos" --paginate --jq '.[].full_name' | while read -r repo; do
gh api "repos/$repo/git/trees/HEAD?recursive=1" \
--jq '.tree[] | select(.path | test(".github/workflows/.*\\.ya?ml$")) | .path' 2>/dev/null \
| while read -r wf; do
content=$(gh api "repos/$repo/contents/$wf" --jq '.content' 2>/dev/null | base64 -d 2>/dev/null || true)
echo "$content" | grep -nE 'trivy-action@(v?[0-9]|master|main)' \
&& echo " ^^ MUTABLE PIN: $repo/$wf" | tee -a "$REPORT"
done
done
echo "" | tee -a "$REPORT"
echo "=== Local litellm version check ===" | tee -a "$REPORT"
# 3. Check installed litellm against PyPI release history — verify against the
# vendor/security-advisory list of known-malicious versions before trusting output
python3 -m pip show litellm 2>/dev/null | tee -a "$REPORT" || echo "litellm not installed locally" | tee -a "$REPORT"
grep -rn "litellm" --include="requirements*.txt" --include="pyproject.toml" --include="poetry.lock" --include="uv.lock" . 2>/dev/null | tee -a "$REPORT" || true
echo "Audit complete. Cross-reference findings with the Aqua Security advisory before rotating or reverting."
2. Rotate every secret that touched an affected pipeline — assume compromise
Because the exposure mechanism was credential theft at build time, patching the action is only half the job. For any workflow that ran the compromised action:
- Rotate all CI secrets: cloud access keys, registry tokens, PyPI/npm tokens, signing keys, deploy keys, database credentials.
- Rotate
GITHUB_TOKEN-adjacent material: review PATs and GitHub App installation tokens usable from workflows; revoke any with unexplained API activity. - Invalidate cloud sessions, not just keys — check CloudTrail/Azure Activity Log for use of the stolen credentials from unfamiliar ASNs/geographies during and after the exposure window.
- Audit PyPI/npm/GitHub release history for packages or releases you didn't publish — this is exactly how LiteLLM got poisoned, and your org's packages may be next on the attacker's list.
3. Review workflow run logs and artifacts for the exposure window
Pull GitHub Actions run logs (via the API: GET /repos/{owner}/{repo}/actions/runs/{run_id}/logs) for every execution of the suspect action. Look for anomalous steps, unexpected outbound connections, base64 blobs, and steps not present in your YAML — injected steps are the smoking gun.
4. Structural hardening so the next action compromise doesn't become yours
- Pin every third-party action to a full commit SHA (GitHub's own security guidance), and enforce it with a linting step or policy-as-code (e.g., StepSecurity/OpenSSF tooling) that fails builds on tag-pinned actions.
- Set the repository/org Actions policy to "Allow only actions created by GitHub or verified marketplace actions" where feasible, with an explicit allowlist for scanner tooling.
- Use OpenID Connect (OIDC) federation for cloud access instead of long-lived stored keys, so stolen runner secrets expire useless.
- Scope
GITHUB_TOKENto least privilege (permissions:block per-job) — a poisoned scanner step should never inherit write access to your releases. - Apply egress filtering on self-hosted runners: allowlist GitHub, package registries, and your artifact stores; alert on everything else. The exfiltration step of this attack dies at the firewall.
- Adopt Sigstore/cosign verification for critical actions and container tooling, and consume scanner images by digest.
5. Vendor and community references
- Aqua Security / Trivy GitHub repository — security advisories and the canonical list of affected tags/versions:
https://github.com/aquasecurity/trivy-action(Security tab) andhttps://github.com/aquasecurity/trivy - LiteLLM repository and PyPI project page — for the known-malicious release versions to hunt in your dependency trees:
https://pypi.org/project/litellm/#history - OpenSSF / GitHub Actions hardening guidance —
https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions
The Defensive Lesson
The 95% statistic is the story. Organizations that scoped their response to "the malicious LiteLLM package" addressed the tail end of the kill chain and left the initial-access vector — a trusted security tool executing with secrets in CI — wide open. Supply-chain IR must trace upstream to the root compromise, not downstream to the most visible artifact. Treat your CI pipeline as production infrastructure: inventory every action, pin everything immutably, minimize what secrets can exist in a runner, and monitor the runner like you'd monitor a domain controller — because to an attacker, it is one.
Related Resources
Security Arsenal Penetration Testing Services AlertMonitor Platform Book a SOC Assessment vulnerability-management Intel Hub
Is your security operations ready?
Get a free SOC assessment or see how AlertMonitor cuts through alert noise with automated triage.