Securing LiteLLM Gateways: Detecting Traffic Hijacking and API Key Theft

The rapid adoption of Large Language Models (LLMs) has centralized trust in AI gateway solutions like LiteLLM. These gateways simplify governance and provide a unified interface, but they also create a single point of failure. By holding the backend API keys for providers like OpenAI, Anthropic, and Cohere, a compromised LiteLLM instance becomes a "crown jewels" target.

Recent research highlights a critical attack surface—"LLM Heist"—where adversaries hijack LiteLLM deployments to perform traffic interception, exfiltrate high-value API keys, and inject malicious tool calls. For defenders, this isn't just about data leakage; it's about preventing the unauthorized execution of tools and actions connected to your AI agents. This post outlines the mechanics of these attacks and provides actionable detection and remediation strategies for 2026.

Technical Analysis

LiteLLM acts as a proxy, standardizing API calls across multiple LLM providers. It stores sensitive credentials—often in environment variables or configuration files—to facilitate these backend connections.

The Attack Vector: The attack chain typically focuses on the compromise of the LiteLLM proxy instance or its configuration environment. Once initial access is gained, adversaries leverage the gateway's privileged position to:

Traffic Interception (Rerouting): Modifying the routing logic or DNS settings of the host to redirect LLM prompts and responses through an adversary-controlled server. This allows for the theft of proprietary prompts and sensitive user data.
Key Theft: Accessing the memory space or configuration files of the LiteLLM process to dump plaintext API keys for backend providers. These keys are then used for unauthorized API consumption or cryptomining.
Tool-Call Injection: LiteLLM often handles "function calling" or tool execution for agents. By hijacking the gateway, attackers can modify the payload to invoke unintended tools (e.g., executing shell commands, making unauthorized financial transactions) before the request reaches the LLM or after the response is returned.

Exploitation Status: Proof-of-concept (PoC) code demonstrating these interception and injection techniques is currently circulating within the red-team community. While no specific CVE is assigned to the core software in this context, the methodology relies on the abuse of valid gateway functionalities and insufficient isolation of the credential management system.

Detection & Response

Detecting a hijacked LiteLLM gateway requires monitoring for anomalies in process behavior, network connections, and configuration integrity.

Sigma Rules

The following Sigma rules detect suspicious process spawning by the LiteLLM proxy (indicating potential code execution) and anomalous network outbound connections.

YAML

---
title: LiteLLM Spawning Unauthorized Shell
id: 8a4b1c92-0f7e-4d3a-9b5c-1d2e3f4a5b6c
status: experimental
description: Detects LiteLLM proxy process spawning a shell, indicating potential RCE or hijacking.
references:
 - https://embracethered.com/blog/posts/2026/hijacking-litellm-for-fun-and-profit/
author: Security Arsenal
date: 2026/05/12
tags:
 - attack.execution
 - attack.t1059
logsource:
 category: process_creation
 product: linux
detection:
 selection:
   ParentImage|endswith: '/python'
   ParentCommandLine|contains: 'litellm'
   Image|endswith:
     - '/sh'
     - '/bash'
     - '/zsh'
     - '/dash'
falsepositives:
 - Legitimate administrative debugging by devs
level: high
---
title: LiteLLM Gateway Anomalous Outbound Connection
id: 9b5c2d03-1g8f-5e4b-0c6d-2e3f4a5b6c7d
status: experimental
description: Detects LiteLLM process establishing connections to non-standard ports or external IPs, potentially indicating C2 or data exfiltration.
references:
 - https://embracethered.com/blog/posts/2026/hijacking-litellm-for-fun-and-profit/
author: Security Arsenal
date: 2026/05/12
tags:
 - attack.exfiltration
 - attack.t1041
logsource:
 category: network_connection
 product: linux
detection:
 selection:
   Image|endswith: '/python'
   CommandLine|contains: 'litellm'
   Initiated: true
   DestinationPort:
     - 4444
     - 5555
     - 6666
     - 8080
     - 1337
   DestinationIsPrivateIP: false
falsepositives:
 - Integration with custom non-standard LLM endpoints
level: medium

KQL (Microsoft Sentinel / Defender)

This hunt query looks for Python processes associated with LiteLLM making network connections to external endpoints that are not known LLM providers.

KQL — Microsoft Sentinel / Defender

DeviceProcessEvents
| where Timestamp > ago(7d)
| where FileName == "python"
| where ProcessCommandLine contains "litellm"
| join kind=inner (
    DeviceNetworkEvents
    | where Timestamp > ago(7d)
    | where InitiatingProcessFileName == "python"
    | where InitiatingProcessCommandLine contains "litellm"
    | where RemotePort not in (443, 80)
    | extend DestinationIP = RemoteIP, DestinationPort = RemotePort
) on DeviceId, InitiatingProcessCommandLine
| project Timestamp, DeviceName, InitiatingProcessCommandLine, DestinationIP, DestinationPort, RemoteUrl

Velociraptor VQL

This VQL artifact hunts for LiteLLM configuration files with weak permissions (world-readable) which could lead to key theft.

VQL — Velociraptor

-- Hunt for LiteLLM config files with overly permissive permissions
SELECT FullPath, Mode, Size, Mtime
FROM glob(globs="/**/litellm_config.yaml")
WHERE Mode =~ "r..r..r.."
   OR Mode =~ "r..r.."

-- Hunt for LiteLLM processes with open network connections
SELECT Pid, Name, Cmdline, Family, RemoteAddress, RemotePort
FROM pslist()
JOIN netstat(pid=Pid) ON Pid
WHERE Name =~ "python" AND Cmdline =~ "litellm" AND RemoteAddress != "127.0.0.1" AND RemoteAddress != "::1"

Remediation Script (Bash)

Use this script to audit LiteLLM configurations and environment for exposed keys on Linux hosts.

Bash / Shell

#!/bin/bash

# LiteLLM Security Hardening & Audit Script
# Checks for exposed API keys and insecure file permissions

echo "[+] Auditing LiteLLM Configuration..."

# Define common config locations
CONFIG_LOCATIONS=("/root/litellm_config.yaml" "/app/litellm_config.yaml" "/etc/litellm/config.yaml" ".env")

for config in "${CONFIG_LOCATIONS[@]}"; do
  if [ -f "$config" ]; then
    echo "[!] Found config at: $config"
    
    # Check permissions
    PERMS=$(stat -c "%a" "$config")
    echo "    - Current Permissions: $PERMS"
    
    if [ "$PERMS" != "600" ] && [ "$PERMS" != "400" ]; then
      echo "    [WARNING] Permissions are too open. Recommended: 600 (rw-------)"
      chmod 600 "$config"
    fi
    
    # Scan for potential keys (heuristic for sk- or api- prefixes)
    if grep -q -E "(sk-|api-|access_key|secret_key)" "$config"; then
      echo "    [CRITICAL] Potential API keys found in plaintext file."
    fi
  fi
done

echo "[+] Checking Environment Variables..."

# Check if running litellm process has env vars exposed
if pgrep -f "litellm" > /dev/null; then
  PID=$(pgrep -f "litellm" | head -n 1)
  echo "[!] LiteLLM running with PID: $PID"
  
  # Dump environment (requires root) to check for keys
  if [ "$EUID" -eq 0 ]; then
    cat /proc/$PID/environ | tr '\0' '\n' | grep -E "(sk-|api-|OPENAI_API_KEY|ANTHROPIC_API_KEY)" && echo "    [CRITICAL] Keys found in process environment." || echo "    [OK] No obvious keys in environment."
  else
    echo "    [!] Run as root to inspect process environment variables."
  fi
else
  echo "[.] LiteLLM process not detected running."
fi

echo "[+] Audit complete."

Remediation

To mitigate the risk of LiteLLM hijacking and protect your AI infrastructure:

Isolate the Gateway: Run LiteLLM in a hardened container or VM with restricted network egress. Only allow outbound traffic to known, legitimate LLM provider endpoints (e.g., api.openai.com, api.anthropic.com).
Secrets Management: Never store API keys in configuration files (litellm_config.yaml) or environment variables. Integrate LiteLLM with a dedicated secrets manager (e.g., HashiCorp Vault, AWS Secrets Manager, Azure Key Vault) to retrieve keys at runtime.
Least Privilege: Ensure the operating system user running the LiteLLM process has read/write access only to necessary directories. Avoid running as root.
Integrity Monitoring: Implement File Integrity Monitoring (FIM) on LiteLLM configuration files to detect unauthorized modifications that might alter routing or tool definitions.
Version Control: Keep LiteLLM updated to the latest version to ensure you have the latest security patches. Review the LiteLLM Security Advisory (Official Vendor URL placeholder) regularly.

Related Resources

Security Arsenal Penetration Testing Services AlertMonitor Platform Book a SOC Assessment vulnerability-management Intel Hub