AI-Driven LLM Gateway Harvesting: Defending Against the Self-Expanding Stolen Inference Supply Chain

A recent SANS Internet Storm Center diary documents something defenders have been predicting since LLM APIs went mainstream: an attacker running a semi-autonomous offensive operation in which an AI coding agent performs the bulk of the work — discovering poorly secured LLM resale gateways, exploiting ordinary web vulnerabilities to obtain API access, farming accounts at scale, validating the stolen inference capacity, and then aggregating all of it behind a single resale gateway of their own.

This is not a theoretical supply-chain scenario. The victim infrastructure in this campaign is the rapidly proliferating layer of LLM proxy and resale gateways — platforms such as LiteLLM, One-API, New-API, and custom FastAPI/Node.js front ends — that organizations and grey-market resellers stand up to broker access to OpenAI, Anthropic, Google, and open-weight model endpoints. Many of these gateways are deployed with default credentials, exposed administrative panels, no authentication on inference endpoints, or hard-coded master keys committed to public repositories. The result is a self-expanding stolen inference supply chain: compromised capacity is resold to downstream customers who have no idea their "cheap API access" is billed to someone else's account.

If your organization operates an LLM gateway, consumes third-party inference APIs, or has developers embedding LLM SDKs into applications, you are in scope for this threat on at least one side of it. The cost impact is direct — inference theft is literally metered theft — and the secondary risks (data exposure through logged prompts, attribution for abuse generated through your keys) are worse.

Technical Analysis

The Attack Chain

Based on the reported operation, the attacker's agent-driven workflow breaks into five observable phases:

Phase 1 — Discovery. The agent performs internet-wide reconnaissance for LLM gateway fingerprints: default landing pages and API paths such as /v1/models, /api/status, /login on One-API/New-API panels, LiteLLM proxy health endpoints (/health/liveliness, /key/info), and Shodan/FOFA-style queries for banners and favicons associated with these projects. Exposed instances on non-standard ports (3000, 8000, 8080) are prioritized.

Phase 2 — Initial Access. Exploitation relies on ordinary web flaws, not exotic zero-days:

• Default or weak credentials on gateway admin panels (One-API/New-API historically ship with a default root/123456 account; LiteLLM proxies are frequently deployed with the UI_USERNAME/UI_PASSWORD left at defaults or the master key set to a guessable value like sk-1234).
• Unauthenticated inference endpoints — misconfigured reverse proxies that expose /v1/chat/completions to the internet with no bearer token requirement.
• Common web vulnerabilities (SQLi, IDOR, path traversal) against custom-built resale panels, which are often hastily written PHP or Node applications.
• Leaked keys harvested from public GitHub/GitLab repositories, front-end JavaScript bundles, and exposed .env files.

Phase 3 — Account Farming. Where direct compromise fails, the agent automates account registration against gateways with open or weakly verified signup (disposable email, no CAPTCHA, no payment verification), accumulating free-tier quotas across hundreds of accounts and rotating through them to evade per-account rate limits.

Phase 4 — Validation. Stolen or farmed credentials are tested with low-cost inference calls (short completions to cheap models) to confirm the key works and to enumerate which upstream models the key can reach — effectively mapping the victim's provisioned capacity without triggering high-cost billing alerts.

Phase 5 — Aggregation and Resale. Validated keys are loaded into the attacker's own gateway (typically another One-API/New-API instance), which normalizes them behind a single OpenAI-compatible endpoint. The attacker's downstream customers send traffic to one URL; the attacker's gateway load-balances across dozens of stolen upstream keys, and the victims receive the invoices.

Why This Works

Three systemic weaknesses make this campaign viable:

1. The LLM gateway ecosystem matured faster than its security posture. Deployment guides prioritize "get it running in 5 minutes," and default configurations routinely expose admin functionality.
2. API keys are bearer tokens with enormous monetary value and almost no theft detection. Unlike a stolen password, a stolen sk-... key works silently from any IP on earth until someone reads a billing dashboard.
3. Inference traffic is voluminous and noisy by design, so malicious usage blends into legitimate load unless you are baselining spend, token volume, and source geography.

Affected Surface

• Self-hosted LLM proxies/gateways: LiteLLM, One-API, New-API, FastChat, ollama-webui and derivatives, plus custom resale panels.
• Organizations exposing inference endpoints directly (Ollama, vLLM, TGI) without an authentication layer.
• Any consumer of third-party "discount" LLM API resellers — you may be transacting on stolen infrastructure and inheriting its legal and availability risk.

Exploitation Status

Confirmed active in-the-wild exploitation of exposed gateways, with the novel element being the use of an autonomous/semi-autonomous agent to run the full kill chain at machine speed. No CVE is associated with this campaign — it is a configuration-and-exposure problem, not a patchable bug — and it is not (and cannot be) in CISA KEV. That makes posture management and detection engineering, not patch Tuesday, the primary defense.

Detection & Response

This is a technical threat with concrete, high-fidelity detection opportunities on both the gateway-hosting and the network-egress sides. The rules below are tuned to the specific behaviors in this campaign: scanning for LLM gateway fingerprints, brute-force/default-credential attempts against gateway admin panels, and anomalous outbound inference traffic patterns consistent with key validation and aggregation.

Sigma Rules

---
title: LLM Gateway Admin Panel Brute Force or Default Credential Attempts
id: 3f9a2b71-8c4d-4e6a-b1f2-9d7c5a3e8f01
status: experimental
description: Detects repeated authentication attempts against common self-hosted LLM gateway admin panels (One-API, New-API, LiteLLM) consistent with automated credential attacks or account farming observed in stolen-inference campaigns.
references:
  - https://isc.sans.edu/diary/rss/33332
  - https://attack.mitre.org/techniques/T1110/
author: Security Arsenal
date: 2026/02/20
tags:
  - attack.credential_access
  - attack.t1110
  - attack.t1078
logsource:
  category: webserver
  product: linux
detection:
  selection_uri:
    cs-uri-stem|contains:
      - '/api/user/login'
      - '/api/user/register'
      - '/login'
      - '/key/generate'
  selection_method:
    cs-method: 'POST'
  filter_known_app:
    cs-uri-stem|contains:
      - '/wp-login'
      - '/xmlrpc'
  condition: selection_uri and selection_method and not filter_known_app
falsepositives:
  - Legitimate user logins to internally hosted gateways; alert on volume thresholds (e.g. >10 POSTs per source IP per 5 minutes) in your SIEM correlation layer
level: medium
---
title: Reconnaissance Against LLM Gateway Inference Endpoints
id: 7c1e4d92-5a3b-4f8c-9e2d-6b8a1c4f7e03
status: experimental
description: Detects external probing of LLM inference and gateway health endpoints typical of automated discovery scanners hunting for exposed LiteLLM, One-API, Ollama, or vLLM instances.
references:
  - https://isc.sans.edu/diary/rss/33332
  - https://attack.mitre.org/techniques/T1595/
author: Security Arsenal
date: 2026/02/20
tags:
  - attack.reconnaissance
  - attack.t1595.002
logsource:
  category: webserver
detection:
  selection:
    cs-uri-stem|contains:
      - '/v1/models'
      - '/health/liveliness'
      - '/key/info'
      - '/api/status'
      - '/api/tags'
      - '/v1/chat/completions'
  condition: selection
falsepositives:
  - Legitimate client inference traffic; correlate on single source IPs hitting multiple discovery paths in short succession, and on sources with no prior inference history
level: low
---
title: LLM Client Process Spawning From Unusual Parent or Path
id: 9b2f5c18-3d7e-4a91-b6c4-2e8f5a1d9c07
status: experimental
description: Detects execution of LLM proxy binaries (litellm, one-api, new-api) from temporary or user-writable directories, consistent with an attacker standing up an aggregation gateway on compromised infrastructure.
references:
  - https://isc.sans.edu/diary/rss/33332
  - https://attack.mitre.org/techniques/T1059/
author: Security Arsenal
date: 2026/02/20
tags:
  - attack.execution
  - attack.t1105
logsource:
  category: process_creation
  product: linux
detection:
  selection_name:
    Image|endswith:
      - '/litellm'
      - '/one-api'
      - '/new-api'
  selection_path:
    Image|contains:
      - '/tmp/'
      - '/var/tmp/'
      - '/dev/shm/'
      - '/home/'
  condition: selection_name or selection_path
falsepositives:
  - Developer testing of gateway builds in home directories; baseline known deployment paths (e.g. /opt/, systemd unit paths) and alert on deviations
level: high

KQL — Microsoft Sentinel / Defender

This hunt assumes web/proxy logs are ingested into CommonSecurityLog (CEF from NGINX/Apache/WAF) or Syslog, and that you are hunting for the validation-phase behavior: an external source enumerating your gateway's available models and keys. A source IP that requests model/key inventory endpoints and then immediately begins inference calls from a geography or ASN with no prior history is the single highest-fidelity signal in this campaign.

// Hunt: External sources enumerating LLM gateway models/keys then initiating inference
let DiscoveryPaths = dynamic(["/v1/models", "/key/info", "/api/status", "/health/liveliness", "/api/tags"]);
let InferencePaths = dynamic(["/v1/chat/completions", "/v1/completions", "/v1/embeddings"]);
let Discovery =
    CommonSecurityLog
    | where TimeGenerated > ago(24h)
    | where RequestURL has_any (DiscoveryPaths)
    | summarize DiscoveryHits=count(), FirstSeen=min(TimeGenerated) by SourceIP, DeviceVendor;
CommonSecurityLog
| where TimeGenerated > ago(24h)
| where RequestURL has_any (InferencePaths)
| summarize InferenceCalls=count(), LastCall=max(TimeGenerated) by SourceIP
| join kind=inner Discovery on SourceIP
| extend TimeDeltaMinutes = datetime_diff("minute", LastCall, FirstSeen)
| project SourceIP, DiscoveryHits, FirstSeen, InferenceCalls, LastCall, TimeDeltaMinutes
| order by InferenceCalls desc;
// Companion hunt: new egress from servers that never previously called LLM APIs (stolen key validation from attacker infra is invisible here,
// but a compromised internal host RUNNING an aggregation gateway is not)
let KnownLLMHosts = dynamic(["api.openai.com", "api.anthropic.com", "generativelanguage.googleapis.com", "openai.azure.com"]);
let Baseline =
    DeviceNetworkEvents
    | where TimeGenerated between (ago(30d) .. ago(1d))
    | where RemoteUrl has_any (KnownLLMHosts)
    | summarize by DeviceName;
DeviceNetworkEvents
| where TimeGenerated > ago(24h)
| where RemoteUrl has_any (KnownLLMHosts)
| where DeviceName !in (Baseline)
| summarize Connections=count(), InitiatingProcesses=make_set(InitiatingProcessFileName), Accounts=make_set(InitiatingProcessAccountName) by DeviceName, RemoteUrl
| order by Connections desc;

Velociraptor VQL

Use this hunt across your Linux server fleet to find unauthorized LLM gateway processes — the aggregation layer an attacker (or a malicious insider) would stand up on compromised infrastructure. LiteLLM runs under Python; One-API/New-API ship as single Go binaries, making the executable name itself a strong indicator when found outside sanctioned deployment paths.

-- Hunt for unauthorized LLM gateway/proxy processes and their listeners
LET gateway_procs = SELECT Pid, Name, CommandLine, Exe, Username, CreateTime
FROM pslist()
WHERE Name =~ '(?i)(one-api|new-api|litellm|fastchat)'
   OR CommandLine =~ '(?i)(litellm.*--config|one-api|new-api|uvicorn.*proxy|/v1/chat/completions)'

SELECT Pid, Name, CommandLine, Exe, Username, CreateTime,
       Exe =~ '(?i)(/tmp/|/var/tmp/|/dev/shm/|/home/)' AS SuspiciousPath
FROM gateway_procs

-- Correlate with listening sockets owned by those processes
LET gateway_pids = SELECT Pid FROM gateway_procs

SELECT Pid, Name, Family, Type, Status, Laddr as LocalAddr, Lport as LocalPort
FROM netstat()
WHERE Status = 'LISTEN'
  AND Pid in (SELECT Pid FROM gateway_pids)

Remediation & Hardening Script

The following Bash script audits a Linux host running a self-hosted LLM gateway (LiteLLM or One-API/New-API) for the exact weaknesses this campaign exploits: default master keys, unauthenticated inference exposure, default admin credentials, and gateways listening on public interfaces. Run it on every gateway host; treat any FAIL output as an incident until proven otherwise.

#!/bin/bash
# LLM Gateway Security Audit - Security Arsenal
# Checks for exposure patterns exploited in the stolen-inference supply chain campaign

FAIL=0

echo "=== [1] LiteLLM master key strength check ==="
# Find litellm config files in common locations
for cfg in /opt/litellm/config.yaml /etc/litellm/config.yaml ./config.yaml ./litellm_config.yaml; do
  if [ -f "$cfg" ]; then
    echo "Found config: $cfg"
    # Flag trivially guessable master keys - the most common root cause of compromise
    if grep -E 'master_key:\s*"?sk-(1234|0000|key|test|admin|changeme)' "$cfg"; then
      echo "[FAIL] Weak/default master_key in $cfg - rotate immediately"
      FAIL=1
    fi
    # Flag missing auth on the proxy
    if ! grep -q 'master_key' "$cfg"; then
      echo "[FAIL] No master_key configured - inference endpoints may be unauthenticated"
      FAIL=1
    fi
  fi
done

echo "=== [2] Gateway processes listening on public interfaces ==="
# One-API/New-API default to :3000, LiteLLM to :4000, Ollama to :11434
ss -tlnp 2>/dev/null | grep -E ':(3000|4000|8000|11434)' | grep -E '0\.0\.0\.0|::' | while read -r line; do
  echo "[WARN] Public listener detected: $line"
  echo "       Verify this is behind an authenticated reverse proxy or bind to 127.0.0.1"
done

echo "=== [3] One-API / New-API default admin credential check ==="
# The default root/123456 account is the primary initial-access vector
for db in /data/one-api.db /opt/one-api/one-api.db ./one-api.db /opt/new-api/new-api.db; do
  if [ -f "$db" ]; then
    echo "Found database: $db - verify the default 'root' account password has been changed"
    echo "      (one-api/new-api ship with root/123456 - change via admin panel immediately)"
  fi
done

echo "=== [4] Registration/abuse surface check ==="
# Open registration enables account farming; disable unless required
for env in /opt/one-api/.env /opt/new-api/.env ./.env; do
  if [ -f "$env" ]; then
    grep -E '^(EMAIL_VERIFICATION|PASSWORD_REGISTER|GENERATE_DEFAULT_TOKEN)' "$env" 2>/dev/null
    echo "      Recommended: disable open registration, require email verification, disable default token grants on signup"
  fi
done

echo "=== [5] Egress check: unexpected LLM API destinations ==="
# Hosts other than your designated gateway should not be calling upstream LLM APIs
ss -tnp 2>/dev/null | grep -E 'ESTAB' | grep -E ':(443)' | grep -viE 'openai|anthropic|googleapis' >/dev/null || true
echo "Review netflow/proxy logs for direct egress to api.openai.com / api.anthropic.com from non-gateway hosts"

echo "=== [6] Billing guardrail reminder ==="
echo "Confirm hard spend limits and per-key budgets are set at the provider console -"
echo "this is your last line of defense against silent inference theft."

if [ "$FAIL" -eq 1 ]; then
  echo ">>> AUDIT FAILED: remediate findings above before this gateway is considered secure"
  exit 1
fi
echo ">>> Audit complete - review WARN items manually"

Remediation

Because this campaign exploits configuration and exposure rather than a single CVE, remediation is a posture program, not a patch. Prioritize in this order:

1. Inventory your LLM attack surface — today. Enumerate every self-hosted gateway, proxy, and inference server in your environment, including shadow-IT developer instances. Scan your own external ranges for the fingerprint paths in the Sigma rules above (/v1/models, /api/status, /health/liveliness). If you find an instance you didn't know about, assume it is already compromised and rotate every key it touches.

2. Rotate and scope your provider keys. Treat every LLM API key currently deployed as potentially exposed. Rotate them at the provider console (OpenAI, Anthropic, Google, Azure OpenAI), then re-issue them with the tightest available scoping: per-project keys, model allowlists, IP allowlists where supported, and — critically — hard monthly spend caps and per-key rate limits. A spend cap converts an unbounded theft into a noisy, self-limiting one.

3. Harden the gateway layer.

• Change all default credentials (One-API/New-API root account, LiteLLM UI credentials) and enforce SSO/OIDC on admin panels where supported.
• Bind gateway listeners to localhost or an internal interface and place them behind an authenticated reverse proxy (NGINX/Envoy with mTLS or OIDC) — never expose admin or inference endpoints directly to the internet.
• Disable open registration and default token grants on signup; require email verification and manual approval for new accounts.
• Keep gateways updated to current releases — the One-API/New-API/LiteLLM projects ship security fixes frequently, and upstream advisories for these projects are published in their respective GitHub repositories.

4. Establish inference baselines and billing alerts. You cannot detect theft you cannot see. Baseline daily token consumption and spend per key, alert on deviations (a 3x spike in gpt-4o input tokens from a key that normally serves embeddings is a smoking gun), and monitor the geographic/ASN distribution of callers hitting your gateway. New countries with no business justification warrant immediate key review.

5. Vet your inference supply chain. If you consume third-party LLM API resellers for cost savings, demand provenance: which upstream providers do they hold agreements with, and can they demonstrate it? Transacting on stolen inference capacity exposes you to sudden service loss, potential data exposure (your prompts flow through the attacker's aggregator and may be logged), and reputational risk. When in doubt, buy direct.

6. Prepare an IR playbook for key compromise. Define the sequence now: revoke key at provider → audit gateway logs for the abuse window → identify the access vector (default creds, exposed endpoint, leaked secret) → sweep code repositories and CI/CD logs for other exposed keys → assess whether prompts containing sensitive data transited during the compromise window. Inference theft is cheap to stop and expensive to investigate — speed matters.

Conclusion

The significance of this campaign is not any single technique — credential stuffing, exposed admin panels, and account farming are decades old. It is the operational model: an AI agent chaining commodity weaknesses into a self-expanding criminal infrastructure, harvesting inference capacity faster than human operators ever could, and laundering it through a legitimate-looking resale gateway. Defenders should expect this pattern to generalize to any metered, API-accessible compute resource. The controls that stop it are unglamorous — inventory, key hygiene, authentication, spend caps, and baseline monitoring — but they are entirely within your control, and every one of them is cheaper than the invoice a silent key compromise will eventually deliver.

Related Resources

Security Arsenal Penetration Testing Services AlertMonitor Platform Book a SOC Assessment vulnerability-management Intel Hub