The 5% Problem: How to Detect and Contain AI Super-Users Hardcoding Unvetted Tools Into Your Business

Security teams have spent the past two years building acceptable-use policies for generative AI — mostly aimed at the average employee pasting meeting notes into a chatbot. According to new research published by Akamai, that focus is misaligned. The genuinely dangerous population is far smaller and far quieter: the top 5% of enterprise AI power users who have moved past casual experimentation and are now hardcoding unvetted AI tools directly into critical business operations.

This is the modern evolution of shadow IT, but with a steeper blast radius. A marketing analyst embedding an LLM API key into a Python script that processes customer data, a finance team member wiring an unvetted AI agent into an invoicing workflow, a developer committing an AI-generated code dependency that has never been reviewed — these are not policy violations. They are unreviewed architectural changes to your production environment, made by people with no security mandate, using third-party services your organization has no contractual, logging, or data-residency relationship with.

From an IR perspective, this matters for three reasons:

1. Data egress you can't see. Data flowing to api.openai.com, api.anthropic.com, or a hundred smaller AI SaaS endpoints over TLS looks identical to legitimate traffic unless you're specifically hunting for it.
2. Credential sprawl. Personal or team-level API keys hardcoded into scripts, notebooks, and CI pipelines are functionally equivalent to service accounts — with no rotation policy, no owner, and no monitoring.
3. Supply-chain ingestion. AI-generated code and AI-recommended packages enter your build pipeline without the review rigor you'd apply to any other third-party component.

If a single one of those super-user workflows processes regulated data (PCI, PHI, PII), you have a reportable compliance exposure even before any adversary gets involved. This post gives you the detection engineering and governance controls to find your 5% before an auditor or an attacker does.

Technical Analysis: What the Super-User Threat Actually Looks Like

Unlike a CVE-driven campaign, this is a behavioral and architectural threat class. There is no patch. The observable patterns, however, are consistent across engagements we've worked:

Pattern 1: Scripted LLM API Integration (Non-Interactive AI Use)

The defining characteristic of a power user versus a casual user is non-interactive invocation. Casual users type into a browser. Super users run python.exe, node.exe, or powershell.exe processes that reach out to LLM API endpoints with bearer tokens in headers or environment variables. This traffic is automation — which means it's embedded in a workflow, which means it's business-critical enough that someone built plumbing for it.

Key observables:

• Command lines containing LLM API hostnames (api.openai.com, api.anthropic.com, generativelanguage.googleapis.com, api.cohere.com) invoked via curl.exe, Invoke-RestMethod, or Invoke-WebRequest
• Scripts referencing environment variables like OPENAI_API_KEY, ANTHROPIC_API_KEY, or AZURE_OPENAI_KEY
• Long-lived scheduled tasks or cron jobs executing these scripts

Pattern 2: Hardcoded API Keys in Source and Notebooks

API keys committed to internal repos, Jupyter notebooks, and shared drives are the highest-signal artifact. OpenAI keys follow the sk- prefix pattern; Anthropic keys use sk-ant-. These keys are frequently personal-billing keys, meaning your corporate data is being processed under an individual's consumer-tier account with zero enterprise data protections.

Pattern 3: Volume Concentration

Akamai's core finding — that ~5% of users drive the disproportionate share of AI traffic — is itself a detection heuristic. In any environment, AI API traffic should be relatively flat across users in organizations without sanctioned AI integration. A Pareto distribution where a handful of users or devices account for the overwhelming majority of LLM-bound bytes is your super-user population, and it is enumerable.

Exploitation Status

This is not a vulnerability with a PoC — it is an exposure class. However, adversary interest in LLM API credentials is well documented through 2025 and into 2026: stolen API keys are sold and abused for LLM-powered phishing infrastructure, and prompt-injection against unsanctioned AI agents embedded in business workflows is an active attack surface. Treat any discovered hardcoded key as compromised until rotation is confirmed.

Detection & Response

The rules below are tuned for high signal. They deliberately exclude browser-based interactive use (that's a DLP/policy problem, not a detection engineering problem) and focus on the super-user behaviors: scripted invocation, hardcoded credentials, and traffic concentration.

Sigma Rules

---
title: Scripted Invocation of LLM API Endpoints via Command Line
id: 8c4e2b17-3f5a-4d91-ae26-7b8c9d0e1f2a
status: experimental
description: Detects command-line tools and scripting engines making direct calls to LLM API endpoints, indicative of unsanctioned AI tooling embedded into scripts or workflows rather than interactive browser use.
references:
  - https://thehackernews.com/2026/08/the-outsized-shadow-why-5-of-ai-users.html
  - https://attack.mitre.org/techniques/T1059/
author: Security Arsenal
date: 2026/08/15
tags:
  - attack.execution
  - attack.t1059
logsource:
  category: process_creation
  product: windows
detection:
  selection_engine:
    Image|endswith:
      - '\curl.exe'
      - '\powershell.exe'
      - '\pwsh.exe'
      - '\python.exe'
      - '\node.exe'
  selection_endpoint:
    CommandLine|contains:
      - 'api.openai.com'
      - 'api.anthropic.com'
      - 'generativelanguage.googleapis.com'
      - 'api.cohere.com'
      - 'openai.azure.com'
  condition: selection_engine and selection_endpoint
falsepositives:
  - Sanctioned AI integrations deployed by engineering teams — maintain an allowlist of approved service principals and hosts
level: medium
---
title: LLM API Key Material Present in Command Line or Script Invocation
id: 1d7f3a42-8c6b-4e59-bf31-2a9c5d6e7f80
status: experimental
description: Detects LLM API key patterns (OpenAI sk-, Anthropic sk-ant-) appearing in process command lines, indicating hardcoded credentials in scripts — a high-risk practice that creates unmonitored data egress and credential exposure.
references:
  - https://thehackernews.com/2026/08/the-outsized-shadow-why-5-of-ai-users.html
  - https://attack.mitre.org/techniques/T1552/
author: Security Arsenal
date: 2026/08/15
tags:
  - attack.credential_access
  - attack.t1552.001
logsource:
  category: process_creation
  product: windows
detection:
  selection_key_pattern:
    CommandLine|contains:
      - 'sk-ant-'
      - 'sk-proj-'
  selection_env:
    CommandLine|contains:
      - 'OPENAI_API_KEY'
      - 'ANTHROPIC_API_KEY'
      - 'AZURE_OPENAI_KEY'
  condition: 1 of selection_*
falsepositives:
  - Approved development activity — investigate key ownership and whether the key is enterprise-managed before suppressing
level: high
---
title: Non-Browser Process Network Connection to LLM API Endpoints
id: 5e2b8c63-4d7a-4f18-9c24-6e3b1a5d8f92
status: experimental
description: Detects non-browser processes establishing network connections to known LLM API endpoints, identifying programmatic AI integration that bypasses browser-based DLP controls.
references:
  - https://thehackernews.com/2026/08/the-outsized-shadow-why-5-of-ai-users.html
  - https://attack.mitre.org/techniques/T1071/
author: Security Arsenal
date: 2026/08/15
tags:
  - attack.command_and_control
  - attack.t1071.001
logsource:
  category: network_connection
  product: windows
detection:
  selection_image:
    Image|endswith:
      - '\python.exe'
      - '\node.exe'
      - '\powershell.exe'
      - '\pwsh.exe'
      - '\curl.exe'
      - '\java.exe'
  selection_destination:
    DestinationHostname|contains:
      - 'api.openai.com'
      - 'api.anthropic.com'
      - 'generativelanguage.googleapis.com'
      - 'api.cohere.com'
      - 'api.mistral.ai'
  condition: selection_image and selection_destination
falsepositives:
  - Sanctioned AI-enabled internal applications — baseline approved integrations by host and process path
level: medium

KQL — Microsoft Sentinel / Defender

This hunt identifies your 5%: it ranks users and devices by volume of network activity to LLM API endpoints and surfaces the concentrated outliers, while also flagging the scripted (non-browser) invocation that distinguishes embedded tooling from casual use.

// Hunt: AI super-user identification — traffic concentration and scripted LLM API usage
// Run over 14 days to establish concentration patterns
let Lookback = 14d;
let LLMDomains = dynamic(["api.openai.com", "api.anthropic.com", "generativelanguage.googleapis.com", "api.cohere.com", "api.mistral.ai", "openai.azure.com"]);
let AITraffic = DeviceNetworkEvents
| where TimeGenerated > ago(Lookback)
| where RemoteUrl in~ (LLMDomains)
| project TimeGenerated, DeviceName, InitiatingProcessAccountName, InitiatingProcessFileName, InitiatingProcessCommandLine, RemoteUrl, InitiatingProcessCreationTime;
// Scripted (non-browser) invocations — highest priority for review
let ScriptedUse = AITraffic
| where InitiatingProcessFileName !in~ ("msedge.exe", "chrome.exe", "firefox.exe", "brave.exe", "safari.exe")
| summarize ScriptedHits = count(), DistinctEndpoints = dcount(RemoteUrl), FirstSeen = min(TimeGenerated), LastSeen = max(TimeGenerated)
    by DeviceName, InitiatingProcessAccountName, InitiatingProcessFileName
| order by ScriptedHits desc;
// Concentration analysis — identify the top percentile of AI users
let Concentration = AITraffic
| summarize TotalHits = count(), BytesOfInterest = count() by InitiatingProcessAccountName
| order by TotalHits desc
| serialize RowNum = row_number()
| extend TotalUsers = toscalar(Concentration | count)
| extend Percentile = round(100.0 * RowNum / TotalUsers, 1)
| where Percentile <= 5.0;
ScriptedUse
| join kind=leftouter (Concentration | project InitiatingProcessAccountName, TotalHits, Percentile) on InitiatingProcessAccountName
| project DeviceName, InitiatingProcessAccountName, InitiatingProcessFileName, ScriptedHits, TotalHits, Percentile, DistinctEndpoints, FirstSeen, LastSeen
| order by ScriptedHits desc

Velociraptor VQL

This hunt artifact sweeps endpoints for live processes with LLM API connections and command lines containing AI API key material — useful for scoping during an assessment or incident.

-- Hunt: Enumerate processes with LLM API connections or hardcoded AI key material
-- Targets the shadow-AI super-user pattern: scripted, credentialed AI integration
SELECT Pid,
       Name,
       CommandLine,
       Exe,
       Username,
       CreateTime
FROM pslist()
WHERE CommandLine =~ '(?i)(api\.openai\.com|api\.anthropic\.com|generativelanguage\.googleapis\.com|api\.cohere\.com|api\.mistral\.ai)'
   OR CommandLine =~ '(sk-ant-[a-zA-Z0-9_-]{20,}|sk-proj-[a-zA-Z0-9_-]{20,})'
   OR CommandLine =~ '(?i)(OPENAI_API_KEY|ANTHROPIC_API_KEY|AZURE_OPENAI_KEY)'

-- Hunt: Live network connections from non-browser processes to LLM endpoints
SELECT Pid,
       Name,
       Path,
       Address AS LocalAddress,
       Raddr AS RemoteAddress,
       Status
FROM netstat()
WHERE Raddr =~ '.*'
  AND Name =~ '(?i)(python|node|powershell|pwsh|curl|java)'

Note: enrich the netstat() results with reverse DNS or correlate against your proxy/DNS telemetry for the LLM domain list, since netstat() returns IPs rather than hostnames. In practice, pairing this with the DNS query log from your forwarder is the faster path to attribution.

Remediation Script

This PowerShell script audits a Windows endpoint (run fleet-wide via your RMM/Intune/Velociraptor) for the two most common super-user artifacts: LLM API keys hardcoded in user script/code locations and AI-related environment variables.

# Security Arsenal — Shadow AI Key & Configuration Audit
# Run as SYSTEM or with read access to user profiles. Outputs CSV for central collection.

$OutputPath = "$env:ProgramData\SecArsenal\ShadowAIAudit_$(hostname)_$(Get-Date -Format yyyyMMdd).csv"
New-Item -ItemType Directory -Path (Split-Path $OutputPath) -Force | Out-Null
$Findings = @()

# 1. Scan common user code/script locations for hardcoded LLM API key patterns
$KeyPatterns = 'sk-ant-[a-zA-Z0-9_\-]{20,}|sk-proj-[a-zA-Z0-9_\-]{20,}|sk-[a-zA-Z0-9]{32,}'
$ScanRoots = @("$env:SystemDrive\Users")
$CodeExtensions = @('*.ps1','*.py','*.js','*.ts','*.ipynb','*.env','*.json','*.yaml','*.yml','*.toml','*.cfg','*.ini','*.sh')

foreach ($root in $ScanRoots) {
    Get-ChildItem -Path $root -Recurse -Include $CodeExtensions -ErrorAction SilentlyContinue |
      Where-Object { $_.Length -lt 5MB -and $_.FullName -notmatch '\\AppData\\(Local|Roaming)\\(Microsoft|Google)\\' } |
      ForEach-Object {
        $file = $_.FullName
        try {
            $matches = Select-String -Path $file -Pattern $KeyPatterns -ErrorAction SilentlyContinue
            foreach ($m in $matches) {
                $Findings += [PSCustomObject]@{
                    FindingType = 'HardcodedAPIKey'
                    Path        = $file
                    Line        = $m.LineNumber
                    # Redact the key itself — record only the prefix for triage
                    Indicator   = ($m.Matches.Value.Substring(0, [Math]::Min(10, $m.Matches.Value.Length)) + '...REDACTED')
                    Host        = hostname
                    Timestamp   = (Get-Date -Format o)
                }
            }
        } catch {}
      }
}

# 2. Check user and machine environment variables for AI API keys
$scope = @('User','Machine')
foreach ($s in $scope) {
    foreach ($var in @('OPENAI_API_KEY','ANTHROPIC_API_KEY','AZURE_OPENAI_KEY','GOOGLE_API_KEY','COHERE_API_KEY')) {
        $val = [Environment]::GetEnvironmentVariable($var, $s)
        if ($val) {
            $Findings += [PSCustomObject]@{
                FindingType = "EnvVariable_$s"
                Path        = "Environment:$var"
                Line        = ''
                Indicator   = ($var + ' = ' + $val.Substring(0, [Math]::Min(10, $val.Length)) + '...REDACTED')
                Host        = hostname
                Timestamp   = (Get-Date -Format o)
            }
        }
    }
}

$Findings | Export-Csv -Path $OutputPath -NoTypeInformation
Write-Output "Audit complete: $($Findings.Count) findings written to $OutputPath"

# IMPORTANT: Any discovered key must be treated as potentially compromised.
# Revoke/rotate through the provider console and migrate to an enterprise-managed,
# proxy-fronted AI gateway with per-service-account credentials.

Remediation and Governance

There is no vendor patch for this problem — the fix is architectural and procedural. Prioritize in this order:

Immediate (0–7 days):

1. Run the audit above fleet-wide. Every hardcoded personal API key you find is unmonitored data egress. Revoke and rotate every discovered key through the provider's console.
2. Stand up egress visibility. Ensure your proxy/CASB/firewall logs categorize traffic to major LLM API domains. If your secure web gateway can't distinguish api.openai.com from generic HTTPS, fix that first — you cannot govern what you can't log.
3. Brief the business, not just IT. The 5% are often your most productive employees. The goal is containment, not punishment — punitive responses drive the behavior further underground.

Short term (30 days): 4. Deploy an AI gateway / LLM proxy. Route all sanctioned AI usage through an enterprise-controlled gateway (e.g., a LiteLLM proxy, Azure AI Foundry, or your CASB's AI controls) with per-service-account credentials, full prompt/response logging, DLP inspection, and data-residency controls. Then block direct egress to LLM API endpoints at the firewall for everything except the gateway. 5. Extend DLP policy to AI endpoints. Apply the same inspection rules you use for email and cloud storage to LLM-bound traffic — especially regulated data patterns (PAN, PHI, SSNs, source code). 6. Add secret scanning to your SDLC. Pre-commit hooks and CI pipeline scanning (gitleaks, truffleHog, or your Git platform's native scanning) must include LLM key patterns.

Structural (quarter): 7. Formalize AI integration review. Any workflow that embeds an AI service into a business process must go through the same architecture and vendor review as any other third-party integration — data flow diagram, subprocessor assessment, contractual data-use terms, and logging requirements. 8. Inventory by concentration. Use the KQL concentration query quarterly. The top 5% of AI users by volume should be a known, documented, sanctioned population. Anyone new appearing in that percentile is a review trigger. 9. Update acceptable-use policy to explicitly distinguish interactive use (low risk, policy-governed) from programmatic integration (high risk, review-governed). Most 2024-era AI policies only address the former.

Compliance Framing

For PCI-DSS and HIPAA-regulated organizations: unsanctioned transmission of CHD or ePHI to a consumer-tier LLM service is a disclosure event under most interpretations. If your audit finds regulated data in super-user AI workflows, engage counsel on notification obligations — don't assume absence of an adversary means absence of a reportable incident. Map these controls to NIST CSF 2.0 (Govern and Identify functions — this is fundamentally an asset-and-data-governance gap) and CIS Control 3 (Data Protection) and Control 15 (Service Provider Management).

The Akamai research is a useful corrective: your AI risk register should not be sized by headcount. It should be sized by workflow criticality. Five percent of your users are making architecture decisions right now. Find them, instrument them, and give them a sanctioned path — before someone else finds the keys they left behind.

Related Resources

Security Arsenal Managed SOC Services AlertMonitor Platform Book a SOC Assessment soc-mdr Intel Hub