On November 12, 2026, AI tool developer Simon Willison released version 0.37 of llm-gemini, the plugin that connects his popular llm CLI utility to Google’s Gemini API. The release adds support for Gemini 4 Hydrogen, Google’s new lowest-cost model in the Gemini 4 family. While this is a routine software update, it is a signal flare for security operations teams: the barrier to deploying powerful generative AI in production and development workflows just dropped again.
For SOC analysts and security engineers, the risk is not a software vulnerability — there is no CVE here — but the rapid, often ungoverned adoption of new AI models. When a developer can run pip install llm-gemini and immediately access a frontier-class model for pennies per million tokens, corporate data can start flowing to external API endpoints before security even knows the integration exists. This post provides the detection, hunting, and hardening guidance you need to stay ahead of that curve.
Technical Analysis
What Changed
llm-gemini is a Python plugin maintained by Simon Willison. It registers Google’s Gemini models with the llm command-line tool, allowing users to pipe data to Gemini from shell scripts, CI/CD pipelines, and local development environments. Version 0.37 maps the newly announced gemini-4-hydrogen model identifier to the plugin’s model registry.
The release notes also highlight Google’s increasingly complex Gemini 4 pricing structure. Hydrogen is positioned as the budget option, but its input/output token costs are close enough to the larger Carbon and Argon models that misconfiguration or unexpected usage spikes could result in significant cloud bills. From a defensive perspective, this pricing opacity makes cost anomaly detection a secondary but useful signal for identifying unauthorized or compromised API usage.
Attack Surface & Threat Model
The llm ecosystem is designed for convenience, which historically correlates with weak credential hygiene. The primary risks introduced or amplified by this release are:
- API Key Exposure – Developers store Google AI Studio or Vertex AI API keys in plaintext environment variables (
GOOGLE_API_KEY,GEMINI_API_KEY) or dotfiles. These keys are frequently committed to Git repositories, baked into container images, or logged by shell history. - Shadow AI / Data Exfiltration – A low-cost model lowers the psychological barrier for employees to paste proprietary source code, customer PII, or internal documents into external LLM prompts. Because
llmcan read from stdin, a single command likecat customer_data.csv | llm -m gemini-4-hydrogen "summarize this"moves regulated data outside your compliance boundary. - CI/CD Poisoning – Build pipelines that integrate
llm-geminifor code review, test generation, or log summarization can be manipulated via prompt injection if the model output is executed or trusted without validation. - Token Theft & Resource Abuse – Stolen Gemini API keys can be used by attackers for free-tier phishing content generation, malware rewriting, or C2 prompt engineering, leaving your organization with the bill and the abuse complaints.
Exploitation Status
There is no CVE associated with llm-gemini 0.37, and no evidence of malicious exploitation of the plugin itself. The threat is technique-based: T1078 (Valid Accounts) and T1567 (Exfiltration Over Web Service) applied to generative AI APIs. Because the tool is open-source and benign, traditional AV/EDR will not flag it. Detection requires behavioral analytics and egress monitoring.
Detection & Response
The following rules and queries assume you ingest process execution logs (Sysmon, auditd, or EDR) and network connection logs (firewall, proxy, or cloud flow logs) into your SIEM. Tune thresholds to match your developer population.
Sigma Rules
---
title: LLM CLI Tool Execution with Google Gemini Model
id: 9f8e7d6c-5b4a-3c2d-1e0f-9a8b7c6d5e4f
status: experimental
description: Detects execution of the llm CLI tool referencing Google Gemini models, which may indicate ungoverned AI usage or data exfiltration risk.
references:
- https://github.com/simonw/llm-gemini/releases/tag/0.37
- https://attack.mitre.org/techniques/T1567/002/
author: Security Arsenal
date: 2026/11/12
tags:
- attack.exfiltration
- attack.t1567.002
logsource:
category: process_creation
product: windows
detection:
selection_img:
Image|endswith:
- '\llm.exe'
- '\python.exe'
- '\pythonw.exe'
selection_cli:
CommandLine|contains:
- 'llm '
- 'gemini'
condition: all of selection_*
falsepositives:
- Legitimate developer use of authorized AI tools
level: medium
---
title: Outbound Connection to Google Gemini API Endpoint
id: 1a2b3c4d-5e6f-7a8b-9c0d-1e2f3a4b5c6d
status: experimental
description: Detects network connections to Google generative AI API endpoints from non-browser processes, suggesting programmatic API usage that may bypass DLP controls.
references:
- https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-hydrogen/
- https://attack.mitre.org/techniques/T1071/001/
author: Security Arsenal
date: 2026/11/12
tags:
- attack.command_and_control
- attack.t1071.001
logsource:
category: network_connection
product: windows
detection:
selection_dst:
DestinationHostname|contains:
- 'generativelanguage.googleapis.com'
- 'aiplatform.googleapis.com'
selection_img:
Image|endswith:
- '\python.exe'
- '\pythonw.exe'
- '\llm.exe'
- '\node.exe'
- '\curl.exe'
- '\wget.exe'
condition: all of selection_*
falsepositives:
- Authorized Vertex AI workloads
- Approved AI-assisted development tools
level: high
---
title: Shell History Containing Gemini API Key Assignment
id: 7e6d5c4b-3a2f-1e0d-9c8b-7a6f5e4d3c2b
status: experimental
description: Detects bash/zsh history files containing plaintext Google API key assignments, indicating potential credential exposure.
references:
- https://attack.mitre.org/techniques/T1552/001/
author: Security Arsenal
date: 2026/11/12
tags:
- attack.credential_access
- attack.t1552.001
logsource:
category: file_event
product: linux
detection:
selection_path:
TargetFilename|endswith:
- '/.bash_history'
- '/.zsh_history'
- '/.config/fish/fish_history'
selection_content:
TargetFilename|contains:
- 'GOOGLE_API_KEY='
- 'GEMINI_API_KEY='
- 'export GOOGLE_API_KEY'
- 'export GEMINI_API_KEY'
condition: all of selection_*
falsepositives:
- None expected; API keys should not appear in shell history
level: high
KQL (Microsoft Sentinel / Defender)
Use this query to hunt for process and network indicators of llm CLI usage across your estate. It assumes you have Sysmon, common event format (CEF) firewall logs, or Defender for Endpoint data flowing into Sentinel.
// Hunt for llm CLI usage and Gemini API egress
let GeminiHosts = dynamic(["generativelanguage.googleapis.com", "aiplatform.googleapis.com"]);
let SuspiciousProcs = dynamic(["llm.exe", "python.exe", "pythonw.exe", "node.exe", "curl.exe"]);
union isfuzzy=true
(DeviceProcessEvents
| where FileName in~ (SuspiciousProcs)
| where ProcessCommandLine has_any ("llm", "gemini")
| project Timestamp, DeviceName, AccountName, FileName, ProcessCommandLine, InitiatingProcessFileName
| extend Indicator = "ProcessExecution"),
(DeviceNetworkEvents
| where RemoteUrl in~ (GeminiHosts)
| where InitiatingProcessFileName in~ (SuspiciousProcs)
| project Timestamp, DeviceName, InitiatingProcessAccountName, InitiatingProcessFileName, InitiatingProcessCommandLine, RemoteUrl, RemoteIP
| extend Indicator = "NetworkEgress"),
(CommonSecurityLog
| where DestinationHostName in~ (GeminiHosts)
| where ApplicationProtocol =~ "HTTPS"
| where SourceProcessName in~ (SuspiciousProcs) or SourceProcessName has "python"
| project TimeGenerated, SourceIP, SourceProcessName, DestinationHostName, DestinationIP, RequestURL
| extend Indicator = "FirewallEgress")
| order by Timestamp desc
| take 500
Velociraptor VQL
Deploy this artifact to hunt for live llm processes and their command lines across Windows and Linux endpoints.
-- Hunt for llm CLI and Gemini API usage
SELECT Pid,
Name,
Exe,
CommandLine,
Username,
CreateTime,
netstat().RemoteAddrIP AS RemoteIP,
netstat().RemoteAddrPort AS RemotePort
FROM pslist()
WHERE CommandLine =~ '(?i)(llm|gemini|generativelanguage|aiplatform)'
OR Exe =~ '(?i)(llm\.exe|python.*\.exe)'
ORDER BY CreateTime DESC
Remediation Script
Use this PowerShell script to audit Windows endpoints for the presence of llm and plaintext Google API keys in common locations. Run it via your EDR remote shell or SCCM.
# Audit-LLMGeminiUsage.ps1
# Checks for llm installation and exposed Gemini API keys
$Results = @()
# Check if llm is installed
$LLMPath = Get-Command llm -ErrorAction SilentlyContinue
if ($LLMPath) {
$Results += [PSCustomObject]@{
Hostname = $env:COMPUTERNAME
Check = "LLM CLI Installed"
Status = "Found"
Details = $LLMPath.Source
}
}
# Check for Gemini plugin
if ($LLMPath) {
$Plugins = & llm plugins list 2>$null | Select-String -Pattern "gemini"
if ($Plugins) {
$Results += [PSCustomObject]@{
Hostname = $env:COMPUTERNAME
Check = "llm-gemini Plugin"
Status = "Found"
Details = ($Plugins -join "; ")
}
}
}
# Check environment variables for API keys
$EnvKeys = Get-ChildItem Env: | Where-Object { $_.Name -match "GOOGLE_API_KEY|GEMINI_API_KEY" }
foreach ($Key in $EnvKeys) {
$Results += [PSCustomObject]@{
Hostname = $env:COMPUTERNAME
Check = "Exposed API Key in Env"
Status = "ALERT"
Details = "$($Key.Name) is set"
}
}
# Check common dotfiles for hardcoded keys
$Dotfiles = @("$env:USERPROFILE\.bashrc", "$env:USERPROFILE\.zshrc", "$env:USERPROFILE\.env")
foreach ($File in $Dotfiles) {
if (Test-Path $File) {
$Content = Get-Content $File -Raw
if ($Content -match "GOOGLE_API_KEY\s*=|GEMINI_API_KEY\s*=") {
$Results += [PSCustomObject]@{
Hostname = $env:COMPUTERNAME
Check = "Hardcoded API Key in $File"
Status = "ALERT"
Details = "File contains plaintext key assignment"
}
}
}
}
if ($Results.Count -eq 0) {
$Results += [PSCustomObject]@{
Hostname = $env:COMPUTERNAME
Check = "LLM/Gemini Audit"
Status = "Clean"
Details = "No indicators found"
}
}
$Results | Format-Table -AutoSize
Remediation
Because this is not a vulnerability with a patch, remediation focuses on governance, credential hygiene, and technical controls:
-
Inventory and Authorize – Run the audit script above across your fleet. Build an approved software list that explicitly includes or excludes
llmand similar AI CLI tools. Block unauthorized execution via AppLocker, WDAC, or your EDR’s application control module. -
Centralize API Key Management – Forbid plaintext API keys in environment variables and dotfiles. Require developers to use enterprise secrets managers (HashiCorp Vault, Azure Key Vault, AWS Secrets Manager) with short-lived tokens. If you use Google Vertex AI, enforce service account authentication with workload identity federation instead of long-lived API keys.
-
Egress Filtering and DLP – Restrict outbound HTTPS to
generativelanguage.googleapis.comandaiplatform.googleapis.comto authorized service accounts and proxy traffic through a DLP inspection point. If you cannot inspect TLS, at minimum log and alert on the SNI/hostname. -
Cost and Quota Guardrails – In Google AI Studio / Vertex AI, set per-project quotas and budget alerts at conservative thresholds. A sudden spike in
gemini-4-hydrogenusage from a single key is often the first indicator of a leaked credential. -
Policy and Training – Update your acceptable use policy to define which data classifications may be sent to external LLMs. Train developers that
cat secrets.txt | llm ...is exfiltration, even if the destination is a legitimate Google API. -
Monitor for Abuse – Subscribe to Google Cloud abuse notifications and monitor your Gemini API usage dashboards for geographic anomalies (e.g., API calls originating from ASNs outside your corporate footprint).
Category
soc-mdr
Related Resources
Security Arsenal Penetration Testing Services AlertMonitor Platform Book a SOC Assessment vulnerability-management Intel Hub
Is your security operations ready?
Get a free SOC assessment or see how AlertMonitor cuts through alert noise with automated triage.