Simon Willison released llm-gemini 0.38.1, a dot release to his popular llm-gemini plugin for the llm command-line tool. The fix itself is modest: Gemini 4 Carbon supports only low, medium, and high reasoning levels, while Gemini 4 Hydrogen and Gemini 4 Neon expose an additional xhigh level — a distinction that was missed when the models were first added to the plugin. The practical impact is meaningful for users: Hydrogen at xhigh produces materially different output volume and cost characteristics than at high (in Willison's comparison grid, the same model at high consumed 5,311 output tokens at 3.99 cents, with xhigh diverging significantly from that baseline).
There is no CVE here, no exploit, and no active threat campaign. So why is a security consultancy writing about it? Because this release is a clean, real-world illustration of a governance gap we see constantly in enterprise environments: LLM client tooling, plugins, and model configurations are being deployed by developers and analysts with none of the change control, version pinning, or behavioral validation we apply to every other piece of software in the stack. A single configuration drift — an unadvertised reasoning level, a changed default, a model routing update — can alter cost, output behavior, data exposure patterns, and the reliability of anything downstream that consumes model output, including detection engineering and AI-assisted triage pipelines.
Technical Analysis
What changed in 0.38.1
- Product:
llm-geminiplugin for thellmCLI (github.com/simonw/llm-gemini) - Version: 0.38.1 (dot release)
- Fix: Reasoning-level support corrected per model family. Gemini 4 Carbon supports
low/medium/high. Gemini 4 Hydrogen and Gemini 4 Neon additionally supportxhigh. - Behavioral impact: Selecting
xhighon Hydrogen significantly changes output token consumption and per-call cost. Any automation, budget control, or output-length assumption built against thehighbaseline will behave differently atxhigh.
Why this matters to defenders
From a defensive standpoint, this class of issue maps to three risk categories we routinely flag in assessments:
-
Unmanaged AI tooling in the software inventory.
llmand its plugin ecosystem are installed via pip, often by individual engineers, frequently outside any package management or allow-listing regime. In most enterprises we assess, LLM CLI tools do not appear in the software inventory at all — which means no patch cadence, no license tracking, and no visibility into what models are being called with what data. -
Configuration-driven behavior change without change control. A reasoning-level flag is a configuration knob that changes cost, latency, and output structure. If your SOC is piping model output into summarization, alert triage, or report generation, a config change upstream is a silent change to your detection-adjacent pipeline. This release fixed a missing knob; the next release in any LLM tool could just as easily change a default.
-
Supply-chain surface. LLM plugins are third-party code with API credentials in scope — typically a Gemini/OpenAI/Anthropic API key sitting in the user's environment or keyring. Every plugin in the chain is code execution with access to those credentials and to whatever data the user pipes through the tool. Plugin version hygiene (knowing you're on 0.38.1, not a stale or typosquatted build) is basic supply-chain hygiene.
Exploitation status
Not applicable — this is a routine software release, not a vulnerability. No CVE is associated with this item, and there is no known malicious activity related to it. The defensive value is in the governance lesson, not in threat response.
Executive Takeaways
-
Inventory your AI tooling like you inventory everything else. LLM CLI clients, IDE assistants, plugins, and MCP servers belong in your software inventory and vulnerability/asset management pipeline. If you can't answer "who in this org has
llminstalled and which plugins are on it," that is a finding. Pull pip inventory from developer endpoints and build an AI tooling bill of materials (AI-BOM). -
Pin and review plugin versions. Dot releases like 0.38.1 change behavior silently. Require pinned versions for any LLM tooling used in production or security workflows, route updates through your normal change process, and review changelogs for default changes — not just security fixes — before upgrading.
-
Treat model configuration as change-controlled infrastructure. Reasoning levels, token budgets, temperature, and model routing all affect downstream consumers of model output. If your SOC uses LLM-assisted summarization or triage, document the exact model and configuration as a dependency, and alert on drift. A silent config change can degrade a triage pipeline without generating a single error.
-
Protect the credentials these tools hold. LLM CLI tools store provider API keys in user environments, config files, or OS keyrings. Those keys are high-value targets: they provide paid API access and, depending on scoping, access to logged prompts and organizational data. Scope keys minimally, rotate them, set spend and rate limits at the provider level, and hunt for API keys committed to repositories.
-
Control cost-amplification knobs. The
xhighexample shows a single flag multiplying token consumption. Set provider-side budget alerts and hard caps so a misconfigured client — or a compromised key being abused — triggers a billing alert before it becomes a five-figure surprise. Cost anomalies are also a useful tripwire for key theft. -
Vet the plugin supply chain before installation. Typosquatting and malicious lookalike packages targeting popular AI tooling are an active pattern across PyPI and npm. Enforce installation from approved internal indexes where feasible, verify package provenance, and apply the same third-party code review you'd give any dependency with credential access.
Remediation
For organizations already running llm with the Gemini plugin:
- Update the plugin to 0.38.1 to get correct reasoning-level support for Gemini 4 Hydrogen and Neon:
# Upgrade llm-gemini to the fixed release
llm install -U llm-gemini
# Verify installed version is 0.38.1 or later
llm plugins
- Audit reasoning-level usage in any scripts or pipelines that invoke Gemini 4 models, and confirm token budgets reflect actual configured levels (especially if
xhighis now available in your workflows). - Review provider-side controls in Google AI Studio / Vertex AI: confirm spend caps, quota alerts, and key scoping are in place for any API keys used by CLI tooling.
- Add LLM tooling to your asset inventory and set a review cadence for plugin updates, treating changelog review as part of change management.
Related Resources
Security Arsenal Penetration Testing Services AlertMonitor Platform Book a SOC Assessment vulnerability-management Intel Hub
Is your security operations ready?
Get a free SOC assessment or see how AlertMonitor cuts through alert noise with automated triage.