llm-gemini 0.33 Adds Server-Side Code Execution for Gemini 3.7 Flash — What Security Teams Must Govern Before Developers Enable It

Simon Willison's llm-gemini plugin hit version 0.33, adding support for Google's newly released Gemini 3.7 Flash, plus gemini-3.6-flash, gemini-3.5-flash-lite, and two embedding models (gemini-embedding-2, gemini-embedding-001). The release also upgrades compatibility with LLM 0.32, exposing reasoning traces and — most consequential from a defensive standpoint — enabling server-side tools with a single flag:

llm -m gemini-3.7-flash -T CodeExecution

This is not a vulnerability disclosure. There is no CVE here, and nothing in this release is malicious — Willison's llm ecosystem is a legitimate, widely adopted open-source toolkit. But from where I sit after 15 years of IR and SOC work, releases like this are exactly how shadow AI takes root in an enterprise: a developer installs a plugin, pipes proprietary data through a third-party API, and enables a server-side code execution tool — all outside any sanctioned AI governance process. The defensive value of this news item is not patching; it is getting visibility and control over LLM CLI tooling before your developers outrun your policy.

Technical Analysis

What Changed in 0.33

Why Defenders Should Care

1. Server-side code execution expands the data-flow surface. When a user runs llm -m gemini-3.7-flash -T CodeExecution, the model can generate and execute code server-side as part of answering a prompt. The execution itself happens in Google's environment — your endpoint isn't running the code — but whatever the user stuffed into the prompt (source code, database extracts, credentials, customer data) is transmitted to a third party and processed by generated code you never reviewed. That is a data governance problem, not a malware problem, and it will not trip a single EDR rule you have deployed.

2. Embedding models are a quiet exfiltration vector. gemini-embedding-2 and gemini-embedding-001 are the kind of feature that ends up in RAG pipelines. An engineer building an internal "chat with our docs" prototype can embed your entire Confluence or SharePoint corpus through an external API in an afternoon. If your DLP and CASB controls don't account for generativelanguage.googleapis.com traffic, you have no idea this is happening.

3. Reasoning traces create a new sensitive artifact. LLM 0.32's reasoning trace support means chain-of-thought output — which frequently regurgitates the full prompt context — can land in terminal scrollback, shell history, CI logs, and local SQLite databases (the llm tool logs every prompt and response to a local logs.db by default). That local log store is a forensic goldmine for an intruder and a liability for you.

4. Supply-chain posture. The llm plugin ecosystem installs via pip/llm install. Every plugin is third-party Python code executing with the developer's privileges and, typically, with access to API keys stored in the environment or llm keys. A typosquatted or compromised plugin in this ecosystem is a credible future supply-chain scenario — which makes inventory and version pinning of these tools a real requirement, not hygiene theater.

Exploitation Status

Not applicable — this is a legitimate software release, not a threat advisory. No CVE is associated with this item, there is no PoC, and nothing is in CISA KEV. The risk is operational and governance-driven, and it is present today in any environment where developers self-serve AI tooling.

Executive Takeaways

Since this news item carries no exploit, the correct defensive output is organizational control, not detection rules. Six recommendations:

1. Inventory LLM CLI tooling now. Query your software inventory (and your EDR's installed-software telemetry) for the llm package and its plugins (llm-gemini, llm-openai, etc.). You cannot govern what you haven't enumerated. Treat unsanctioned instances the same way you treat unsanctioned cloud storage.

2. Publish an approved-models-and-tools list, and make the friction low. Developers route around governance that blocks them. If Gemini 3.7 Flash is acceptable for your data classification tiers, say so explicitly, define which data classes may be sent, and pre-approve llm-gemini 0.33 (pinned version, verified hash) through your internal package mirror or artifact repository.

3. Classify server-side tool flags as privileged features. -T CodeExecution and equivalents should require explicit approval per use case. The flag itself executes nothing locally, but it authorizes third-party processing of whatever the prompt contains. Your acceptable-use policy should name this pattern specifically.

4. Control egress to generative AI endpoints. At the proxy/ZTNA layer, alert on — and where policy dictates, restrict — traffic to generativelanguage.googleapis.com and peer AI API endpoints from non-approved hosts and service accounts. Baseline first: a developer population adopting Gemini 3.7 Flash this week will light up your egress logs, and that baseline is your enforcement foundation.

5. Treat local LLM logs and reasoning traces as sensitive data. The llm tool's default SQLite log (logs.db) captures every prompt and response. Include these paths in DLP scope, endpoint data-at-rest discovery, and — critically — your IR evidence-collection playbooks, because they also record exactly what an attacker or insider asked the model to do.

6. Harden the plugin supply chain. Require installation from an internal PyPI mirror with hash pinning, block pip install from public indexes on managed endpoints, and rotate any Gemini API keys that developers have stored in plaintext via llm keys on shared or unmanaged machines.

Remediation

There is no patch to deploy — there is posture to establish. Concrete steps:

• Pin and verify the package. If you approve llm-gemini, distribute version 0.33 from your internal artifact repository with a recorded SHA-256 hash. Pull directly from the canonical release only: https://github.com/simonw/llm-gemini/releases/tag/0.33
• Audit existing installations. Run a fleet-wide check for llm installations and their plugin lists (llm plugins), then reconcile against your approved list. On macOS/Linux endpoints, check ~/.local/share/io.datasette.llm/ (and legacy paths) for the logs.db artifact during any investigation involving suspected data leakage to AI services.
• Update your AI acceptable-use policy to explicitly cover CLI-based LLM tools, server-side tool flags, and embedding pipelines — most policies written in 2024-2025 cover chat web UIs and miss this entire class of usage.
• Add AI API endpoints to egress monitoring in Sentinel/your SIEM and define an alerting threshold for first-seen model invocation patterns (e.g., new user-agents or sudden volume to generativelanguage.googleapis.com from hosts that never touched it before).
• No vendor or CISA deadline applies to this item; the deadline is internal — before the next model release cycle makes your unmanaged AI footprint one size larger.

Related Resources

Security Arsenal Penetration Testing Services AlertMonitor Platform Book a SOC Assessment vulnerability-management Intel Hub