Simon Willison's llm-gemini plugin hit version 0.35 this week, continuing the rapid release cadence of one of the most widely used open-source bridges between the command line and Google's Gemini model family. On its face, this is a routine developer-tooling announcement — there is no CVE, no exploit, and no vulnerability disclosed in this release. So why is a security consultancy writing about it?
Because tools like llm and its Gemini plugin are exactly the kind of software that enters your environment quietly, gets authenticated with high-value API credentials, and starts moving sensitive data to third-party endpoints — all outside the visibility of most security programs. In 2026, the fastest-growing unmanaged attack surface we encounter during assessments isn't a forgotten VPN appliance or an unpatched CMS. It's AI tooling: CLI wrappers, IDE plugins, Jupyter kernels, and automation scripts that authenticate to LLM providers with broad-scope API keys and transmit whatever data the developer pastes, pipes, or points them at.
This post uses the llm-gemini 0.35 release as the hook for a conversation security teams need to have now: how command-line LLM tooling operates in your environment, where the credentials live, what data leaves your perimeter, and how to govern it without banning productivity outright.
Technical Analysis
What llm-gemini is
llm-gemini is a plugin for Simon Willison's llm CLI utility — a Python tool (distributed via PyPI, installable with pip or uv) that provides a unified command-line interface for interacting with large language models. The plugin adds support for Google's Gemini model family, allowing users to run prompts, process files and images, maintain conversation logs in a local SQLite database, and chain model calls from shell scripts and CI pipelines.
Why this matters from a defensive perspective
There is nothing malicious about this software. It is well-maintained, open-source, and developed by a reputable engineer. The risk is architectural, and it generalizes across the entire class of LLM CLI tooling:
1. API keys stored in plaintext on endpoints. The llm tool stores provider keys (e.g., a Google AI Studio / Gemini API key) via its llm keys set mechanism, which writes credentials to a keys.json file in the tool's configuration directory — typically ~/.config/io.datasette.llm/keys.json on Linux/macOS or the equivalent AppData path on Windows. These are plaintext, user-readable files. Any infostealer, post-exploitation triage script, or curious insider who can read a user's home directory can harvest a working Gemini API key. Gemini keys frequently have billing attached; key theft translates directly into financial abuse (quota-burning attacks) and, depending on the Google Cloud project scope, potential pivoting material.
2. Unmonitored egress to AI endpoints. Every prompt sent through the tool is an HTTPS request to generativelanguage.googleapis.com (or Vertex AI endpoints). Content pasted into these prompts — source code, log snippets containing credentials, customer data, incident notes — leaves your perimeter. Traditional DLP rarely inspects or categorizes this traffic meaningfully, and most proxy policies treat Google API domains as benign infrastructure.
3. Local conversation logging. The llm tool persists full prompt-and-response history to a local SQLite database (logs.db in the same config directory). This is a feature for developers — and a goldmine for an attacker or forensic subject. It is a consolidated record of everything the user asked an AI, which in practice often includes proprietary code, architecture details, and secrets the user didn't realize they'd included.
4. Supply chain exposure via PyPI. These tools install via pip install llm-gemini and pull dependency trees. Typosquatting and dependency-confusion attacks against AI-adjacent package names have been an active technique throughout 2025 and into 2026 — developers moving quickly to try a new model plugin are precisely the audience an attacker targets with a malicious lookalike package.
5. Scripting and automation amplification. Because llm is designed to be piped into shell workflows (cat incident_notes.txt | llm -m gemini-...), it gets embedded in cron jobs, CI/CD pipelines, and automation — places where credentials are often hardcoded or exported as environment variables (LLM_GEMINI_KEY or GOOGLE_API_KEY) in shell profiles, .env files, and pipeline definitions that end up committed to repositories.
Exploitation status
There is no vulnerability in llm-gemini 0.35 and no active exploitation associated with this release. The threat described here is the systemic, present-day pattern of infostealers harvesting AI provider API keys from endpoint configuration stores and of sensitive data egressing via ungoverned AI tooling — both techniques we routinely observe in incident response engagements in 2025–2026.
Detection & Response
This news item is a software release announcement rather than an active threat, so in place of threat-specific detection rules, the following are the executive and operational actions we recommend. Where endpoint telemetry exists, the governing principle is simple: treat LLM CLI tools like any other credential-bearing, data-egressing application — inventory them, watch their credential stores, and control where their traffic goes.
Executive Takeaways
1. Inventory AI tooling as software, not as curiosity. Add llm, llm-gemini, and comparable LLM CLI wrappers (aichat, mods, tgpt, and IDE-embedded agents) to your software inventory and application control policy. If your EDR/software inventory can't tell you which endpoints have PyPI AI packages installed, you have a shadow-AI visibility gap. Hunt installed Python packages and the presence of the io.datasette.llm config directory across the fleet.
2. Treat LLM API keys as privileged credentials. Gemini and other provider keys must live in a secrets manager (or at minimum an OS keychain-backed store), never in keys.json, shell profiles, .env files, or CI variables without rotation policy. Enforce per-developer keys with spend caps and quota alerts in Google AI Studio / Cloud console — a stolen key with no billing ceiling is an open checkbook. Add AI provider key patterns to your secret-scanning pipeline for repos and Slack.
3. Monitor and classify egress to AI endpoints. Create explicit proxy/CASB categories for generativelanguage.googleapis.com, *.googleapis.com AI surfaces, api.openai.com, api.anthropic.com, and similar. You don't need to block them — you need to know who is talking to them, at what volume, and from which processes. A sudden spike in Gemini API traffic from a non-developer endpoint is an anomaly worth a ticket.
4. Establish a data-handling policy for AI prompts — and enforce it technically. Policy alone fails; developers paste what's in their clipboard. Define what data classes may never enter third-party LLM prompts (credentials, PHI/PII, customer-confidential material, unreleased code where contractually restricted), then back it with DLP inspection on AI endpoint categories and periodic review. For regulated workloads, steer users toward enterprise agreements (Vertex AI with data-processing terms) rather than consumer-tier API keys.
5. Harden your package supply chain for Python AI tooling. Require installation through an internal PyPI mirror/proxy (e.g., Artifactory/Nexus) with typosquat and malware screening. Pin versions (llm-gemini==0.35) in any production automation, verify the package author, and alert on net-new Python package installations on servers and CI runners — an AI plugin appearing on a build server that never had one is a genuine anomaly.
6. Know where the logs live — for forensics and for privacy. The local SQLite conversation log (logs.db) cuts both ways: it's useful evidence in an IR engagement and a liability if the endpoint is compromised. Decide deliberately whether enterprise deployments should disable or encrypt local logging, and add the llm config directory to your forensic collection playbooks — it has already proven valuable in insider-risk investigations we've supported.
Remediation
There is no patch to apply — llm-gemini 0.35 contains no disclosed vulnerability. The remediation path is governance hardening:
- Immediate (this week): Scan endpoints and repositories for plaintext Gemini/Google API keys (search for the
AIzakey prefix pattern in files and forLLM_GEMINI_KEY/GOOGLE_API_KEYin shell profiles and pipeline configs). Rotate any key found in a committed file or shared location. - Short term (30 days): Enable per-key quota caps and billing alerts in Google Cloud / AI Studio; add AI endpoint domains to proxy logging; add
keys.jsonandlogs.dbpaths to EDR file-access monitoring policies for sensitive files. - Medium term (quarter): Publish a sanctioned AI tooling list, route Python package installs through a screened internal mirror, and migrate developer LLM access to enterprise-tier endpoints (Vertex AI) covered by data-processing agreements where regulated data is involved.
- Vendor references: the llm-gemini project and release notes live at simonwillison.net and the package on PyPI; Google's API key security guidance is available in the Google Cloud documentation.
Conclusion
llm-gemini 0.35 is a legitimate, useful tool from a reputable developer — and that is precisely the point. The AI tooling that lands in your environment won't announce itself as a risk. It arrives as a productivity win, authenticates with a plaintext key, logs every conversation to disk, and streams data to an external API your DLP doesn't categorize. Security teams that get ahead of this class of tooling now — with inventory, credential hygiene, egress visibility, and supply-chain screening — will avoid learning about it the hard way during an incident. If you need help baselining AI-tool usage across your fleet or building detections around it, that's exactly the kind of assessment we run.
Related Resources
Security Arsenal Penetration Testing Services AlertMonitor Platform Book a SOC Assessment vulnerability-management Intel Hub
Is your security operations ready?
Get a free SOC assessment or see how AlertMonitor cuts through alert noise with automated triage.