llm 0.37 CLI Release: Security Implications of OpenAI-Compatible Endpoints, New Default Models, and Enterprise AI Tooling Governance

Simon Willison released version 0.37 of llm, his widely adopted open-source CLI tool and Python library for interacting with large language models. The release — roughly two months of accumulated features — includes support for new models (gpt-6.1-sol, gpt-6.1-terra), a new llm openai endpoint add command for configuring arbitrary OpenAI-compatible API endpoints, and a change of the default model to gpt-6-luna, half the price of the previous default gpt-5.6-luna. It also expands embeddings support.

There is no CVE here and no active exploitation. So why does a security consultancy care? Because tools like llm represent one of the fastest-growing and least-governed attack surfaces in the enterprise: developer AI tooling that holds cloud API credentials, pipes sensitive data to third-party inference endpoints, and now makes it trivially easy to point that data at any OpenAI-compatible server on the internet. If your SOC doesn't know where these tools are installed, what endpoints they're configured to talk to, and where the API keys live, you have a shadow AI problem — and this release is a good forcing function to fix that.

Technical Analysis

What's in the release

Per the 0.37 release notes, the security-relevant changes are:

New model support: gpt-6.1-sol and gpt-6.1-terra added to the model roster.
llm openai endpoint add command: a streamlined workflow for registering additional models hosted on any OpenAI-compatible endpoint — which includes a large ecosystem of third-party inference providers, self-hosted servers (vLLM, Ollama, LocalAI, LiteLLM), and, critically, any attacker-controlled or lookalike endpoint that speaks the OpenAI API protocol.
Default model change: users who never set a default are silently moved to gpt-6-luna. Silent default changes matter to defenders because cost-driven model downgrades can change data-handling characteristics (retention terms, regional processing, provider-side logging) without any change ticket ever being filed.
Expanded embeddings support: embeddings pipelines are a common path for bulk corporate data — documents, tickets, code — to leave the perimeter.

Why this is a defensive concern

From a defender's perspective, the risk chain looks like this:

Credential storage. llm stores API keys in a plaintext JSON file (keys.json, typically under the user's config directory, e.g. ~/.config/llm/ on Linux/macOS). Infostealer malware families have aggressively expanded their target lists to include AI tool credential stores over the past 18 months — OpenAI and Anthropic keys are now high-value loot because they convert directly to compute, and compromised keys have been observed being resold and used for fraud, spam generation, and proxying malicious traffic through a victim's billing account.
Endpoint redirection. The new endpoint add workflow lowers the friction for pointing the tool — and every prompt, file, and embedding you send through it — at an arbitrary URL. A typosquatted provider domain, a malicious config snippet pasted from a forum, or a compromised developer machine can redirect sensitive prompts and proprietary code to an attacker-operated OpenAI-compatible relay that logs everything and forwards it upstream so nothing looks broken.
Shadow AI and data egress. CLI LLM tools are routinely installed by developers outside any procurement or review process. Prompts frequently contain source code, credentials, customer data, and internal documents. Without inventory and egress monitoring, you have no visibility into which providers hold your data or under what retention terms.
Supply chain. llm is distributed via PyPI/Homebrew with a plugin architecture (llm-* packages). Third-party plugins execute arbitrary Python in the user's context with access to the key store. Plugin typosquatting and dependency confusion are realistic exposure paths for any pip-installable toolchain.

Exploitation status: No vulnerability is associated with this release, and there is no known exploitation. This is a governance and hardening post, not an incident advisory.

Executive Takeaways

Because this item is a software release rather than an exploit or intrusion, the value here is organizational. Six actions your security team should take this quarter:

Inventory AI CLI tooling. Add llm, aider, openai, anthropic, and similar packages to your software inventory queries (osquery, Defender, your EDR's installed-software tables). You cannot govern what you haven't enumerated. Treat any unapproved instance as shadow AI and route it through your exception process.
Govern the endpoint allowlist. Publish an approved list of inference endpoints (your enterprise OpenAI/Azure OpenAI/Anthropic contracts, plus any vetted self-hosted gateways). With endpoint add making custom endpoints a one-liner, policy without technical enforcement is decorative — enforce egress rules at the proxy/firewall so only allowlisted API hosts are reachable from developer subnets.
Protect and monitor the key stores. Audit permissions on ~/.config/llm/keys.json and equivalents (should be owner-read-only, 0600). Where possible, migrate developers to environment-variable or secrets-manager-backed key injection instead of on-disk files. Alert on infostealer-style access patterns to these paths — many EDRs let you add custom file-access watchlists.
Treat model default changes as change-management events. The silent switch to gpt-6-luna is a reminder that tool updates can change where and how data is processed. Pin model defaults via configuration management, and require a data-handling review when a provider or model tier changes — retention and training-use terms differ between tiers.
Control the plugin supply chain. llm plugins are arbitrary Python. Maintain an allowlist of approved plugins, pin versions, and install from an internal PyPI mirror or with hash-checking (pip install --require-hashes) to blunt typosquatting and dependency-confusion attacks.
Log and review usage. llm maintains a local SQLite log of prompts and responses. For sanctioned deployments, decide deliberately whether that log is a forensic asset (it can answer "what data went to which model, when") or a sensitive data store that itself needs protection and retention limits — it is frequently both.

Remediation and Hardening

No patch is required — but if you're standardizing on llm in your environment, apply the following baseline. First, audit what's deployed and how it's configured:

Bash / Shell

# Audit llm installations and configurations across a Linux/macOS fleet (run per-host or via your MDM/EDR shell)

# 1. Locate llm installs and versions
which llm && llm --version

# 2. Check key store permissions — must be 0600, owner-only
find /home /Users -name keys.json -path '*llm*' -exec ls -l {} \; 2>/dev/null

# 3. Enumerate configured custom endpoints — review against your allowlist
find /home /Users -name '*.json' -path '*llm*' 2>/dev/null
llm models list 2>/dev/null

# 4. Harden the key store if found
cnfg_dir="$(llm logs path 2>/dev/null | xargs dirname 2>/dev/null)"
[ -n "$cnfg_dir" ] && chmod 700 "$cnfg_dir" && chmod 600 "$cnfg_dir"/keys.json 2>/dev/null

Additional hardening steps:

Upgrade to 0.37 deliberately, not accidentally. Pull from the official GitHub releases or PyPI with version pinning; review the changelog before fleet rollout, because default-model changes alter data routing.
Egress control: restrict developer workstations to allowlisted API hosts (api.openai.com, your Azure OpenAI resource endpoints, approved internal gateways) at the proxy. A developer who adds a rogue endpoint via llm openai endpoint add should hit a deny rule, not the internet.
Key hygiene: rotate any OpenAI/Anthropic keys found stored on disk in unmanaged locations; enforce per-developer keys (not shared team keys) so revocation and attribution are possible; set spend limits and alerting on the provider side to catch key abuse quickly.
DLP alignment: confirm your DLP/CASB tooling inspects traffic to LLM API endpoints — many legacy rulesets still exclude api.openai.com and similar hosts as "trusted SaaS."

The Bottom Line

llm 0.37 is a solid release from a respected developer, and nothing in it is malicious. But every capability it adds — frictionless custom endpoints, cheaper default models, richer embeddings — increases the volume of enterprise data flowing to external inference providers through channels most SOCs don't monitor. The defensive lesson isn't about this tool specifically; it's that AI CLI tooling has quietly become credential-holding, data-egressing production infrastructure. Inventory it, allowlist its destinations, protect its keys, and log its use — before an infostealer or a pasted config snippet makes those decisions for you.

Related Resources

Security Arsenal Penetration Testing Services AlertMonitor Platform Book a SOC Assessment vulnerability-management Intel Hub