llm-gemini 0.34 and Gemini 3.8 Flash: A Defender's Guide to Safely Adopting LLM CLI Tooling in Security Operations

On September 2, 2026, Simon Willison released llm-gemini 0.34, the Gemini plugin for his widely used llm command-line toolkit. The release adds support for Google's new Gemini 3.8 Flash model (with low, medium, and high thinking levels) and fixes a defect where async responses failed to record the resolved model version — a quiet but meaningful audit-trail bug credited to contributor Charlie Tonneslan.

There is no CVE here and no active exploitation. So why is a security consultancy writing about a point release of a Python CLI plugin? Because LLM tooling has quietly become part of the security operations toolchain. Analysts use llm to summarize alerts, triage phishing emails, draft IR timelines, and parse logs — often piping live incident data through it. Every such tool is a supply-chain dependency with an API key attached and a data path to an external provider. When the tool updates, defenders should treat it like any other infrastructure change: verify provenance, understand what changed, and re-validate the data-handling controls around it.

Two details in this release deserve a defender's attention specifically:

1. The async model-version logging fix. If your team uses llm's SQLite-backed conversation logging for auditability (and many do, precisely because Willison built it to be inspectable), async invocations prior to this fix recorded incomplete model attribution. That undermines forensic reconstruction of which model processed which data — a gap that matters during incident reviews, e-discovery, and compliance audits.
2. New model onboarding (gemini-3.8-flash). A new model identifier appearing in your environment means new data-flow questions: what thinking levels are in use, what data classification is permitted to reach it, and is it covered by your organization's existing AI usage policy and DLP controls?

Technical Analysis

What shipped in 0.34

• New model support: gemini-3.8-flash mapped for Google's Gemini 3.8 Flash release, including configurable thinking levels (low, medium, high). From a governance perspective, higher thinking levels mean longer prompts and responses retained in logs — potentially more sensitive content persisted locally in logs.db.
• Bug fix (#137): Async responses (llm -a / async SDK paths) now correctly record the resolved model version. Prior versions could log a request without the final resolved model string, breaking the chain of custody for AI-assisted analysis.
• Release provenance: Published via the official GitHub repository (simonw/llm-gemini, tag 0.34) and PyPI.

The defender's threat model for LLM CLI tooling

This is where the real value is. Tools like llm sit at a dangerous intersection:

• Supply chain: PyPI packages are a perennial typosquatting and dependency-confusion target. llm-gemini versus a malicious llm-gemini2 or llm_gemini lookalike is exactly the kind of mistake that happens during a rushed install at 2 AM during an incident. This is not hypothetical — it is the same technique class behind the package-impersonation campaigns that have hit PyPI throughout 2025 and 2026.
• Credential exposure: The plugin stores or references a Google API key (llm keys set gemini). On shared analyst workstations or jump boxes, that key is a target. Keys committed to dotfiles repos, shell history, or CI logs are one of the most common cloud-credential leak vectors we see in IR engagements.
• Data egress: Anything piped into the model leaves your perimeter. Alert summaries containing internal hostnames, IP addresses, employee names, or customer data go to Google's API. Whether that is acceptable depends on your data-processing agreements, retention settings with the provider, and regulatory posture (HIPAA and PCI-DSS scoping questions get very real, very fast).
• Audit integrity: The fixed bug is a reminder that AI-assisted decisions need reproducible logging. If an analyst used an LLM to help classify an incident severity and you cannot later prove which model produced which output, your post-incident review has a hole in it.

Exploitation status

None. There is no vulnerability in this release — this is a proactive governance and hardening exercise, not an emergency response.

Executive Takeaways

1. Pin and verify your LLM tooling supply chain. Install llm-gemini only from PyPI by exact name, pin versions in requirements files (llm-gemini==0.34), verify the project URL resolves to github.com/simonw/llm-gemini before installing anything new, and audit installed plugins quarterly with llm plugins. Treat analyst LLM tooling with the same rigor as EDR agents.

2. Upgrade to 0.34 if you use async invocations for anything audit-relevant. The model-version logging fix restores the integrity of the local conversation log. If your IR or threat-hunting workflows rely on llm's SQLite log for reconstructing AI-assisted analysis, unpatched async calls have attribution gaps — identify and annotate affected log entries from before the upgrade.

3. Rotate and scope your Gemini API keys. Store keys via llm keys set gemini (not environment variables in shared shell profiles), restrict keys in Google AI Studio / Cloud console to the minimum required models, and set an organizational rotation cadence. Hunt for keys accidentally committed to git and shell history.

4. Update your AI usage policy before enabling gemini-3.8-flash. Define which data classifications may be sent to external LLM APIs, whether thinking-level settings (which expand logged content) are approved, and which Gemini data-retention terms your organization has accepted. New model, old policy gap — close it proactively.

5. Monitor egress to generativelanguage.googleapis.com. Proxy and DNS logs will tell you who in the organization is using Gemini-backed tooling, sanctioned or not. Shadow AI usage by analysts pasting incident data into unapproved tools is a bigger practical risk than the tool itself.

6. Include AI tooling in your asset inventory. If you cannot answer "which workstations have LLM CLI tools and provider keys installed," you cannot assess exposure when a real supply-chain event hits this ecosystem — and given the cadence of PyPI compromise attempts in 2025–2026, it is a matter of when, not if.

Hardening and Verification

The following script validates an llm installation for the controls above: confirming the installed plugin version, verifying package provenance, and checking for API keys leaking into shell history or dotfiles.

#!/usr/bin/env bash
# llm-gemini 0.34 adoption & hardening verification
# Run on analyst workstations before/after upgrade

set -euo pipefail

echo "=== [1] Installed llm plugins ==="
llm plugins || { echo "llm not installed"; exit 0; }

echo
echo "=== [2] Verify llm-gemini version (expect >= 0.34) ==="
pip show llm-gemini 2>/dev/null | grep -E '^(Name|Version|Home-page|Project-URL)' \
  || echo "llm-gemini not found via pip"

echo
echo "=== [3] Upgrade to 0.34 (pinned) ==="
echo "Run: pip install 'llm-gemini==0.34'"
echo "Confirm PyPI project URL resolves to github.com/simonw/llm-gemini BEFORE installing."

echo
echo "=== [4] Check for typosquat lookalikes in site-packages ==="
pip list 2>/dev/null | grep -iE 'llm.?gemini|gemini.?llm' \
  | grep -viE '^llm-gemini ' \
  && echo "WARNING: unexpected gemini-related packages installed — investigate" \
  || echo "OK: only expected llm-gemini present"

echo
echo "=== [5] Hunt for Gemini API keys in shell history / dotfiles ==="
grep -rInE 'AIza[0-9A-Za-z_-]{35}' \
  ~/.bash_history ~/.zsh_history ~/.bashrc ~/.zshrc ~/.profile 2>/dev/null \
  && echo "WARNING: API key material found in plaintext — rotate immediately" \
  || echo "OK: no Gemini key patterns in common dotfiles"

echo
echo "=== [6] Audit local llm conversation log for async attribution gaps ==="
LLM_DB="$(llm logs path 2>/dev/null || true)"
if [ -n "$LLM_DB" ] && [ -f "$LLM_DB" ]; then
  echo "Log DB: $LLM_DB"
  sqlite3 "$LLM_DB" \
    "SELECT COUNT(*) AS missing_model FROM responses WHERE model IS NULL OR model = '';" \
    2>/dev/null || echo "Could not query log DB (schema may vary)"
else
  echo "No llm log database found"
fi

echo
echo "=== [7] Confirm key is stored via llm keyring, not env var ==="
if env | grep -qiE 'GEMINI_API_KEY|GOOGLE_API_KEY'; then
  echo "WARNING: provider key present in environment — prefer 'llm keys set gemini'"
else
  echo "OK: no provider key in environment"
fi

For Sentinel-enabled environments, egress monitoring for unsanctioned LLM API usage is a high-signal, low-noise hunt:

// Hunt: outbound connections to Google Generative Language API from non-approved hosts
// Tune the approved-device list to your sanctioned AI tooling inventory
let ApprovedHosts = dynamic(["SOC-ANALYST-01", "SOC-ANALYST-02"]);
DeviceNetworkEvents
| where TimeGenerated > ago(7d)
| where RemoteUrl has "generativelanguage.googleapis.com"
   or RemoteIP in ("142.250.0.0/15")  // illustrative; enrich via proxy/DNS instead where possible
| where DeviceName !in~ (ApprovedHosts)
| summarize FirstSeen=min(TimeGenerated), LastSeen=max(TimeGenerated), Connections=count()
  by DeviceName, InitiatingProcessName, InitiatingProcessCommandLine, RemoteUrl
| sort by Connections desc

Remediation

1. Upgrade: pip install 'llm-gemini==0.34' from PyPI only. Verify the package's Project-URL points to github.com/simonw/llm-gemini before installation — do not install from third-party mirrors or search-engine ads.
2. Audit async logs: If async invocations were used pre-0.34, query the SQLite log for responses lacking a resolved model and annotate your IR records accordingly.
3. Rotate API keys found anywhere outside the llm key store, and restrict key scope in Google's console.
4. Update policy and DLP to explicitly cover gemini-3.8-flash and its thinking-level settings before analysts begin using it.
5. Baseline egress to generativelanguage.googleapis.com and alert on deviation from approved hosts.

Official references: the llm-gemini 0.34 release notes, the upstream issue #146 and PR #137, and Google's Gemini 3.8 Flash announcement.

Related Resources

Security Arsenal Penetration Testing Services AlertMonitor Platform Book a SOC Assessment vulnerability-management Intel Hub