Back to Intelligence

SQLite 3.54.1 Homebrew Regression Breaks LLM CLI Logging — Upgrade to llm 0.37.3 to Preserve Security Telemetry

SA
Security Arsenal Team
November 16, 2026
11 min read

A silent failure in your security logging pipeline is one of the most dangerous conditions a SOC can operate under. This week, users of Simon Willison's llm command-line tool — a utility increasingly embedded in AI-assisted security workflows, incident response automation, and developer toolchains — experienced exactly that.

The release of llm 0.37.3 addresses a critical logging failure triggered by Homebrew's distribution of SQLite 3.54.1. The issue was not a traditional memory corruption vulnerability or authentication bypass; it was a behavioral regression in SQLite's JSON handling that caused the llm tool to fail when logging prompts and responses to its default SQLite database.

For defenders, this is a security telemetry integrity issue. The llm tool logs every interaction — including sensitive prompts, API keys passed in headers, model responses, and tool outputs — to a local SQLite database. When that logging silently breaks due to a dependency change, organizations lose forensic visibility into AI usage, potentially missing data exfiltration, policy violations, or malicious prompt injection activity.

If your security engineers, DFIR analysts, or developers use llm on macOS via Homebrew, verify your version immediately and ensure logging is functioning.

Technical Analysis

Root Cause: SQLite JSON Behavior Change

The regression stems from a fix in SQLite 3.54.1 (shipped by Homebrew in late 2026) that altered how json_object() handles subqueries returning JSON arrays.

In SQLite 3.54.0, the following query:

SQL
select json_object('a', (select json_array(1)));

Would return:

JSON
{"a":[1]}

However, in SQLite 3.54.1, the same query began returning a different structure (effectively treating the subquery result as a string literal rather than a JSON array), breaking applications that relied on the previous behavior.

The llm tool's logging layer uses similar JSON construction patterns to store structured prompt/response data. When Homebrew users upgraded SQLite (a routine brew upgrade), llm began failing to write logs, throwing errors or silently dropping records depending on the error handling path.

Affected Products and Platforms

  • Tool: llm CLI (Simon Willison's open-source LLM interface)
  • Affected Versions: All versions prior to 0.37.3
  • Fixed Version: llm 0.37.3
  • Trigger: Homebrew installations running SQLite 3.54.1
  • Platform: macOS (Homebrew package manager), though Linux users building against SQLite 3.54.1 may also be affected

Security Impact

While this is not a remote code execution vulnerability, the operational security impact is significant:

  1. Loss of AI Usage Telemetry: Security teams cannot audit what data was sent to external LLM APIs if logs are missing.
  2. Compliance Gap: Organizations subject to HIPAA, PCI-DSS, or internal AI governance policies may have unlogged data exposure windows.
  3. Forensic Blindness: During incident response, investigators rely on llm logs to reconstruct analyst queries, detect prompt injection attempts, or identify when sensitive data was pasted into AI prompts.
  4. Supply Chain Risk: This demonstrates how a transitive dependency update (SQLite) can cascade into application-layer security control failures without any code changes in the application itself.

Exploitation Status

This is a functional regression, not an actively exploited vulnerability. There is no CVE associated with this issue, and no evidence of malicious exploitation. However, the window of exposure — where logs were silently failing — creates a retrospective visibility gap that defenders must account for in their detection strategies.

Detection & Response

This incident represents a technical threat to security telemetry integrity. While no adversary TTPs are directly involved, the detection content below focuses on identifying broken logging states, version mismatches, and validating the integrity of the llm logging database.

Sigma Rules

The following rules detect the presence of vulnerable llm versions and SQLite 3.54.1 on endpoints, as well as potential silent logging failures.

YAML
---
title: LLM CLI Vulnerable Version Execution
id: 8f3e2a1b-4c5d-4e6f-8a9b-0c1d2e3f4a5b
status: experimental
description: Detects execution of llm CLI versions prior to 0.37.3 which are vulnerable to silent logging failures when paired with SQLite 3.54.1.
references:
  - https://github.com/simonw/llm/releases/tag/0.37.3
  - https://github.com/simonw/llm/issues/1879
author: Security Arsenal
date: 2026/11/16
tags:
  - attack.collection
  - attack.t1074
logsource:
  category: process_creation
  product: windows
detection:
  selection:
    CommandLine|contains:
      - 'llm '
      - 'llm.exe'
  filter:
    CommandLine|contains:
      - '--version'
      - 'version'
  condition: selection and not filter
falsepositives:
  - Legitimate use of llm CLI by developers and analysts
level: medium
---
title: SQLite 3.54.1 Detected on System
id: 9a4b3c2d-5e6f-7a8b-9c0d-1e2f3a4b5c6d
status: experimental
description: Detects the presence of SQLite 3.54.1 which introduces JSON handling changes breaking llm logging functionality.
references:
  - https://www.sqlite.org/releaselog/3_54_1.html
  - https://sqlite.org/bugs/info/2026-10-12T13:52:22Z
author: Security Arsenal
date: 2026/11/16
tags:
  - attack.defense_evasion
  - attack.t1562
logsource:
  category: process_creation
  product: linux
detection:
  selection:
    CommandLine|contains:
      - 'sqlite3 --version'
      - 'sqlite3 -version'
    CommandLine|contains:
      - '3.54.1'
  condition: selection
falsepositives:
  - Legitimate version checking by system administrators
level: low
---
title: LLM Logging Database Write Failure
id: 1a2b3c4d-5e6f-7a8b-9c0d-1e2f3a4b5c6d
status: experimental
description: Detects potential silent failure in llm CLI logging by identifying process execution without corresponding database modification.
references:
  - https://github.com/simonw/llm/issues/1879
author: Security Arsenal
date: 2026/11/16
tags:
  - attack.defense_evasion
  - attack.t1070
logsource:
  category: file_event
  product: macos
detection:
  selection:
    TargetFilename|endswith:
      - '/logs.db'
      - '/llm/logs.db'
    EventType: 'FileCreated'
  condition: selection
falsepositives:
  - Initial database creation is legitimate; alert if llm process runs but no file events occur within 5 minutes
level: medium

KQL (Microsoft Sentinel / Defender)

Use these queries to hunt for vulnerable llm versions and SQLite 3.54.1 across your macOS and Linux estate via Defender for Endpoint or Syslog ingestion.

KQL — Microsoft Sentinel / Defender
// Hunt for llm CLI execution on endpoints with SQLite 3.54.1 installed
// This identifies systems at risk of silent logging failure
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where FileName =~ "llm" or ProcessCommandLine contains "llm "
| where ProcessCommandLine !contains "--version" and ProcessCommandLine !contains "version"
| extend LLMVersion = extract(@"llm version ([0-9.]+)", 1, ProcessCommandLine)
| extend SQLiteVersion = extract(@"sqlite3 --version.*?3\.54\.1", 0, ProcessCommandLine)
| where isnotempty(LLMVersion) and LLMVersion != "0.37.3"
| project TimeGenerated, DeviceName, FileName, ProcessCommandLine, LLMVersion, InitiatingProcessFileName
| order by TimeGenerated desc

// Alternative: Check for Homebrew SQLite 3.54.1 installations via file creation events
DeviceFileEvents
| where TimeGenerated > ago(7d)
| where FileName contains "sqlite3"
| where FolderPath contains "/usr/local/Cellar/sqlite/3.54.1" or FolderPath contains "/opt/homebrew/Cellar/sqlite/3.54.1"
| project TimeGenerated, DeviceName, FileName, FolderPath, ActionType, InitiatingProcessFileName
| order by TimeGenerated desc

// Hunt for llm processes that may have failed to write logs (correlation with file events)
let llmProcesses = DeviceProcessEvents
| where TimeGenerated > ago(24h)
| where FileName =~ "llm" or ProcessCommandLine contains "llm "
| project ProcessTime=TimeGenerated, DeviceName, ProcessId, ProcessCommandLine;
let logWrites = DeviceFileEvents
| where TimeGenerated > ago(24h)
| where FileName contains "logs.db" and FolderPath contains "llm"
| project WriteTime=TimeGenerated, DeviceName, FileName;
llmProcesses
| join kind=leftanti logWrites on DeviceName, $left.ProcessTime < $right.WriteTime and $left.ProcessTime > $right.WriteTime - 5m
| project ProcessTime, DeviceName, ProcessCommandLine
| extend Alert = "Potential silent logging failure - llm ran but no logs.db modification detected"

Velociraptor VQL

For DFIR teams using Velociraptor, hunt for vulnerable llm installations and validate logging database integrity across macOS and Linux endpoints.

VQL — Velociraptor
-- Hunt for llm CLI installations with version < 0.37.3 and check SQLite version
-- Also verifies logs.db existence and recent modification times

LET llm_processes = SELECT Pid, Name, CommandLine, Exe, Username, CreateTime
FROM pslist()
WHERE CommandLine =~ 'llm'
  AND CommandLine !~ '--version'

LET llm_version_check = SELECT * FROM execve(
  argv=['llm', '--version'],
  length=1000000
)
WHERE Stdout =~ '0\.37\.[0-2]$'

LET sqlite_version = SELECT * FROM execve(
  argv=['sqlite3', '--version'],
  length=1000000
)
WHERE Stdout =~ '3\.54\.1'

LET log_files = SELECT FullPath, Size, Mtime, Ctime
FROM glob(globs='/Users/*/Library/Application Support/llm/logs.db')
WHERE Mtime > now() - 86400  -- Modified in last 24 hours

SELECT 
  'llm Installation' as CheckType,
  llm_version_check.Stdout as Evidence,
  'Vulnerable version detected - upgrade to 0.37.3' as Recommendation
FROM llm_version_check
UNION ALL
SELECT 
  'SQLite Version' as CheckType,
  sqlite_version.Stdout as Evidence,
  'SQLite 3.54.1 detected - ensure llm >= 0.37.3 is installed' as Recommendation
FROM sqlite_version
UNION ALL
SELECT 
  'Log File Status' as CheckType,
  FullPath + ' (Last modified: ' + Mtime + ')' as Evidence,
  CASE 
    WHEN Size = 0 THEN 'CRITICAL: Empty log file detected - logging may be broken'
    ELSE 'Log file exists and has content'
  END as Recommendation
FROM log_files

Remediation Script

Run the following Bash script on macOS endpoints to detect vulnerable configurations and upgrade llm to the fixed version.

Bash / Shell
#!/bin/bash
# LLM CLI SQLite Regression Remediation Script
# Detects vulnerable llm versions and SQLite 3.54.1, then upgrades to llm 0.37.3

set -euo pipefail

LOG_FILE="/var/log/llm_remediation_$(date +%Y%m%d_%H%M%S).log"
exec > >(tee -a "$LOG_FILE") 2>&1

echo "[$(date)] Starting LLM CLI security validation..."

# Check if llm is installed
if ! command -v llm &> /dev/null; then
    echo "[INFO] llm CLI not found. No action required."
    exit 0
fi

# Get current llm version
LLM_VERSION=$(llm --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' || echo "unknown")
echo "[INFO] Current llm version: $LLM_VERSION"

# Check if vulnerable version
if [[ "$LLM_VERSION" != "0.37.3" ]] && [[ "$LLM_VERSION" != "unknown" ]]; then
    echo "[WARNING] Vulnerable llm version detected (< 0.37.3)"
    VULNERABLE=true
else
    echo "[OK] llm version 0.37.3 or later detected"
    VULNERABLE=false
fi

# Check SQLite version
if command -v sqlite3 &> /dev/null; then
    SQLITE_VERSION=$(sqlite3 --version | awk '{print $1}')
    echo "[INFO] SQLite version: $SQLITE_VERSION"
    
    if [[ "$SQLITE_VERSION" == "3.54.1" ]]; then
        echo "[WARNING] SQLite 3.54.1 detected - this version triggers the logging regression"
        SQLITE_VULNERABLE=true
    else
        SQLITE_VULNERABLE=false
    fi
else
    echo "[INFO] sqlite3 CLI not found in PATH"
    SQLITE_VULNERABLE=false
fi

# Check if Homebrew is managing llm
if command -v brew &> /dev/null; then
    if brew list --formula | grep -q "^llm$"; then
        echo "[INFO] llm installed via Homebrew"
        INSTALL_METHOD="homebrew"
    else
        INSTALL_METHOD="other"
    fi
else
    INSTALL_METHOD="other"
fi

# Remediation
if [[ "$VULNERABLE" == true ]] || [[ "$SQLITE_VULNERABLE" == true ]]; then
    echo "[ACTION] Remediation required..."
    
    if [[ "$INSTALL_METHOD" == "homebrew" ]]; then
        echo "[ACTION] Upgrading llm via Homebrew..."
        brew update
        brew upgrade llm
        
        # Verify upgrade
        NEW_VERSION=$(llm --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' || echo "unknown")
        if [[ "$NEW_VERSION" == "0.37.3" ]]; then
            echo "[SUCCESS] llm upgraded to 0.37.3"
        else
            echo "[ERROR] Upgrade may have failed. Current version: $NEW_VERSION"
            exit 1
        fi
    else
        echo "[ACTION] Please upgrade llm manually to version 0.37.3 or later"
        echo "[ACTION] If installed via pip: pip install -U llm"
        echo "[ACTION] If installed via pipx: pipx upgrade llm"
    fi
    
    # Test logging functionality
    echo "[ACTION] Testing logging functionality..."
    TEST_DB=$(find ~ -name "logs.db" -path "*llm*" 2>/dev/null | head -1)
    if [[ -n "$TEST_DB" ]]; then
        echo "[INFO] Found log database: $TEST_DB"
        echo "[INFO] Database size: $(stat -f%z "$TEST_DB" 2>/dev/null || stat -c%s "$TEST_DB" 2>/dev/null) bytes"
    else
        echo "[WARNING] No logs.db found - may be fresh install or logging path issue"
    fi
    
else
    echo "[OK] No vulnerable configuration detected"
fi

echo "[$(date)] Remediation check complete. Log saved to: $LOG_FILE"

# Security recommendation
cat << 'EOF'

SECURITY RECOMMENDATIONS:
1. If llm was used for sensitive prompts between SQLite 3.54.1 installation and llm 0.37.3 upgrade, 
   assume those interactions were NOT logged.
2. Review API provider logs (OpenAI, Anthropic, etc.) for usage during the gap window.
3. Consider rotating API keys if sensitive data may have been exposed without local logging.
4. Implement version pinning for critical security tools in your environment.

EOF

Remediation

Immediate Actions

  1. Upgrade llm CLI: Update to version 0.37.3 or later immediately.

    • Homebrew: brew upgrade llm
    • pip: pip install -U llm
    • pipx: pipx upgrade llm
  2. Verify SQLite Version: Check if SQLite 3.54.1 is installed on your systems.

    • Run sqlite3 --version and document any instances of 3.54.1
    • Note that SQLite 3.54.1 is not inherently vulnerable — it is the combination with llm < 0.37.3 that causes failures
  3. Validate Logging Integrity: After upgrading, verify that llm is writing to logs.db:

    Bash / Shell

llm "test prompt" sqlite3 ~/Library/Application\ Support/llm/logs.db "SELECT COUNT(*) FROM logs;"

Code

### Forensic Gap Assessment

If your organization used `llm` between the Homebrew SQLite 3.54.1 update (approximately November 10-16, 2026) and the deployment of `llm` 0.37.3:

- **Assume a logging gap exists** for that period
- **Review cloud LLM provider logs** (OpenAI, Anthropic, Google, etc.) for API usage during the window
- **Check for sensitive data exposure**: If analysts were querying internal data or pasting incident details into prompts during the gap, treat it as unlogged sensitive data exposure
- **Document the gap** for compliance audits (HIPAA, PCI-DSS, SOC 2) that require AI usage logging

### Long-Term Hardening

1. **Dependency Pinning**: Pin critical security tools and their dependencies in your environment. Consider using `pipx` with virtual environments or containerized versions of `llm` to isolate from system package manager updates.

2. **Telemetry Validation**: Implement monitoring that alerts when security tools run but do not generate expected log output. The KQL query above can be adapted into a Sentinel analytic rule.

3. **Supply Chain Monitoring**: Subscribe to release notifications for both `llm` (via GitHub releases) and SQLite, as behavioral changes in underlying libraries can break security tooling without traditional CVEs.

4. **Alternative Logging**: Consider wrapping `llm` usage with additional logging layers (e.g., `script` command, shell history auditing, or network-level DLP monitoring) as defense-in-depth for AI usage telemetry.

### References

- **Official Fix**: [llm 0.37.3 Release](https://github.com/simonw/llm/releases/tag/0.37.3)
- **Issue Details**: [GitHub Issue #1879](https://github.com/simonw/llm/issues/1879)
- **SQLite Change**: [SQLite 3.54.1 Release Notes](https://www.sqlite.org/releaselog/3_54_1.html)
- **Root Cause**: [SQLite Bug Report](https://sqlite.org/bugs/info/2026-10-12T13:52:22Z)

## Related Resources

[Security Arsenal Penetration Testing Services](https://securityarsenal.com/services/penetration-testing)
[AlertMonitor Platform](https://securityarsenal.com/products/alertmonitor)
[Book a SOC Assessment](https://securityarsenal.com/contact)
[vulnerability-management Intel Hub](https://securityarsenal.com/intel/incident-response)

Is your security operations ready?

Get a free SOC assessment or see how AlertMonitor cuts through alert noise with automated triage.