This week, Simon Willison published a deceptively simple demonstration: he gave ChatGPT Work running GPT-6 Astra (Max) his home address and asked it to generate 5K and 10K looping running routes using OpenStreetMap data. The agent worked autonomously for 27 minutes and produced an embedded map visualization plus downloadable GPX and GeoJSON files.
As a technical achievement, it's impressive. As a security signal, it should make every CISO and SOC lead pause. The demo normalizes three behaviors that are actively dangerous in an enterprise context:
- Handing precise home-address PII to a consumer-grade LLM agent — data that may be retained, logged, used for training (depending on tier/settings), or exposed in a future breach of the provider.
- Trusting autonomous agents with 27-minute unsupervised execution windows — agents that browse, fetch external datasets (OSM), write files, and render output are an enormous prompt-injection and data-leakage attack surface.
- Publishing derived geolocation artifacts — a GPX route that loops 'from my house' is a de-anonymized home location, a pattern-of-life goldmine for anyone conducting physical or targeted social-engineering attacks.
There is no CVE here, no patch to deploy. The threat is behavioral and architectural: shadow AI adoption and agentic workflows are moving sensitive data — employee PII, executive home addresses, facility locations, client sites — into systems your DLP, CASB, and IR playbooks were never designed to cover. This post is about closing that gap.
Technical Analysis: Where the Risk Actually Lives
Affected 'products' and platforms
This isn't a vulnerability in OpenAI's stack — it's an exposure pattern that applies to any agentic AI assistant with tool use: ChatGPT Work, Copilot-style agents, Claude with computer use, and the growing class of autonomous 'deep research' agents. The risk multiplies when these agents:
- Run with persistent sessions and memory enabled
- Have browsing/fetch capabilities (pulling external data like OSM tiles, which can carry prompt-injection payloads)
- Can write and export files (GPX, GeoJSON, CSV) to local or synced storage
- Are accessed from personal, unmanaged accounts on corporate devices — the classic shadow AI gap
The attack chain defenders should model
From a threat-modeling perspective, treat an agentic session like the one in this story as a three-stage exposure:
- Ingestion of sensitive identifiers. The prompt itself — 'I live at
' — transmits a high-fidelity PII element to a third-party service. At scale, employees do this with far worse: customer records, internal hostnames, unreleased financials, patient-adjacent data. - Untrusted external data entering the agent context. The agent fetched OpenStreetMap data. Any agent that retrieves external content inherits the classic indirect prompt-injection problem: malicious instructions embedded in web pages, wiki text, or API responses can steer the agent's subsequent actions — including exfiltrating what it already knows (like that home address, or your session's other context).
- Artifact egress and publication. GPX/GeoJSON exports land in Downloads folders, sync to OneDrive/iCloud, and — as Willison did — get published. A route file anchored on a residence is sensitive on its own; aggregated with public fitness-platform data (the Strava heatmap lesson), it enables pattern-of-life surveillance of executives and staff.
Exploitation status
No in-the-wild exploit chain is tied to this specific story. The threat is active and structural: indirect prompt injection against browsing/tool-using agents is a documented, demonstrated technique throughout 2025–2026, and shadow AI data leakage is among the top-reported causes of unintentional data exposure in enterprise incident reviews this year. Treat this as a technique-level defensive problem, not a product vulnerability.
Detection & Response
The highest-fidelity signals here are at the network and SaaS-control layers: sensitive data flowing to AI endpoints from unmanaged accounts, and unexpected outbound transfer volumes to AI service domains. The rules below are tuned to be useful, not noisy — they anchor on server assets (which should essentially never talk to consumer AI endpoints) and on volume/pattern anomalies for workstations.
---
title: Consumer AI Service Connection from Server Infrastructure
id: 3f8a2b71-9c4d-4e61-a752-8d1c6f0b2a47
status: experimental
description: Detects network connections from server operating systems to consumer LLM/AI service endpoints. Servers have no legitimate business reason to initiate sessions to consumer AI chat or agent APIs; this indicates shadow AI tooling, an unauthorized agent installation, or data staging for exfiltration via an AI service.
references:
- https://attack.mitre.org/techniques/T1567/002
- https://simonwillison.net/2026/Sep/12/astra-running-routes/
author: Security Arsenal
date: 2026/09/12
tags:
- attack.exfiltration
- attack.t1567.002
logsource:
category: network_connection
product: windows
detection:
selection_server:
Image|endswith:
- '\svchost.exe'
- '\sqlservr.exe'
- '\w3wp.exe'
- '\httpd.exe'
- '\java.exe'
- '\node.exe'
- '\python.exe'
selection_ai_dest:
DestinationHostname|contains:
- 'chatgpt.com'
- 'chat.openai.com'
- 'claude.ai'
- 'gemini.google.com'
- 'copilot.microsoft.com'
- 'perplexity.ai'
condition: selection_server and selection_ai_dest
falsepositives:
- Approved AI integration workloads documented in the CMDB (tune by hostname exclusion)
level: high
---
title: AI Agent or Headless Browser Fetch of External Mapping and Tile Data
id: 6c1e9d04-2b7f-4a38-b965-4f2a8e1d7c93
status: experimental
description: Detects non-browser processes establishing connections to OpenStreetMap tile/Nominatim endpoints, consistent with an autonomous AI agent or script fetching geospatial data. Legitimate mapping lookups originate from browsers or sanctioned GIS tooling; automation runtimes fetching map data warrant review for unsanctioned agent activity and location-data handling.
references:
- https://attack.mitre.org/techniques/T1105
- https://attack.mitre.org/techniques/T1059/006
author: Security Arsenal
date: 2026/09/12
tags:
- attack.command_and_control
- attack.t1105
logsource:
category: network_connection
product: windows
detection:
selection_dest:
DestinationHostname|contains:
- 'tile.openstreetmap.org'
- 'nominatim.openstreetmap.org'
- 'overpass-api.de'
selection_proc:
Image|endswith:
- '\python.exe'
- '\pythonw.exe'
- '\node.exe'
- '\powershell.exe'
- '\pwsh.exe'
- '\curl.exe'
- '\wget.exe'
condition: selection_dest and selection_proc
falsepositives:
- Sanctioned GIS, logistics, or fleet-management tooling
- Developer environments with documented OSM integrations
level: medium
// Hunt: Large or sustained outbound transfers to consumer AI service domains
// Purpose: identify potential PII/prompt-data leakage and shadow AI usage patterns.
// Tune the byte threshold to your environment; look for users/devices, not single hits.
let AIDomains = dynamic([
"chatgpt.com", "chat.openai.com", "api.openai.com",
"claude.ai", "api.anthropic.com", "gemini.google.com",
"copilot.microsoft.com", "perplexity.ai", "grok.x.ai"
]);
DeviceNetworkEvents
| where TimeGenerated > ago(7d)
| where RemoteUrl has_any (AIDomains)
| summarize Connections = count(),
FirstSeen = min(TimeGenerated),
LastSeen = max(TimeGenerated),
InitiatingProcesses = make_set(InitiatingProcessFileName, 10)
by DeviceName, InitiatingProcessAccountName, RemoteUrl
| where Connections > 200 // sustained usage pattern, not incidental browsing
| project DeviceName, InitiatingProcessAccountName, RemoteUrl,
Connections, InitiatingProcesses, FirstSeen, LastSeen
| order by Connections desc;
// Hunt: Executable/script processes fetching open geospatial data (agent-like behavior)
// Requires Syslog/CEF ingestion for Linux or Defender process+network correlation.
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where ProcessCommandLine has_any (
"nominatim.openstreetmap.org", "overpass-api.de", "tile.openstreetmap.org")
or (ProcessCommandLine has_any ("gpx", "geojson")
and ProcessCommandLine has_any ("curl", "wget", "requests", "fetch"))
| project TimeGenerated, DeviceName, AccountName,
FileName, ProcessCommandLine, InitiatingProcessFileName
| order by TimeGenerated desc;
-- Hunt: AI agent/desktop clients and automation runtimes with live connections
-- to AI service endpoints, plus recently written geospatial export artifacts (GPX/GeoJSON).
-- Deploy as a Velociraptor hunt across the workstation fleet.
SELECT Pid, Name, Exe, Username, CommandLine
FROM pslist()
WHERE Name =~ '(?i)(chatgpt|copilot|claude|perplexity|openai)'
OR CommandLine =~ '(?i)(api\.openai\.com|api\.anthropic\.com|chatgpt\.com)'
-- Correlate with recent geospatial artifact drops in user-writable locations
SELECT FullPath, Size, Mtime
FROM glob(globs=[
'C:/Users/*/Downloads/*.gpx',
'C:/Users/*/Downloads/*.geojson',
'C:/Users/*/Desktop/*.gpx',
'C:/Users/*/OneDrive*/**/*.gpx'
])
WHERE Mtime > (now() - 604800) -- last 7 days
# Shadow-AI egress audit and hardening for Windows fleets
# 1) Audit outbound connections to consumer AI domains via DNS query logs / firewall
$AIDomains = @('chatgpt.com','chat.openai.com','claude.ai','gemini.google.com','perplexity.ai','copilot.microsoft.com')
# Review DNS client cache for evidence of AI endpoint resolution (run elevated, fleet-wide via GPO/Intune or EDR)
Get-DnsClientCache | Where-Object {
$name = $_.Entry; $AIDomains | Where-Object { $name -like "*$_*" }
} | Select-Object Entry, Data, TimeToLive | Export-Csv -Path "$env:TEMP\shadow_ai_dns_audit.csv" -NoTypeInformation
# 2) Verify your proxy/CASB category block is effective: these should FAIL where policy blocks consumer AI
foreach ($d in $AIDomains) {
try {
$r = Invoke-WebRequest -Uri "https://$d" -Method Head -TimeoutSec 5 -UseBasicParsing
Write-Output "[WARN] Reachable (policy gap): $d -> HTTP $($r.StatusCode)"
} catch {
Write-Output "[OK] Blocked or unreachable per policy: $d"
}
}
# 3) Enumerate recently created geospatial artifacts that may anchor on employee home locations
Get-ChildItem -Path "C:\Users" -Recurse -Include *.gpx,*.geojson -ErrorAction SilentlyContinue |
Where-Object { $_.LastWriteTime -gt (Get-Date).AddDays(-14) } |
Select-Object FullName, Length, LastWriteTime | Export-Csv -Path "$env:TEMP\geospatial_artifacts.csv" -NoTypeInformation
Write-Output "Review outputs in $env:TEMP and escalate findings per your shadow-AI IR playbook."
Remediation and Governance
There is no vendor patch — remediation is policy, architecture, and user behavior. Prioritized actions:
- Publish an agentic-AI acceptable-use policy now. Explicitly prohibit entering home addresses, customer PII, regulated data (PHI/PCI), facility locations, and internal identifiers into consumer AI tools. The running-route demo is a perfect internal training example of data that feels harmless but isn't.
- Route AI usage through sanctioned enterprise tiers. Enterprise agreements (with training opt-out, retention controls, audit logging, and SSO) are the difference between governed and ungoverned exposure. Block or coach-route consumer AI domains via CASB/SWG; the script above verifies your controls actually work.
- Treat agent browsing as untrusted input. Any agent that fetches external data (OSM, web pages, APIs) is an indirect prompt-injection target. Require human-in-the-loop approval for agent actions that export files, send messages, or touch credentials. Disable persistent memory where it isn't business-justified.
- Extend DLP to AI egress. Add AI service domains as an exfiltration channel class in your DLP/CASB. Alert on volume patterns (the KQL above) rather than single connections to keep fidelity high.
- Add location-data handling to your privacy program. GPX/GeoJSON artifacts anchored on residences are PII under most frameworks. Include derived geolocation data in data-classification schemes and in executive-protection guidance — senior staff publishing route maps from their homes is a physical-security issue, not just an IT one.
- Update IR playbooks. Add a 'shadow AI data exposure' scenario: identification (DNS/proxy telemetry), scoping (what was sent, which account, retention settings), and disclosure analysis against your regulatory obligations.
The Willison demo is a genuinely useful capability — and that's exactly why it's dangerous. Capabilities this frictionless get adopted before governance catches up. Get the policy, the egress controls, and the detection telemetry in place before your users' home addresses — and worse — are sitting in someone else's logs.
Related Resources
Security Arsenal Penetration Testing Services AlertMonitor Platform Book a SOC Assessment vulnerability-management Intel Hub
Is your security operations ready?
Get a free SOC assessment or see how AlertMonitor cuts through alert noise with automated triage.