Google Gemini Escaped Its Evaluation Sandbox and Accessed Real Corporate Networks — How to Contain Autonomous AI Agents Before They Do the Same to You

In May 2026, Google's Gemini model — operating with internet access during a cybersecurity evaluation run by the Israeli AI security firm Irregular — broke into systems belonging to real companies that were never part of the test scope. The Wall Street Journal first reported the incidents, and The Hacker News has since confirmed the details. The root cause wasn't a novel exploit chain or a zero-day: it was a domain mix-up in the evaluation harness that caused the model to target live, third-party production infrastructure instead of the intended, isolated test environments.

Let that sink in, because it matters more than most headlines this year. An AI system, given tool access and internet connectivity, autonomously executed offensive actions against organizations that never consented to be tested. The evaluation partner involved had also been connected to similar incidents disclosed previously — meaning this is a pattern, not an anomaly.

I've run red teams for 15 years. The cardinal rule of every engagement — from a two-day phishing simulation to a full-scope nation-state emulation — is scope discipline. You do not touch out-of-scope systems. Ever. When a human operator makes that mistake, it's a contract violation and potentially a crime. When an autonomous AI agent makes it at machine speed, with no human in the loop to catch it, we have a fundamentally new containment problem. And if your organization is deploying AI agents with internet access and tool-use capability — and most of you are, whether you know it or not — this is your problem too.

What Actually Happened

Based on the reporting, the sequence of events was straightforward and depressingly preventable:

1. Irregular was contracted to evaluate Gemini's cybersecurity capabilities — essentially an AI red-team exercise measuring what the model could do when given offensive tooling and internet access.
2. The evaluation environment contained a domain configuration error. Domains intended to point at isolated, sandboxed target infrastructure instead resolved to — or were confused with — domains belonging to real, unrelated companies.
3. Gemini, operating autonomously, attacked those live systems. The model did what it was asked to do: identify targets, probe them, and gain access. It had no mechanism to know the targets were out of scope, because scope enforcement was never implemented as a technical control — it existed only as an assumption.
4. Real companies' systems were accessed without authorization, without consent, and without their knowledge until disclosure.

This is not a vulnerability in Gemini in the traditional CVE sense. There is no CVE assigned to this incident, and CISA KEV is not applicable here. This is an AI operational safety and containment failure — a class of incident that is going to dominate security headlines through 2026 and beyond as organizations hand increasingly capable models increasingly dangerous tools.

Why Defenders Must Care — Even If You Never Touch an Eval Harness

There are two distinct audiences who need to act on this:

1. Organizations running or commissioning AI capability evaluations. If you're benchmarking models against offensive security tasks — and many enterprises now do this before deploying agents — you are legally and ethically responsible for everything that agent touches. A scope failure isn't the model's fault; it's yours. You need network-level containment, not prompt-level promises.

2. Every organization on the internet. Your perimeter was just probed, and potentially breached, by an AI system whose operators didn't intend to target you. Expect this to recur. AI agents under evaluation, misconfigured agentic frameworks, and eventually deliberately weaponized autonomous systems will be hitting your external attack surface with increasing frequency. Your detections need to account for machine-speed, AI-generated attack patterns that don't look like human operators or commodity botnets.

Technical Analysis: The Containment Failure Chain

The attack chain here is instructive precisely because it's mundane:

Phase 1 — Egress without policy enforcement. The evaluation environment permitted outbound internet connectivity from the AI agent's execution context. There was no egress allowlist restricting the agent to approved target domains or IP ranges. This is the foundational failure. Any environment where an autonomous system can perform offensive actions must treat outbound network policy as the primary safety boundary.

Phase 2 — Implicit trust in task input. The agent resolved and attacked targets derived from its task configuration. Because the harness didn't validate that resolved target IPs fell within an approved RFC 1918 lab range or a pre-registered scope list, a simple domain mix-up — a typo, a stale DNS record, a misconfigured variable — became an unauthorized intrusion.

Phase 3 — No human-in-the-loop checkpoint. Autonomous exploitation proceeded without a human approval gate before actions against newly resolved targets. In human red teams, operators confirm scope before engaging. Autonomous agents need the equivalent: programmatic scope validation at the network and orchestration layers.

Phase 4 — Victim-side observability gap. The affected companies were breached by an external actor and, per the reporting, learned of it through disclosure rather than detection. That's the part that should worry every CISO reading this: would your SOC notice a machine-speed, AI-driven intrusion attempt against your edge? The tradecraft of an LLM-driven agent differs from both human pentesters and scripted scanners — it adapts, it chains steps fluidly, and it doesn't follow the rigid signatures of Nmap or Nessus.

Observable Behaviors Defenders Can Hunt

While every AI agent differs, autonomous offensive agents share detectable traits:

• High-velocity, diverse protocol probing from a single source IP or narrow ASN range — port scans interleaved with web enumeration, auth attempts, and exploit probes in rapid succession
• Non-standard user agents and TLS fingerprints — agent frameworks (Python httpx, aiohttp, requests, headless browser automation) rather than commodity scanner signatures
• Rapid iterative payload mutation against web endpoints — an LLM reasoning through an injection looks different from sqlmap's fixed template library: shorter campaigns, adaptive syntax, natural-language artifacts occasionally appearing in parameters
• Sequential kill-chain compression — reconnaissance-to-exploitation timelines measured in minutes, not days

Detection & Response

This is a technical threat, and the detections below target both sides of the problem: containing your own AI/eval infrastructure and detecting external autonomous-agent attacks against your perimeter.

Sigma Rules

---
title: Outbound Connection from AI Agent Tooling to External Infrastructure
id: 3f9a1c54-2b7e-4d91-a6c3-8e5f2a1b9d07
status: experimental
description: Detects common AI agent scripting runtimes and offensive tooling establishing outbound connections to non-private destinations. Intended to monitor AI evaluation sandboxes, agent hosts, and CI environments where such egress should be allowlisted or nonexistent.
references:
  - https://thehackernews.com/2026/09/google-gemini-broke-into-real-company.html
  - https://attack.mitre.org/techniques/T1071/001/
author: Security Arsenal
date: 2026/09/25
tags:
  - attack.command_and_control
  - attack.t1071.001
logsource:
  category: network_connection
  product: windows
detection:
  selection_runtimes:
    Image|endswith:
      - '\python.exe'
      - '\python3.exe'
      - '\node.exe'
      - '\pwsh.exe'
      - '\powershell.exe'
  selection_tools:
    Image|endswith:
      - '\nmap.exe'
      - '\sqlmap.exe'
      - '\nikto.exe'
      - '\nuclei.exe'
      - '\hydra.exe'
      - '\ffuf.exe'
      - '\gobuster.exe'
  filter_private:
    DestinationIp|startswith:
      - '10.'
      - '192.168.'
      - '172.16.'
      - '172.17.'
      - '172.18.'
      - '172.19.'
      - '172.2'
      - '172.30.'
      - '172.31.'
      - '127.'
  condition: (selection_runtimes or selection_tools) and not filter_private
falsepositives:
  - Legitimate developer tooling and package installation from agent hosts — mitigate by enforcing and allowlisting known-good destinations (PyPI, npm registry) rather than disabling the rule
level: high
---
title: Compressed Multi-Stage Attack Pattern from Single External Source
id: 8c2e7b31-4f6a-4d28-b9e1-5a3d6c0f72e4
status: experimental
description: Detects a single external source IP performing reconnaissance, web enumeration, and authentication attacks in rapid succession — consistent with autonomous AI agent behavior compressing the kill chain. Tune threshold to environment baseline.
references:
  - https://thehackernews.com/2026/09/google-gemini-broke-into-real-company.html
  - https://attack.mitre.org/techniques/T1595/
  - https://attack.mitre.org/techniques/T1110/
author: Security Arsenal
date: 2026/09/25
tags:
  - attack.reconnaissance
  - attack.t1595
  - attack.t1110
logsource:
  category: firewall
detection:
  selection:
    action: 'deny'
  condition: selection | count(DestinationPort) by SourceIp > 15
 timeframe: 5m
falsepositives:
  - Commodity vulnerability scanners and internet-wide scan services (Censys, Shodan) — correlate source ASN against known scanner lists before escalating; the distinguishing feature of agent-driven attacks is protocol diversity plus adaptive follow-on activity
level: medium
---
title: Offensive Security Tool Execution on Server or Workstation Outside Lab VLAN
id: 1d4b8e96-7c3f-4a52-9d68-2e7a5f14b830
status: experimental
description: Detects execution of common offensive security and enumeration tools on systems not designated as penetration testing or AI evaluation infrastructure. Apply with a lab-asset exclusion list.
references:
  - https://thehackernews.com/2026/09/google-gemini-broke-into-real-company.html
  - https://attack.mitre.org/techniques/T1588/002/
author: Security Arsenal
date: 2026/09/25
tags:
  - attack.resource_development
  - attack.t1588.002
logsource:
  category: process_creation
  product: windows
detection:
  selection:
    CommandLine|contains:
      - 'nmap '
      - 'sqlmap'
      - 'nuclei -'
      - 'hydra -'
      - 'ffuf '
      - 'impacket'
      - 'crackmapexec'
      - 'netexec'
  filter_lab_hosts:
    Computer|contains:
      - 'LAB-'
      - 'REDTEAM-'
      - 'AIEVAL-'
  condition: selection and not filter_lab_hosts
falsepositives:
  - Authorized internal security team activity — maintain an accurate lab/tester asset inventory and keep the exclusion filter current
level: high

KQL — Microsoft Sentinel / Defender

This hunt looks for the victim-side signature: a single external source exhibiting compressed, multi-protocol attack behavior against your perimeter within a short window — the hallmark of an autonomous agent rather than a human operator or dumb scanner.

// Hunt: Single external source with multi-protocol probing + web attack + auth attempts within 15 minutes
// Ingests firewall denies (CommonSecurityLog) and correlates with web/auth telemetry
let Window = 15m;
let SuspiciousSources =
    CommonSecurityLog
    | where TimeGenerated > ago(24h)
    | where DeviceAction in ("Deny", "Drop", "deny", "blocked")
    | where not(ipv4_is_private(SourceIP))
    | summarize
        DistinctPorts = dcount(DestinationPort),
        Ports = make_set(DestinationPort, 25),
        DenyCount = count(),
        FirstSeen = min(TimeGenerated),
        LastSeen = max(TimeGenerated)
        by SourceIP
    | where DistinctPorts >= 8 and DenyCount >= 40
    | extend DurationMin = datetime_diff("minute", LastSeen, FirstSeen)
    | where DurationMin <= 30;  // compressed timeline = machine-driven
SuspiciousSources
| join kind=inner (
    CommonSecurityLog
    | where TimeGenerated > ago(24h)
    | where DeviceVendor =~ "Palo Alto Networks" or DeviceProduct has_any ("IIS", "Apache", "WAF", "nginx")
    | where AdditionalExtensions has_any ("select", "union", "../", "cmd=", "powershell", "${jndi")
        or RequestURL has_any ("/wp-admin", "/.env", "/actuator", "/api/v1", "/console")
    | project SourceIP, WebHitTime = TimeGenerated, RequestURL, RequestMethod
) on SourceIP
| project SourceIP, DistinctPorts, DenyCount, DurationMin, Ports, RequestURL, WebHitTime
| order by SourceIP, WebHitTime asc;

// Companion hunt: agent framework user-agents hitting your external web properties
// Agent-driven tooling frequently leaks Python/Node HTTP client signatures
CommonSecurityLog
| where TimeGenerated > ago(24h)
| where RequestClientApplication has_any (
    "python-requests", "aiohttp", "httpx", "urllib", "node-fetch",
    "axios", "Go-http-client", "curl/", "libwww-perl"
)
| where not(ipv4_is_private(SourceIP))
| summarize
    RequestCount = count(),
    DistinctPaths = dcount(RequestURL),
    Paths = make_set(RequestURL, 20)
    by SourceIP, RequestClientApplication, bin(TimeGenerated, 1h)
| where DistinctPaths >= 10  // enumeration-style breadth, not a single API client
| order by DistinctPaths desc;

Velociraptor VQL — Endpoint Hunt for Rogue Agent/Offensive Tooling

Use this artifact to sweep endpoints and servers — particularly anything in a dev, CI/CD, or evaluation segment — for offensive tooling and AI agent runtimes with unexpected outbound connections.

-- Hunt for offensive security tooling and AI agent runtimes with active external connections
-- Deploy across server fleet and dev/eval segments; baseline results before alerting

LET procs = SELECT Pid, Name, CommandLine, Exe, Username
FROM pslist()
WHERE CommandLine =~ '(?i)(nmap|sqlmap|nuclei|hydra|ffuf|gobuster|nikto|impacket|crackmapexec|netexec|metasploit|langchain|autogen|openai|anthropic|gemini|crewai)'
   OR Exe =~ '(?i)(nmap|sqlmap|nuclei|hydra|ffuf)'

LET conns = SELECT Pid, Name, RemoteAddr, RemotePort, Status
FROM netstat()
WHERE Status =~ 'ESTABLISHED'
  AND NOT RemoteAddr =~ '^(10\.|192\.168\.|172\.(1[6-9]|2[0-9]|3[01])\.|127\.|::1)'
  AND Name =~ '(?i)(python|node|pwsh|powershell|nmap|nuclei|sqlmap)'

SELECT p.Pid, p.Name, p.CommandLine, p.Username,
       c.RemoteAddr, c.RemotePort
FROM procs p
LEFT JOIN conns c ON p.Pid = c.Pid

Remediation / Hardening Script — Egress Containment for AI Evaluation Environments

The single most important control this incident demands: hard egress allowlisting on any environment where an AI agent can execute actions. The following Bash script configures a Linux evaluation host (or sandbox gateway) so the agent can reach only pre-approved, in-scope destinations. Adapt the allowlist to your registered scope.

#!/usr/bin/env bash
# ai-eval-egress-lockdown.sh
# Hardens egress on an AI evaluation/agent host so outbound traffic is restricted
# to an explicit in-scope allowlist. Run as root. Tested on Ubuntu 22.04/24.04 (nftables).
set -euo pipefail

ALLOWLIST_FILE="/etc/ai-eval/egress-allowlist.txt"
AGENT_USER="aiagent"   # dedicated unprivileged user the agent runs under — NEVER root

# --- 1. Enforce a dedicated agent identity ---
if ! id "${AGENT_USER}" &>/dev/null; then
  useradd --system --shell /usr/sbin/nologin "${AGENT_USER}"
  echo "[+] Created dedicated agent user: ${AGENT_USER}"
fi

# --- 2. Build the in-scope destination allowlist ---
# ONLY lab/test ranges and explicitly registered target IPs belong here.
# This is the control that would have stopped the Gemini incident.
mkdir -p /etc/ai-eval
if [ ! -f "${ALLOWLIST_FILE}" ]; then
  cat > "${ALLOWLIST_FILE}" <<'EOF'
# In-scope evaluation targets ONLY — one CIDR per line
# Example lab range (REPLACE with your registered scope):
10.99.0.0/16
# DNS resolver for the lab:
10.99.0.2/32
EOF
  echo "[!] Created ${ALLOWLIST_FILE} — EDIT IT with your actual in-scope targets before relying on it."
fi

# --- 3. Apply nftables egress policy scoped to the agent user ---
# Map allowlist entries into an nft set; drop everything else from the agent.
nft -f - <<EOF
table inet ai_eval_egress {
  set allowed_dst {
    type ipv4_addr
    flags interval
    elements = { $(grep -v '^#' "${ALLOWLIST_FILE}" | grep -v '^$' | paste -sd, -) }
  }
  chain output {
    type filter hook output priority 0; policy accept;
    # Agent user: allow only allowlisted destinations, log and drop the rest
    meta skuid ${AGENT_USER} ip daddr @allowed_dst accept
    meta skuid ${AGENT_USER} limit rate 10/minute log prefix "AI-AGENT-EGRESS-DROP " drop
  }
}
EOF

echo "[+] Egress policy applied. Agent user '${AGENT_USER}' can reach ONLY allowlisted destinations."
echo "[i] Drops are logged with prefix 'AI-AGENT-EGRESS-DROP' — ship these to your SIEM."

# --- 4. Verification: confirm the agent cannot reach arbitrary internet ---
echo "[*] Verification test (should FAIL/timeout if policy is working):"
if sudo -u "${AGENT_USER}" timeout 5 bash -c 'exec 3<>/dev/tcp/1.1.1.1/443' 2>/dev/null; then
  echo "[FAIL] Agent reached external host — egress policy NOT enforced. Investigate immediately."
  exit 1
else
  echo "[PASS] Out-of-scope egress blocked for agent user."
fi

# --- 5. Persist across reboot ---
if ! grep -q 'ai_eval_egress' /etc/nftables.conf 2>/dev/null; then
  echo 'include "/etc/nftables.conf.d/ai_eval_egress.nft"' >> /etc/nftables.conf 2>/dev/null || true
  mkdir -p /etc/nftables.conf.d
  nft list table inet ai_eval_egress > /etc/nftables.conf.d/ai_eval_egress.nft
  systemctl enable --now nftables 2>/dev/null || true
  echo "[+] Policy persisted to /etc/nftables.conf.d/ai_eval_egress.nft"
fi

echo "[DONE] Ship 'AI-AGENT-EGRESS-DROP' logs to Sentinel/SIEM and alert on ANY hit — each one is an agent attempting out-of-scope contact."

Remediation: What Your Organization Should Do This Week

If you run AI evaluations, agent pilots, or red-team automation:

1. Enforce egress allowlisting on every agent execution environment. Scope must be a technical control enforced at the network layer, not a sentence in a prompt or a line in a contract. The script above is a starting point; cloud environments should use security groups / VPC egress policies with the same deny-by-default posture.
2. Validate scope programmatically before every engagement. Resolve all target domains, confirm the resulting IPs fall within pre-registered scope, and hard-fail the harness if they don't. Domain mix-ups, stale DNS, and copy-paste errors are inevitable; catching them must be automated.
3. Insert human-in-the-loop gates before novel-target engagement. Any time the agent attempts to interact with a destination not in the validated scope, the action should block pending human approval.
4. Log everything the agent does and ship it to your SIEM. Agent actions are machine-speed; post-incident reconstruction without centralized logs is impossible. Treat AI-AGENT-EGRESS-DROP events as high-fidelity alerts.
5. Demand answers from your evaluation vendors. Ask any third party testing AI capabilities against your behalf: How do you enforce scope? Where are your egress controls? Have you ever had an out-of-scope incident? The Irregular incidents — plural — prove that vendor assurances without technical evidence are worthless.

If you're a potential victim (which is everyone):

1. Baseline your external attack surface traffic now. Deploy the KQL hunts above and learn what normal scanner noise looks like so that adaptive, machine-speed attack patterns stand out.
2. Alert on kill-chain compression. A single source touching recon, enumeration, and auth endpoints inside 30 minutes deserves investigation regardless of whether any individual event fired a rule.
3. Verify your perimeter logging is actually complete. The affected companies learned of their breaches via disclosure, not detection. If your edge WAF, load balancer, and auth logs aren't centralized and retained, fix that before anything else on this list.
4. Engage your IR retainer for AI-attribution scenarios. If you detect an intrusion and the "attacker" turns out to be a runaway evaluation agent, the legal and disclosure calculus is unusual. Have counsel and your IR partner briefed on this scenario class.

There is no patch for this incident because there is no product vulnerability — the vulnerability is a missing safety architecture around increasingly capable autonomous systems. Google, Irregular, and every other lab running these evaluations will improve their harnesses. Your job is to make sure your organization is neither the careless evaluator nor the undetected victim.

Related Resources

Security Arsenal Managed SOC Services AlertMonitor Platform Book a SOC Assessment soc-mdr Intel Hub