Social Engineering 3.0: Defending Against Autonomous AI Phishing Agents — Detection and Hardening Guide

For a decade, email security followed one model: scan the message, find something malicious — a bad URL, a weaponized attachment, a known-bad sender — and block it. That model died in two stages. First, the danger moved out of the payload and into the message's intent: business email compromise (BEC) and callback phishing carry no malicious content at all, so content scanners have nothing to latch onto. Now the second shift is here: the sender is no longer a person. Attackers are deploying autonomous AI agents that perform reconnaissance on targets, generate hyper-personalized lures at scale, and — critically — conduct real-time, multi-turn conversations with victims to walk them past every checkpoint your awareness training taught them.

This is the defining email-threat reality of 2026. Every SOC, IR team, and email security owner needs to understand what changed, why perimeter content filtering cannot see it, and where the actual defensive leverage points are. This post breaks down the attack mechanics and delivers the detection rules, hunt queries, and hardening steps your team can operationalize this week.

What Actually Changed: From Bad Content to Bad Intent to Autonomous Agents

The three generations

• Social Engineering 1.0 — Bad Content. Malware attachments, malicious links, macro-laden documents. Detection was a signature and sandbox problem. Gateways handled it well.
• Social Engineering 2.0 — Bad Intent. BEC, vendor email compromise (VEC), and callback phishing. The message is text. There is nothing to detonate. Detection shifted to impersonation logic, display-name spoofing, and anomaly scoring — imperfectly.
• Social Engineering 3.0 — Autonomous Agents. The phishing operation itself is now run by an AI agent. It scrapes LinkedIn, breach corpora, org charts, and press releases to build target dossiers; generates flawless, contextually aware lures in any language; registers lookalike infrastructure on demand; and — the step that breaks most defenses — replies to the victim in real time. When your CFO's assistant responds "is this really you?", the agent answers convincingly, in the impersonated party's voice, citing real context. The human tells we trained users to spot — broken grammar, generic greetings, odd urgency — are gone.

Why this matters to defenders

1. Volume and personalization are no longer a tradeoff. A human phishing crew could run high-touch spear phishing against maybe dozens of targets. An agent does it against thousands simultaneously, each message unique — which also defeats hash-based and bulk-pattern detection.
2. The kill chain extends past the inbox. The agent's goal is usually a conversation outcome: a wire transfer, credential entry on an adversary-in-the-middle (AiTM) page, MFA fatigue push acceptance, or an OAuth consent grant. Your telemetry has to cover the post-click, post-reply behavior — not just message ingestion.
3. Defense is becoming agent-versus-agent. Static rules and even trained classifiers lose against an adaptive adversary that rewrites itself per target. Defensive AI that reasons about intent — asking "does this request make sense given this relationship, this timing, this thread history?" — is moving from nice-to-have to mandatory. But technology alone won't save you; the controls below will.

Attack Chain: What an Agent-Driven Campaign Looks Like

From an IR seat, agent-run campaigns follow a consistent chain:

1. Autonomous reconnaissance. Agent harvests public data (org charts, vendor relationships, recent deals from press releases) to select impersonation and target pairs.
2. Infrastructure provisioning. Lookalike domains (often with valid SPF/DKIM/DMARC because the attacker controls them), AiTM phishing kits for session token theft, and redirector chains.
3. Initial lure. Highly personalized, thread-appropriate, no payload. Often references a real invoice, real project name, or real mutual contact.
4. Interactive manipulation. The victim replies; the agent handles objections, provides fake "verification," and escalates urgency — sometimes pivoting to voice (vishing) or messaging apps.
5. Objective execution. Credential/session theft via AiTM, OAuth consent grant to a malicious app, payment redirection, or malware delivery only after trust is established (so the payload arrives mid-thread, where gateway reputation is weakest).
6. Post-compromise persistence. Inbox rules to hide replies, external forwarding, additional OAuth grants, or mailbox delegation — the classic BEC follow-through, now executed at machine speed.

The critical insight: steps 5 and 6 are where your detection lives. You cannot reliably stop step 3 with content filtering anymore. You can detect and interrupt the post-compromise behaviors, and you can harden the environment so step 5 fails.

Exploitation Status

This is not theoretical. Security teams across the industry are reporting AI-generated, highly personalized phishing at scale, multi-turn conversational lures that survive user scrutiny, and deepfake-assisted vishing paired with email threads. No single CVE applies — this is a technique-class threat, not a patchable bug. Treat it with the same urgency as a KEV-listed vulnerability: it is being exploited in the wild, today, against organizations of every size.

Detection & Response

The detections below target the observable behaviors of these campaigns — inbox rule manipulation, malicious OAuth consent, anomalous outbound patterns, and AiTM session anomalies — rather than trying to signature the lure itself. That's deliberate: the lure is unique per victim, but the post-compromise actions converge.

Sigma Rules

---
title: Suspicious Inbox Rule Hiding or Deleting Mail
description: Detects creation of inbox rules that delete messages, move them to hidden folders, or mark them read — a hallmark of BEC and phishing post-compromise cleanup, including agent-driven campaigns that must suppress victim awareness of attacker replies.
references:
  - https://attack.mitre.org/techniques/T1098/002/
  - https://attack.mitre.org/techniques/T1114/002/
author: Security Arsenal
date: 2026/08/01
status: experimental
id: 3f8c1a92-7b4d-4e5a-9c1f-2d6e8a0b3f41
logsource:
  product: office365
  service: exchange
detection:
  selection_operation:
    Operation:
      - 'New-InboxRule'
      - 'Set-InboxRule'
  selection_hide:
    Parameters|contains:
      - 'DeleteMessage'
      - 'MoveToFolder'
      - 'MarkAsRead'
  condition: selection_operation and selection_hide
falsepositives:
  - Legitimate user-created filing rules; tune by correlating with recent sign-in anomalies or impossible travel
level: high
---
title: Mailbox External Forwarding Enabled
description: Detects mailbox forwarding or redirect rules targeting external addresses, and Set-Mailbox forwarding changes. Common exfiltration and monitoring technique after agent-driven credential theft.
references:
  - https://attack.mitre.org/techniques/T1114/003/
author: Security Arsenal
date: 2026/08/01
status: experimental
id: 8a2e5d17-4c6b-4f1a-b3d9-5e7c2a8f0b62
logsource:
  product: office365
  service: exchange
detection:
  selection_operation:
    Operation:
      - 'New-InboxRule'
      - 'Set-InboxRule'
      - 'Set-Mailbox'
  selection_forward:
    Parameters|contains:
      - 'ForwardTo'
      - 'ForwardAsAttachmentTo'
      - 'RedirectTo'
      - 'ForwardingSmtpAddress'
  condition: selection_operation and selection_forward
falsepositives:
  - Executive assistant delegation and sanctioned forwarding; maintain an allowlist of approved forwarding pairs
level: high
---
title: OAuth Consent to New or Unverified Application
description: Detects consent grants to applications — a primary objective of conversational phishing agents that steer victims to approve mail-read or offline-access permissions instead of stealing a password.
references:
  - https://attack.mitre.org/techniques/T1528/
  - https://attack.mitre.org/techniques/T1550/001/
author: Security Arsenal
date: 2026/08/01
status: experimental
id: 5c1f9b34-6e8a-4d2c-a7f3-1b4d6e9c2a85
logsource:
  product: azure
  service: auditlogs
detection:
  selection:
    OperationName:
      - 'Consent to application'
      - 'Add service principal'
      - 'Add OAuth2PermissionGrant'
  condition: selection
falsepositives:
  - Normal SaaS onboarding; alert on first-seen app names and publishers, and scope to high-value users first
level: medium

KQL — Microsoft Sentinel / Defender Hunting

This query chains the behaviors that matter: an anomalous sign-in, followed within a short window by inbox rule creation or OAuth consent — the signature of a phishing agent converting a stolen session into persistence.

let lookback = 14d;
let window = 2h;
let SuspiciousSignins = SigninLogs
| where TimeGenerated > ago(lookback)
| where ResultType == 0
| extend RiskEvents = tostring(RiskEventTypes)
| where RiskLevelDuringSignIn in ('high','medium')
   or RiskEvents has_any ('anonymizedIPAddress','unfamiliarFeatures','maliciousIPAddress','suspiciousIPAddress','tokenTheft')
| project SigninTime = TimeGenerated, UserPrincipalName, IPAddress, AppDisplayName, RiskEvents, SessionId;
SuspiciousSignins
| join kind=inner (
    OfficeActivity
    | where TimeGenerated > ago(lookback)
    | where Operation in~ ('New-InboxRule','Set-InboxRule','Set-Mailbox')
    | where Parameters has_any ('DeleteMessage','ForwardTo','RedirectTo','ForwardingSmtpAddress')
    | project RuleTime = TimeGenerated, UserId = tolower(tostring(UserId)), Operation, Parameters, ClientIP
    ) on $left.UserPrincipalName == $right.UserId
| where RuleTime between (SigninTime .. SigninTime + window)
| project SigninTime, UserPrincipalName, IPAddress, AppDisplayName, RiskEvents,
          RuleTime, Operation, Parameters, ClientIP
| order by SigninTime desc;

Hunt malicious OAuth consent grants — the "consent phishing" path agents prefer because it bypasses MFA entirely:

AuditLogs
| where TimeGenerated > ago(14d)
| where OperationName in ('Consent to application','Add service principal')
| extend AppName = tostring(TargetResources[0].displayName)
| extend ConsentScopes = tostring(AdditionalDetails)
| where ConsentScopes has_any ('Mail.Read','Mail.ReadWrite','offline_access','MailboxSettings.ReadWrite','Files.Read.All')
   or Result == 'success'
| mv-expand TargetResources
| extend InitiatedBy = tostring(parse_json(tostring(InitiatedBy.user)).userPrincipalName)
| summarize FirstSeen = min(TimeGenerated), LastSeen = max(TimeGenerated),
            ConsentCount = count(), Users = make_set(InitiatedBy, 20)
          by AppName
| order by ConsentCount desc;

For organizations ingesting email gateway or proxy logs via CEF/Syslog, hunt lookalike-domain contact — the infrastructure agent campaigns provision on demand:

CommonSecurityLog
| where TimeGenerated > ago(7d)
| where DeviceVendor in ('Proofpoint','Microsoft','Mimecast') or DeviceProduct has_any ('Exchange','Email')
| extend SenderDomain = tolower(extract(@'@([a-z0-9\.\-]+)$', 1, tostring(SourceUserID)))
| where isnotempty(SenderDomain)
// Flag domains containing your org name with lookalike mutations — adjust to your tenant
| where SenderDomain has_any ('secure-arsenal','arsenal-billing','arsena1','securityarsenaI')
   or (SenderDomain contains 'arsenal' and SenderDomain != 'securityarsenal.com')
| summarize FirstSeen = min(TimeGenerated), MessageCount = count(),
            Recipients = make_set(DestinationUserID, 25)
          by SenderDomain, SourceIP
| order by FirstSeen desc;

Velociraptor VQL

When a phishing agent succeeds, the follow-on payload often arrives mid-thread after trust is built. This hunt finds endpoints where browsers or Office processes spawned unexpected child processes — the classic "trusted conversation turned malicious" artifact:

-- Hunt for Office/browser processes spawning script interpreters or LOLBins
-- Indicator of mid-thread payload delivery after agent-driven social engineering
SELECT Pid, Ppid, Name, Exe, CommandLine, Username, CreateTime,
       get_process_details(pid=Ppid) AS Parent
FROM pslist()
WHERE Name =~ '(?i)(cmd|powershell|pwsh|wscript|cscript|mshta|rundll32|regsvr32)\.exe'
  AND (
    Parent.Exe =~ '(?i)(winword|excel|powerpnt|outlook|msedge|chrome|firefox|acrord32)'
    OR Exe =~ '(?i)\\appdata\\roaming\\microsoft\\office\\'
  )
ORDER BY CreateTime DESC

A complementary hunt for persistence planted post-compromise:

-- Enumerate recently modified Run-key persistence and user-writable startup artifacts
SELECT Name, FullPath, Size, Mtime,
       get_file_metadata(path=FullPath) AS Meta
FROM glob(globs='C:/Users/*/AppData/Roaming/Microsoft/Windows/Start Menu/Programs/Startup/*')
WHERE Mtime > now() - 86400*7
ORDER BY Mtime DESC

Remediation & Audit Script

Run this in an Exchange Online PowerShell session (after Connect-ExchangeOnline) to sweep for the persistence artifacts these campaigns create, and disable the legacy settings attackers abuse:

# Security Arsenal — BEC / Agent-Phishing Post-Compromise Audit
# Requires: Connect-ExchangeOnline -UserPrincipalName admin@yourdomain.com

$report = @()

# 1. Find inbox rules that delete, hide, forward, or redirect mail
Get-Mailbox -ResultSize Unlimited | ForEach-Object {
    $mbx = $_
    Get-InboxRule -Mailbox $mbx.UserPrincipalName -ErrorAction SilentlyContinue | ForEach-Object {
        if ($_.DeleteMessage -or $_.ForwardTo -or $_.RedirectTo -or
            $_.ForwardAsAttachmentTo -or ($_.MoveToFolder -match 'RSS|Archive|Conversation History|Deleted')) {
            $report += [PSCustomObject]@{
                Mailbox   = $mbx.UserPrincipalName
                RuleName  = $_.Name
                Delete    = $_.DeleteMessage
                ForwardTo = ($_.ForwardTo -join ';')
                Redirect  = ($_.RedirectTo -join ';')
                MoveTo    = $_.MoveToFolder
            }
        }
    }
}
$report | Export-Csv .\SuspiciousInboxRules.csv -NoTypeInformation
Write-Host "[+] Exported $($report.Count) suspicious rules to SuspiciousInboxRules.csv" -ForegroundColor Yellow

# 2. Find mailbox-level forwarding (invisible to users' rules UI)
Get-Mailbox -ResultSize Unlimited |
  Where-Object { $_.ForwardingSmtpAddress -or $_.ForwardingAddress } |
  Select-Object UserPrincipalName, ForwardingSmtpAddress, ForwardingAddress, DeliverToMailboxAndForward

# 3. BLOCK external forwarding tenant-wide (recommended hardening)
Set-HostedOutboundSpamFilterPolicy -Default -AutoForwardingMode Off

# 4. Kill legacy auth paths AiTM kits and agents abuse
Set-OrganizationConfig -OAuth2ClientProfileEnabled $true
New-AuthenticationPolicy -Name 'Block-Legacy-Auth' -AllowBasicAuthPop:$false `
    -AllowBasicAuthImap:$false -AllowBasicAuthSmtp:$false `
    -AllowBasicAuthMapi:$false -AllowBasicAuthOfflineAddressBook:$false `
    -AllowBasicAuthPowershell:$false -AllowBasicAuthWebServices:$false
Set-OrganizationConfig -DefaultAuthenticationPolicy 'Block-Legacy-Auth'

# 5. Verify unified audit logging is on (required for the detections above)
Get-AdminAuditLogConfig | Select-Object UnifiedAuditLogIngestionEnabled

Remediation & Hardening Priorities

There is no patch for this threat class. Defense is an architecture problem. Prioritize:

1. Phishing-resistant MFA, full stop. FIDO2/passkeys for all users, mandatory for executives, finance, and admins. AiTM kits and conversational agents harvest session cookies; TOTP and push MFA do not stop that. Hardware-bound passkeys do.
2. Kill OAuth consent abuse. Set Azure AD user consent to disabled or verified publishers only, route everything else through admin consent workflow, and alert on every grant with mail or file scopes.
3. Enforce DMARC at p=reject on your own domains and require it (with DKIM/SPF alignment) from partners — but understand its limit: attacker-owned lookalike domains pass DMARC legitimately. That is why display-name and cousin-domain logic must live in your detection stack, not your gateway alone.
4. Deploy intent-based email defense. This is the agent-versus-agent reality: defensive models that analyze thread context, relationship graphs, request semantics ("is a payment-account change normal for this vendor relationship?"), and conversational anomalies — not just URLs and attachments. If your email security stack is still content-signature-first, it is structurally blind to this threat.
5. Out-of-band verification for money and credentials, enforced by process, not by training alone. Any bank-detail change, payment redirection, or new payee requires callback to a known number from your directory — never one supplied in the thread. Agents are optimized to talk users out of this step.
6. Monitor the post-compromise convergence points. Inbox rules, forwarding, OAuth grants, and session anomalies are where unique lures collapse into common, detectable behavior. Wire the Sigma and KQL above into production with allowlists tuned per department.
7. Retrain for the conversational era. Update awareness programs: tell users explicitly that flawless grammar, correct internal context, and multi-turn responsiveness are no longer indicators of legitimacy. The new tell is the request itself — urgency plus secrecy plus a change to an established process.
8. Exercise it. Run adversary-simulation (BEC and AiTM scenarios, including multi-turn conversational lures) against finance and executive staff. Your red team or purple team should be testing whether an agent-grade lure gets past your controls — not whether a template phish does.

Bottom Line

The economics of phishing just inverted. Attackers no longer choose between scale and quality — agents deliver both, in every language, around the clock, with real-time conversational follow-through. Defenses built to find bad content are structurally obsolete against it. The winning posture in 2026 layers phishing-resistant authentication, consent and forwarding lockdowns, intent-based email analysis, and aggressive hunting on the post-compromise behaviors every one of these campaigns must still perform. Deploy the detections above, close the consent and legacy-auth gaps this week, and get phishing-resistant MFA on your high-value users before the next agent picks their dossier out of a scrape.

Related Resources

Security Arsenal Red Team Services AlertMonitor Platform Book a SOC Assessment pen-testing Intel Hub