AI Social Engineering: Automating SOC Triage Against High-Volume Phishing

Introduction

We have officially entered the era of high-velocity social engineering. As reported in recent industry news, AI has transformed social engineering from a manual, labor-intensive "numbers game" into an automated volume machine. Attackers are leveraging Large Language Models (LLMs) to generate convincing emails, clone legitimate login pages, and tailor lures to specific organizations in minutes.

For Security Operations Centers (SOCs), this represents a critical tipping point. Every polished AI-generated message adds a new case to the Tier 1 queue. As these queues swell, the risk of alert fatigue increases exponentially, creating the perfect cover for credential theft or malicious software delivery to slip through the cracks. Defenders cannot simply "hire more analysts"; they must automate the identification and disposition of this noise to focus on genuine threats.

Technical Analysis

The Threat Vector: AI-Augmented Phishing and Business Email Compromise (BEC)

Unlike traditional phishing campaigns, which often suffer from poor grammar, formatting inconsistencies, and obvious translation errors, AI-generated content eliminates these heuristic red flags. The attack chain typically involves:

Automated Reconnaissance: Attackers scrape LinkedIn, company websites, and press releases to feed context into AI models.
Content Generation: LLMs generate context-aware emails mimicking internal communication styles (e.g., HR policy changes, urgent invoice payments).
Infrastructure at Scale: Automated scripts spin up unique subdomains and credential harvesting pages, evading traditional hash-based blocklists.
Volume Delivery: Campaigns are delivered in massive bursts, intending to saturate SOC alerting and user-reporting channels.

Affected Platforms & Systems:

Email Gateways: Microsoft 365 Defender, Google Workspace, Mimecast, Proofpoint.
Endpoints: Windows, macOS, Linux (user workstations).
Identity Providers: Active Directory Federation Services (ADFS), Okta, Entra ID.

The Risk: The primary danger is not the complexity of a single email, but the sheer volume. Tier 1 analysts are tasked with inspecting links and attachments for alerts that "cannot be dismissed at a glance." This manual validation is a failure point; a sophisticated credential harvesting attempt buried in a queue of 500 other AI-generated spam is likely to be mishandled.

Detection & Response

To combat this volume, SOCs must shift from manual review to behavioral hunting. We need to detect the interaction with these threats rather than just the delivery. Below are detection mechanisms focused on the post-click behavior—when a user interacts with a lure—triggering high-fidelity alerts that bypass the noise of the inbox.

Sigma Rules

These rules focus on suspicious child processes spawned from mail clients or browsers, which often indicates a user has clicked a malicious link leading to a credential harvester or malware downloader.

YAML

---
title: Suspicious Process Spawned by Mail Client
id: a8b9c0d1-e2f3-4a5b-8c6d-7e8f9a0b1c2d
status: experimental
description: Detects suspicious command-line utilities or script hosts spawned from common mail clients. This indicates a potential phishing link execution or malware delivery.
references:
  - https://attack.mitre.org/techniques/T1566/
author: Security Arsenal
date: 2026/06/15
tags:
  - attack.initial_access
  - attack.t1566
logsource:
  category: process_creation
  product: windows
detection:
  selection_parent:
    ParentImage|contains:
      - '\OUTLOOK.EXE'
      - '\thunderbird.exe'
  selection_child:
    Image|endswith:
      - '\powershell.exe'
      - '\cmd.exe'
      - '\mshta.exe'
      - '\wscript.exe'
      - '\cscript.exe'
  condition: selection_parent and selection_child
falsepositives:
  - Legitimate automation tools interacting with Outlook
level: high
---
title: Browser Spawned Network Process to Suspicious Directory
id: b2c3d4e5-f6a7-4b8c-9d0e-1f2a3b4c5d6e
status: experimental
description: Detects browser processes spawning network tools (like curl) or scripts, common in web-based credential theft or drive-by downloads.
references:
  - https://attack.mitre.org/techniques/T1059/
author: Security Arsenal
date: 2026/06/15
tags:
  - attack.execution
  - attack.t1059.003
logsource:
  category: process_creation
  product: windows
detection:
  selection_parent:
    ParentImage|contains:
      - '\chrome.exe'
      - '\msedge.exe'
      - '\firefox.exe'
  selection_child:
    Image|endswith:
      - '\powershell.exe'
      - '\curl.exe'
      - '\wget.exe'
  selection_path:
    CurrentDirectory|contains:
      - '\Downloads'
      - '\AppData\Local\Temp'
  condition: selection_parent and selection_child and selection_path
falsepositives:
  - Legitimate web-based tools used by developers
level: medium


**KQL (Microsoft Sentinel / Defender)**

Hunt for users accessing login pages on recently registered domains. AI phishing campaigns often use domains registered within the last 24-48 hours.

KQL — Microsoft Sentinel / Defender

// Hunt for connections to domains registered within the last 30 days
// Requires enrichment via ThreatIntelligenceIndicator or Whois data
let suspiciousTlds = dynamic([".xyz", ".top", ".gq", ".tk", ".ml"]);
DeviceNetworkEvents
| where Timestamp > ago(1d)
| where RemoteUrl has_any ("login", "signin", "account", "verify", "update")
| where UrlOriginal matches regex @"https?://[a-zA-Z0-9-]+\.(" .. string_cat(suspiciousTlds, "|") .. ")"
| where ActionType in ("ConnectionAllowed", "ConnectionSuccess")
| summarize Count = count(), DistinctURLs = dcount(RemoteUrl) by DeviceName, InitiatingProcessFileName, AccountName
| where Count > 3 // Threshold tuning required per environment
| project DeviceName, AccountName, InitiatingProcessFileName, Count, DistinctURLs


**Velociraptor VQL**

Hunt endpoint browser history for login-related keywords on non-corporate domains, indicative of credential harvesting.

VQL — Velociraptor

-- Hunt for browser history entries containing login keywords
-- on suspicious top-level domains or IP addresses
SELECT 
    Timestamp,
    URL,
    Title,
    Browser,
    Username
FROM chrome_history()
WHERE 
   URL =~ 'login' OR 
   URL =~ 'signin' OR 
   URL =~ 'password' OR 
   URL =~ 'verify' OR 
   URL =~ 'account'
   AND NOT URL =~ 'securityarsenal.com' -- Exclude corporate domains
   AND (URL =~ 'http://' AND NOT URL =~ 'https://') -- Look for non-HTTPS login pages (high risk)
LIMIT 100


**Remediation Script (PowerShell)**

This script configures Microsoft 365 (Exchange Online) Safe Links and Anti-Phishing policies to automatically track and rewrite URLs, reducing the need for Tier 1 manual link inspection. Note: Requires Exchange Online Management Module.

PowerShell

# Ensure Connect-ExchangeOnline is run before executing

# 1. Enable Safe Links for all users (TrackClicks = true)
$safeLinksPolicyName = "Security Arsenal Global SafeLinks"

try {
    $policy = Get-SafeLinksPolicy -Identity $safeLinksPolicyName -ErrorAction Stop
    Write-Host "Updating existing Safe Links Policy..."
    Set-SafeLinksPolicy -Identity $safeLinksPolicyName -EnableSafeLinksForEmail $true -EnableSafeLinksForTeams $true -TrackClicks $true -AllowClickThrough $false
} catch {
    Write-Host "Creating new Safe Links Policy..."
    New-SafeLinksPolicy -Name $safeLinksPolicyName -EnableSafeLinksForEmail $true -EnableSafeLinksForTeams $true -TrackClicks $true -AllowClickThrough $false
}

# Ensure the rule is enabled
try {
    $rule = Get-SafeLinksRule -Identity $safeLinksPolicyName -ErrorAction Stop
    Set-SafeLinksRule -Identity $safeLinksPolicyName -Enabled $true -SafeLinksPolicy $safeLinksPolicyName
} catch {
    New-SafeLinksRule -Name $safeLinksPolicyName -SafeLinksPolicy $safeLinksPolicyName -Enabled $true -RecipientDomainIs (Get-AcceptedDomain).Name
}

Write-Host "Safe Links configured to rewrite URLs automatically, reducing Tier 1 manual inspection load."

Remediation

Configure Automated URL Rewriting: Ensure your Secure Email Gateway (SEG) or native cloud provider (M365/Google) has "Safe Links" or "URL Rewriting" enabled. This forces all links to be scanned at click-time, providing a safety net that allows Tier 1 analysts to dismiss alerts faster.
Implement SOAR Playbooks for Triage: Deploy a playbook (e.g., in Cortex XSOAR, Splunk SOAR, or Microsoft Sentinel) that automatically:
- Parses the body of reported emails.
- Extracts URLs and submits them to sandbox detonation.
- Checks URLs against threat intelligence feeds (VirusTotal, URLHaus).
- Auto-closes the ticket if the verdict is "Clean" (with user notification).
Hardening Authentication: Since the end goal of AI phishing is credential theft, enforce phishing-resistant MFA (FIDO2/WebAuthn). This prevents the impact of a successful credential harvest.
DMARC Enforcement: Move your organization's DMARC policy from p=none to p=reject or p=quarantine immediately. This stops AI-generated spoofed emails from reaching the inbox in the first place.

Related Resources

Security Arsenal Penetration Testing Services AlertMonitor Platform Book a SOC Assessment vulnerability-management Intel Hub