CrowdStrike, Palo Alto Networks and Sophos Join SE Labs PIVOT: What the Shift Away from MITRE Engenuity Evaluations Means for Defender Procurement

A significant realignment is underway in how endpoint and threat detection vendors prove their products actually work. A group of major cyber threat detection providers — including CrowdStrike, Palo Alto Networks, and Sophos — have joined SE Labs' PIVOT program, signaling a shift away from the MITRE Engenuity ATT&CK Evaluations that have served as the industry's de facto third-party benchmark for years, according to Infosecurity Magazine.

This is not a vulnerability or an active intrusion — but for CISOs, SOC managers, and procurement teams, it is arguably just as consequential. Detection efficacy testing is the evidence layer underneath every EDR/XDR buying decision, every renewal negotiation, and every board-level claim that "our tooling would catch that." When the biggest names in the market change the yardstick they're willing to be measured by, defenders need to understand what changed, why, and how to adjust their own evaluation methodology.

What Happened

Per the reporting, multiple leading threat detection vendors have signed on to SE Labs' PIVOT program, a UK-based testing initiative from SE Labs — an independent security testing organization known for its quarterly endpoint protection reports and its emphasis on realistic, full-attack-chain testing rather than isolated technique checks.

The context that matters: MITRE Engenuity's ATT&CK Evaluations have long been the reference point vendors cite in marketing — "we detected X% of techniques in the latest MITRE eval round." Those evaluations emulate specific adversary groups (historically APT29, Wizard Spider/Sandworm, Turla, and others) and measure visibility across ATT&CK techniques. However, several dynamics have strained the model:

Participation fatigue and opt-outs. Major vendors have periodically skipped rounds, weakening the comparative value of the results.
Marketing distortion. Vendors cherry-pick technique-level results into misleading headline claims, forcing practitioners to dig into raw data to get an honest picture.
Methodology criticisms. The evaluations measure visibility and telemetry coverage more than protection outcomes — a vendor can "detect" a technique in telemetry while still allowing the intrusion to succeed.
Program evolution at MITRE. MITRE has been reshaping how its evaluations operate, and vendors appear to be hedging their third-party validation strategy.

SE Labs' PIVOT program positions itself as an alternative centered on real-world attack simulation with protection-scored outcomes — closer to SE Labs' existing methodology, where products are graded on whether they actually stop attacks, not just whether they see them.

Technical Analysis: Why the Testing Model Matters to Your SOC

From a practitioner's standpoint, the distinction between the two models is not academic — it changes what evidence you can rely on:

MITRE ATT&CK Evaluations (visibility-centric):

Emulates named APT TTPs across the attack chain.
Scores telemetry coverage: did the product generate detection data for each technique?
Results are published as raw technique-level data — valuable, but labor-intensive to interpret correctly.
Does not traditionally score whether the attack was blocked.

SE Labs approach (protection-centric):

Uses live, realistic attack chains including genuine malware and hands-on-keyboard tradecraft.
Scores outcomes on a protection/accuracy basis: was the attack prevented, detected, or allowed to complete?
Produces comparative ratings that are easier to brief upward, but historically focused on commodity threats more than APT emulation.

The practical risk for defenders: if your procurement and renewal decisions were anchored to MITRE evaluation results, and your vendors stop participating, your evidence chain breaks. You may find yourself in a 2026 renewal cycle where the data you used to justify the platform three years ago no longer has a current equivalent.

Executive Takeaways

Diversify your third-party validation sources now. Do not let any single testing program — MITRE or SE Labs — be your sole efficacy reference. Track SE Labs PIVOT results, remaining MITRE evaluation participants, AV-Comparatives, and AV-TEST in parallel, and reconcile discrepancies before renewals.
Interrogate what each test actually measures. When a vendor cites a testing result in a sales cycle, ask specifically: was this a visibility/telemetry score or a protection/blocking score? Were detections out-of-the-box or post-configuration? Demand the raw methodology, not the marketing summary.
Run your own adversary emulation against your own stack. Third-party tests are generic by nature. Use purple team exercises and tools like Atomic Red Team or CALDERA to emulate the threat actors that actually target your sector, against your tenant, with your configurations. Your own detection coverage data beats any industry benchmark.
Update procurement criteria and contract language. If your RFPs reference MITRE evaluation participation as a requirement, revise them to accept multiple accredited testing frameworks — otherwise you may inadvertently exclude vendors who moved to PIVOT or lock yourself into stale criteria.
Watch for marketing spin during the transition. Expect vendors to frame the shift as "better testing." Some of that may be true — protection-scored outcomes are genuinely useful — but treat every claim as unverified until you've seen the underlying methodology and raw results.
Brief leadership on the change. Boards and executives have absorbed "MITRE ATT&CK" as a trust signal over the past five years. Proactively explain that the benchmark landscape is fragmenting, what you're doing about it, and why your evaluation methodology is now multi-sourced.

What Defenders Should Do Now

Inventory your validation dependencies. Identify where MITRE evaluation results appear in your vendor selection records, risk assessments, and compliance documentation (particularly for NIST CSF and CIS Control 8 audit narratives).
Request PIVOT participation details from your current EDR/XDR vendors during your next QBR — ask whether they joined, what scope was tested, and when results will be public.
Baseline your current detection coverage against MITRE ATT&CK using your own telemetry (technique coverage mapping from your SIEM/EDR), so you have an internal benchmark that survives any vendor testing politics.
Monitor both programs' 2026 results and compare vendors that participate in each — divergence between visibility scores and protection scores is itself actionable intelligence about product gaps.

The testing ecosystem is consolidating around outcome-based proof. That is ultimately good for defenders — but only if you adapt your evaluation process before your next renewal cycle, not after.

Related Resources

Security Arsenal Managed SOC Services AlertMonitor Platform Book a SOC Assessment soc-mdr Intel Hub