Securing Clinical AI: CHOC Children's Adoption — Governance and Integrity Controls

Introduction

The healthcare sector is witnessing a pivotal shift in how clinical workflow technology is deployed. At Children's Hospital of Orange County (CHOC Children's), part of the Rady Children's Health system, clinicians are leveraging new Artificial Intelligence (AI) tools to alleviate the administrative burdens imposed by Electronic Health Records (EHR). While Dr. Steven Martel and his team aim to reverse the trend of provider burnout, security leaders must recognize that integrating Ambient Clinical Intelligence (ACI) and Generative AI into the care continuum introduces a new attack surface.

The urgency for defenders lies in the convergence of sensitive Protected Health Information (PHI) and third-party AI processing. The risks are twofold: data exfiltration via insecure AI pipelines and integrity failures where AI "hallucinations" alter clinical documentation. Security teams must act now to establish guardrails before these tools become ubiquitous across the enterprise.

Technical Analysis

While this news item highlights a technological adoption rather than a specific CVE exploitation, the deployment of AI in clinical environments introduces specific architectural risks that must be analyzed:

• Affected Components: Ambient Clinical Intelligence (ACI) devices, Generative AI Large Language Models (LLMs), and EHR integration APIs (e.g., Epic, Cerner).
• Attack Vector (Data Privacy): AI tools often require audio or text inputs to be processed by cloud-hosted models. If not strictly configured, patient identifiers (PID) or clinical conversations could be logged by the vendor for training purposes, violating HIPAA minimum necessary standards and potentially exposing data in a vendor breach.
• Attack Vector (Prompt Injection): Malicious actors or compromised insiders could manipulate input prompts to cause the AI to generate incorrect medical orders (Suggestibility Attacks) or to ignore safety guardrails (Jailbreaking), leading to downstream patient safety incidents.
• Supply Chain Risk: Reliance on third-party AI models (e.g., Nuance, Microsoft Azure OpenAI) introduces supply chain risks. A compromise in the model update pipeline could serve malicious logic to hospital endpoints.
• Exploitation Status: While no specific active exploit against CHOC is disclosed, the rapid, often shadow-IT driven adoption of these tools creates a high risk of "Shadow AI," where clinicians use unapproved tools to process PHI, bypassing DLP controls.

Executive Takeaways

Given the operational nature of this news, defensive priorities focus on governance and data protection rather than specific exploit signatures.

1. Enforce Strict Data Residency and Logging: Ensure that your AI vendor agreements explicitly forbid the use of PHI for model training or retraining. Demand granular audit logs of every AI interaction (input prompt, generated output, and user context) to support forensic investigations.

2. Implement Human-in-the-Loop (HITL) Verification: AI-generated clinical notes or orders must be treated as "suggested" rather than "final." Security policies should mandate a manual review and attestation step within the EHR workflow before AI-suggested content is committed to the permanent medical record.

3. Conduct Vendor Risk Management (VRM) for AI Models: Treat AI providers like Business Associates (BAs). Require current BAAs, conduct third-party risk assessments (e.g., HITRUST CSF certification), and verify that their data retention policies align with your organization's data lifecycle requirements.

4. Deploy Network Segmentation for AI Appliances: If physical AI devices (listening walls/devices) are deployed, place them on a dedicated, isolated VLAN. Restrict egress traffic strictly to known, whitelisted AI processing endpoints (IPs/FQDNs) to prevent data tunneling or C2 activity.

5. Establish Acceptable Use Policies (AUP): Explicitly prohibit the use of public, consumer-grade Generative AI tools (e.g., ChatGPT, Bard) for processing patient data. Reinforce this with DLP rules that detect and block the upload of medical terminology or PID to non-approved external domains.

Remediation

To secure the adoption of clinical AI tools similar to those deployed at CHOC, execute the following defensive steps:

1. Update Information Security Policy: Revise the Acceptable Use Policy to include specific clauses on Generative AI and Ambient Intelligence usage. Define authorized tools and explicitly prohibit the input of PHI into public models.

2. Verify Vendor Contracting: Immediately review contracts with any AI/EHR integration vendors. Ensure they sign a HIPAA Business Associate Agreement (BAA) and include a "Right to Audit" clause. Confirm that data is encrypted in transit (TLS 1.3) and at rest (AES-256).

3. Configure DLP for GenAI: Update Data Loss Prevention (DLP) rules to fingerprint standard medical forms and patient ID formats. Block attempts to copy/paste or upload this data to unapproved AI domains.

4. Audit EHR Integration Points: Review logs within the EHR (e.g., Epic Bridges or Cerner PowerNotes) to identify any unauthorized API connections attempting to push or pull data using OAuth tokens associated with AI tools.

5. Tabletop Exercises: Conduct a tabletop exercise simulating an AI integrity event (e.g., an AI tool incorrectly suggesting a dosage change). Ensure the IR plan includes clinical workflows to recall erroneous orders and technical workflows to disable the AI integration point.

Related Resources

Security Arsenal Healthcare Cybersecurity AlertMonitor Platform Book a SOC Assessment healthcare Intel Hub