Beyond the Model: Securing Healthcare's New AI Workforce

MIAMI, FL – December 10, 2025 – As healthcare organizations rush to deploy artificial intelligence, a new and powerful workforce is emerging: autonomous AI agents. These systems promise to revolutionize everything from patient scheduling and clinical trial data analysis to revenue cycle management. Yet, as they are woven into the fabric of hospital operations—connecting to EHRs, billing systems, and patient databases—they create a complex and largely invisible attack surface that traditional security measures were not designed to see.

Most AI security to date has focused on the Large Language Model (LLM) itself, testing it in a sterile, isolated environment. This approach, however, misses the most critical point of failure. The real danger isn't just a flawed model; it's a perfectly good model given the keys to the kingdom and tricked into misusing them. Security teams are discovering that once an AI agent is integrated with internal APIs, databases, and third-party tools, its potential for harm multiplies. This interconnectedness creates a maze of potential attack paths that static model testing completely overlooks.

This gap in visibility is the challenge that a new generation of security tools is scrambling to address. Miami-based Pillar Security recently entered the fray with RedGraph, a platform it bills as the industry’s first to continuously map and test the attack surface of AI agents in their live, operational environments. The launch signals a critical shift in the AI security paradigm: from protecting the AI's brain to securing its entire nervous system.

The Invisible Threat of 'Excessive Agency'

The fundamental security challenge with AI agents lies in what experts are calling 'excessive agency'—the risk of an agent performing unintended actions that go beyond its designated purpose. A staggering 80% of organizations already using AI agents report that their systems have performed such actions, from accessing unauthorized data to interacting with incorrect systems.

These agents operate with no inherent sense of loyalty or secrecy. They can be manipulated through sophisticated 'prompt injection' attacks, a vulnerability the OWASP Top 10 for LLMs lists as the number one threat. An attacker could, for example, embed a malicious instruction in a document that an AI agent is tasked with summarizing. When the agent processes the document, it executes the hidden command—perhaps instructing it to query a patient database for sensitive information and exfiltrate it through a connected, seemingly benign tool.

“Most AI security testing happens in a vacuum,” said Dor Sarig, CEO & Co-Founder of Pillar Security, in the company’s announcement. “Teams test the model but miss the system. RedGraph changes this by taking the attacker's perspective - it thinks like a hacker, maps the entire attack surface, and validates every finding with a real exploit.”

This attacker-centric view is crucial. Instead of just running theoretical checks, this new approach involves deploying autonomous adversarial agents to actively probe for weaknesses. If a path is blocked, the testing agent pivots and tries another, mimicking the persistent, adaptive behavior of a real human attacker but operating at machine speed. This allows it to uncover not just AI-specific flaws but also traditional vulnerabilities like Cross-Site Scripting (XSS) that manifest in new ways through agentic systems.

From Attack Path Mapping to Active Defense

What sets platforms like RedGraph apart is the focus on visualizing the entire chain of connections. By representing an organization's AI estate as a graph—with nodes for agents, tools, databases, and permissions, and edges representing the relationships between them—security teams can finally see how a single vulnerability could cascade into a major breach. This 'Graph-First Attack-Path View' illuminates unintended relationships and highlights where risk truly accumulates.

This approach is already providing value in complex enterprise environments. “We have numerous AI initiatives throughout the company,” noted Tomer Maman, CISO at digital intelligence firm Similarweb, a customer of Pillar. “Unlike traditional red teaming, RedGraph continuously validates vulnerabilities in our AI agents' attack surface in production, providing complete attack paths that the engineering team can fix immediately.”

The ultimate goal is not just discovery but remediation. The insights gained from continuous testing are used to create and adapt security guardrails in real-time. If the testing agent discovers that an AI assistant tasked with scheduling can also access and render sensitive financial reports, a rule can be instantly deployed to block that unintended capability. This creates a closed-loop system where offense informs defense, allowing an organization's security posture to evolve in lockstep with its AI systems and the threats targeting them.

Meeting the Compliance Challenge in Healthcare

For healthcare organizations, the stakes are exceptionally high. A compromised AI agent could lead to a catastrophic breach of protected health information (PHI), manipulation of clinical data, or disruption of critical hospital operations. The regulatory landscape is also solidifying, placing a greater burden of proof on organizations to demonstrate the security and resilience of their AI systems.

Frameworks like the EU AI Act, which came into force this year, mandate that high-risk AI systems—a category that will undoubtedly include many healthcare applications—be designed to be resistant to third-party attempts to exploit system vulnerabilities. Similarly, the NIST AI Risk Management Framework emphasizes a holistic approach to governance and security. Simply stating that a model passed a pre-deployment test will no longer be sufficient.

Organizations will need to provide continuous, auditable evidence that they are actively managing the risks across their entire integrated AI ecosystem. Tools that provide attack surface mapping and validated exploit paths offer a concrete way to meet this requirement. They transform AI security from a theoretical exercise into a demonstrable, data-driven practice, providing the assurance that regulators, patients, and boards of directors will increasingly demand.

As AI agents become more autonomous and more deeply integrated into clinical and administrative workflows, the ability to see and secure their every interaction will become a non-negotiable aspect of healthcare cybersecurity. The race is on to build defenses that can keep pace with the very same technology they are designed to protect.