AI Security: The New Attack Surface
AI systems introduce attack techniques that traditional application security was never built to catch — prompt injection, data poisoning, and model manipulation among them.
The business problem
Traditional application security assumes a relatively predictable input surface — form fields, API parameters, file uploads — that can be validated with known techniques. AI systems, especially those built on large language models, accept natural language as input, which opens an attack surface that conventional input validation was never designed to handle.
The AI-specific risks worth knowing
- Prompt injection — an attacker embeds instructions inside content the AI system processes (an email, a document, a webpage) to hijack its behavior, sometimes without the end user ever seeing anything unusual
- Data poisoning — corrupting the data a model is trained or fine-tuned on, so its output is subtly wrong or manipulated in ways that are hard to detect after the fact
- Model extraction and inversion — an attacker probing a model’s outputs to reconstruct its training data or replicate its behavior, potentially exposing sensitive information the model was trained on
- Excessive agency — an AI agent given more permission to take action (send emails, execute code, move money) than its actual reliability warrants
“An AI system that can read untrusted content and take action is, from a security standpoint, an application with a very large, very persuadable input field.”
Why it matters
As organizations move from AI that only answers questions to AI that takes action — sending emails, executing code, querying internal systems — the consequence of a successful prompt injection or excessive-agency failure grows accordingly. Security teams that treat an AI integration like any other application without accounting for these AI-specific risks are leaving a real gap.
Practical guidance
Treat any content an AI system processes from an untrusted source (an inbound email, a scraped webpage, a user-uploaded document) as potentially containing hidden instructions. Scope AI agents’ permissions to the minimum needed for their task, the same least-privilege principle applied to human accounts. Log and review AI system actions, especially ones with real-world effect, so anomalies are catchable after the fact.
See how AI-specific risks fit into a broader cyber risk program.
Explore Cyber Risk AdvisoryQuestions, answered directly.
It’s a way of hiding instructions inside content an AI system will read — an email, a webpage, a document — so the AI follows the attacker’s instructions instead of (or in addition to) the user’s.
No — that’s what makes it distinct. The attacker only needs to get content in front of the AI system through a channel it already processes, such as an inbound email or a webpage it’s asked to summarize.
It means the agent has been given more ability to take independent action — send messages, run code, move data — than its current reliability and oversight can safely support for that specific task.
It adds a new set of risks specific to how these systems process language and take action, on top of — not instead of — every traditional security control (access management, logging, vendor risk) that still applies.
Talk to a CISO about securing your AI systems.
30 minutes. No obligation. No sales pitch.