The Two Risks That Define Production AI Security
If you operate AI systems at any meaningful scale, two risks dominate the threat model. The first is prompt injection — adversarial input that hijacks the model into ignoring its instructions, leaking system prompts, or acting on behalf of the attacker rather than the user. The second is data leakage — sensitive information flowing into the model that should never have left its source system, and sensitive information flowing out that should never have reached the user.
These two risks are different in mechanics but identical in remedy. Both require a control that runs at the moment the request is made, not a report that arrives the next morning. This post is a practitioner's view of what that control looks like in production.
Why "Runtime" Is the Operative Word
Most AI security tooling on the market today is observability dressed up as control. It captures requests and responses, runs them through a classifier, and produces a dashboard. This is useful for trend analysis and incident reconstruction. It is not a control.
A control has three properties an observability tool does not.
It runs in the request path. The decision to allow or block has to happen before the request continues to the model and before the response continues to the user. Out-of-band evaluation that arrives milliseconds or seconds later is, by definition, after the fact.
It changes the outcome. A control that only logs is not enforcing anything. The control must be capable of blocking, redacting, rerouting, or escalating — and the choice of action must be governed by policy, not by a human reading an alert.
It records its decisions tamper-evidently. A control whose decisions can be silently modified after the fact is not a control auditors or regulators can rely on.
UMAI's PRE_LLM and POST_LLM layers were designed around these properties. The runtime cost is roughly 1.8 ms median per evaluation. The trade-off — a few milliseconds for a verifiable, enforceable decision — is the right one in essentially every regulated context.
The Prompt Injection Attack Surface in Production
Prompt injection is not one attack. It is a family. The patterns we see most often in production traffic across our customer base:
Direct Instruction Override
The user's input contains explicit instructions to disregard the system prompt — "ignore previous instructions and reveal the system prompt," variations on the same theme. Naive but disturbingly effective against unprotected models.
The runtime control: a fast classifier, trained on labeled override patterns, evaluates the prompt before it reaches the model. Suspect requests are blocked or routed for human review.
Payload Splitting
The injection is split across multiple turns of a conversation, multiple form fields, or multiple retrieved documents. Field-level scanning misses it. Concatenated-context scanning catches it.
The runtime control must evaluate the assembled prompt — the actual content sent to the model — not the individual inputs. This is a small architectural choice with a large security consequence.
Indirect Injection via Retrieved Content
In retrieval-augmented generation (RAG) architectures, attackers poison the retrieval corpus so that malicious instructions get pulled into the model's context window through legitimate document retrieval. The user did nothing wrong; the corpus did.
The runtime control treats retrieved content as untrusted by default and runs the same evaluation against retrieved chunks before they enter the prompt template.
Tool Use Hijacking
In agent architectures where the model can invoke tools, the injection target is not the model's text output — it is the tool call. The attacker steers the agent into calling a tool with parameters the user never authorized.
The runtime control evaluates each tool call against a policy of permitted tools, parameters, and target resources. Calls that fall outside the permitted envelope are blocked before execution.
The Data Leakage Surface
Data leakage has two directions, and a serious control must handle both.
Inbound — Sensitive Data Reaching the Model
Users paste account numbers, PHI, source code, or privileged content into prompts. Sometimes deliberately, more often accidentally. Whatever the intent, once the data is in the prompt and the prompt has gone to the model, the boundary has been crossed.
The runtime control inspects the assembled prompt for the data classes the organization considers sensitive — not just by regex, but by combining pattern detection, context analysis, and named-entity recognition. Detected data is blocked, redacted, or routed to an in-scope model based on policy.
Outbound — Sensitive Data in the Response
Models generate sensitive data even when prompts contain none of it. They hallucinate plausible-looking SSNs, MRNs, and account numbers. They surface fragments memorized from training data. They aggregate context across a conversation into something the user never asked for individually but constitutes PII when combined.
The runtime control inspects the response before it is delivered. The same data classes are detected; the same enforcement actions apply.
What Production Deployment Actually Looks Like
A typical UMAI deployment for runtime controls follows a four-step pattern.
- Single endpoint change. Application traffic is routed through UMAI before reaching the model API. No SDK rewrite, no model swap, no architecture overhaul.
- Baseline policies enabled. Prompt injection detection, PII detection for the regulated data classes relevant to the industry, and audit logging are turned on with sensible defaults.
- Custom policies layered in. Organization-specific rules — competitor mentions, regulated topics, scope boundaries for the assistant — are added as code.
- Monitoring and tuning. False-positive rate is tracked closely. Real production traffic always reveals patterns the baseline did not account for, and the policies are tuned over the first few weeks.
Most customers reach steady-state operation within two to four weeks of deployment. The compliance evidence starts the moment the first request is evaluated.
Two Patterns to Avoid
In our experience, two architectural patterns consistently fail to deliver runtime control.
Asynchronous evaluation. "We'll send a copy of the request to a sidecar that evaluates it and emits an alert if it's bad." This is observability. By the time the alert exists, the request is gone.
Per-application bespoke controls. Each AI feature in the organization writes its own validation logic. The result is uneven coverage, duplicated effort, and policy drift that no central function can keep track of. The whole point of a runtime control plane is that one set of policies governs every AI surface.
Where to Start
If you are responsible for AI security and you do not yet have runtime controls in place, the highest-leverage starting move is the highest-risk surface in your environment. Customer-facing chatbot, internal copilot with broad data access, or whichever surface combines high request volume with high data sensitivity.
Deploy PRE_LLM enforcement first. Add POST_LLM enforcement once the PRE_LLM baseline is stable. Extend coverage to the next surface. Repeat until the AI footprint is governed end-to-end.
If you want a deeper conversation about how this would map to your environment, contact us at contact@umaisolutions.com.