The Near-Zero Cost of Secure AI

There's a common assumption baked into most enterprise AI discussions: security and performance are in tension. Add security controls, pay a performance penalty. In regulated industries — banking, insurance, financial services — this assumption often slows adoption or produces systems where security is bolted on as an afterthought rather than designed in.

After building AI-based workflow automation for clients in regulated industries, we've stress-tested this assumption. The honest conclusion: security layers have a real but manageable cost, and for most architectures that cost is small enough to be a non-issue — provided the layers are designed correctly from the start.

This post covers four specific layers — tool call verification, serverless isolation, PII redaction, and self-hosted visual extraction — with benchmarks where we have them.

The threat model

Before arguing about cost, it helps to be precise about what we're defending against.

AI systems in regulated contexts typically process untrusted input: customer emails, scanned documents, uploaded forms. An attacker who can influence that input can attempt to manipulate the system's behavior — redirecting tool calls, exfiltrating data, or causing the workflow to act outside its intended scope. This is not theoretical; it's the central threat of prompt injection, and it's well-documented in systems where AI has direct access to tools with real-world effects.

The four layers below address this threat at different points in the stack.

Layer 1: A verification layer between AI output and tool execution

The simplest architecture lets the AI emit a tool call and executes it directly. This is also the most vulnerable: if an attacker can manipulate the system's output — through a prompt injection in an uploaded document, form, or message — they get direct write access to whatever the tool can do.

The alternative is to treat the AI's output as proposed intent, not as an instruction. A deterministic verification layer sits between the workflow and the executor:

AI output (structured JSON)
                ↓
        Deterministic verifier:
          - Schema validation
          - State machine check (valid action at this workflow stage?)
          - Parameter sanity (amount within range, target in whitelist)
          - Permission/role enforcement
          - Audit log write
                ↓
        Tool executor

The key design decision: the verifier must be deterministic, not another AI call. A second model can be manipulated just as the first can. A rule-based validator cannot be socially engineered.

What does this cost? Schema validation and state machine checks run in microseconds. The overhead of a well-implemented verification layer is typically in the range of 1–5ms — not a rounding error on a 2-second inference call, but not a meaningful penalty either.

The more relevant cost comparison is the other direction: what does a missing verifier cost when a prompt injection succeeds? In any regulated context, the answer involves incident response, regulatory notification, and potentially material client impact. The verifier is cheap insurance.

External benchmarks worth citing:

• AgentDojo (ETH Zurich, 2024) provides concrete attack success rates against direct tool-calling systems vs. those with defensive layers.
• InjecAgent benchmarks indirect prompt injection in tool-calling workflows specifically.

Both show meaningful attack success rate reductions with even basic defensive architecture in place.

Layer 2: Serverless isolation (Cloud Run vs. threads on a VM)

For AI workloads that process untrusted content, the execution environment matters as much as the code itself.

On a single VM with threads or containers, all workloads share process memory, environment variables and credentials, filesystem state, and any long-lived network connections. An attacker who achieves code execution in one context can potentially pivot to others running on the same VM. Worse, a compromised process on a long-lived VM can establish persistence — a foothold that survives across requests.

Serverless functions (Cloud Run, Lambda, etc.) change this calculus:

• Each invocation gets an ephemeral execution context. Compromise one request, and the damage is bounded to that request's context and its IAM scope.
• No long-lived process means no persistent footholds.
• Per-service IAM roles allow minimal privilege assignment — each function gets exactly the permissions it needs and nothing more.
• Audit logging is native and per-invocation.

What does this cost? Cold starts on Cloud Run are approximately 200–400ms for a typical Python container. For async, batch-style document processing, this is usually irrelevant — the workflow latency is measured in seconds, not milliseconds. A min-instances=1 configuration keeps at least one warm instance available and eliminates cold start latency on the critical path entirely.

Compute cost comparison is often favorable. An always-on VM that's 30% utilized overnight is paying for idle compute that Cloud Run doesn't bill for. For document processing workflows with predictable peaks, serverless can be genuinely cheaper than a warm VM.

The honest tradeoff: Serverless gives weaker network egress control than a VM with full iptables access. Routing Cloud Run through a VPC connector adds operational complexity. For environments where granular egress control is a compliance requirement, this needs careful architecture rather than a default assumption.

Layer 3: PII redaction at ingestion

The security argument for PII redaction is straightforward: data that never enters the AI's context cannot be leaked, logged, or exfiltrated through the model. For workflows processing customer correspondence — across financial services, healthcare, insurance, or government — this typically means names, account numbers, ID numbers, email addresses, and phone numbers.

The performance question is more interesting. We ran a benchmark across three model tiers on a regulated-industry email classification task, comparing raw (baseline) vs. Presidio-anonymized inputs. Results across 10 runs per condition:

gpt-5.4

Token delta: −213 (−2.6%)  |  Latency delta: +0.32s (+16.3%)  |  Accuracy delta: 0

gpt-5.4-mini

Token delta: −213 (−2.6%)  |  Latency delta: −0.08s (−7.8%)  |  Accuracy delta: 0

gpt-5.4-nano

Token delta: −213 (−2.6%)  |  Latency delta: −0.06s (−3.8%)  |  Accuracy delta: −0.20

Reading the results honestly

The token reduction is consistent and small across all conditions — roughly 2.6%. This makes sense: PII tokens are a small fraction of a typical customer document, and normalized replacement codes (PERSON_1, ORG_1) are slightly shorter than the names they replace.

For gpt-5.4 and gpt-5.4-mini, accuracy is unchanged at 1.0. These models are robust to anonymization — the task-relevant signal (intent, category, required action) survives redaction intact. The latency numbers show some noise at n=10, but there is no systematic penalty on either tier.

For gpt-5.4-nano, the accuracy drop is real and worth being direct about: 0.70 → 0.50. Smaller models rely more heavily on named entities for contextual grounding, and anonymization removes those anchors. The practical implication is clear: nano-tier models should not be used for anonymized classification tasks in regulated workflows. Mini and above handle it without degradation.

This is a small internal benchmark and should be treated as directional. But the direction is unambiguous for production-grade model tiers.

Operational overhead of Presidio: Adding Microsoft Presidio to a processing pipeline typically adds 10–30ms per document for standard PII entity types. At document-processing scale, this is not a bottleneck.

Layer 4: Self-hosted visual extraction

Regulated workflows routinely involve documents that arrive as scans, photos, or PDFs — contracts, identity documents, account statements, clinical records, correspondence. Extracting structured information from these has traditionally meant sending them to a commercial vision API.

The security problem with this approach is fundamental: the document leaves your infrastructure. A customer's scanned ID or financial statement transiting to a third-party API is a data processor relationship that requires contractual coverage, regulatory disclosure, and in some jurisdictions (GDPR, India's DPDP Act, MAS outsourcing guidelines, HIPAA) specific controls and approvals.

Self-hosted open-source visual extraction eliminates this entirely. A stronger 2026 default is a two-model stack:

• Qwen-3.6-VL as the primary document parser: OCR, layout extraction, table understanding, and multilingual scanned inputs.
• Gemma-4 as the reasoning and normalization layer after extraction: field reconciliation, schema repair, confidence scoring, and escalation tagging.

This split is useful in production. Qwen handles vision-heavy work; Gemma handles deterministic post-processing logic and audit-friendly summaries. In practice, this gives better reliability than forcing a single model to do both perception and workflow reasoning.

The benchmark bears this out. Across 100 ID and banking documents, gemma-4-26B-A4B running as a single-model stack — responsible for both reading/extracting and reasoning over the document — correctly handled 91% of the cases that Gemini 3 Flash got right. A real gap. But when the two responsibilities were separated, with gemma-4-26B-A4B handling only extraction and Gemini 3 Flash handling reasoning on the extracted output, the performance gap disappeared entirely.

The implication: the local model's limitation isn't perception — it's reasoning under complexity. Extraction is a retrieval task; it maps well to a local model's strengths. Reasoning about what to do with extracted fields is where frontier models earn their place. Splitting along that line gives you the data residency guarantee on the sensitive raw document and Gemini 3 Flash-quality accuracy on the workflow decisions that follow.

The cost argument here inverts. Commercial vision APIs charge per page or per image. Self-hosted inference amortizes over volume. For workflows processing thousands of documents per day, self-hosted is typically 50–70% cheaper than commercial API pricing — while also eliminating the data egress risk entirely.

Latency is still comparable to commercial APIs: a Qwen-3.6-VL extraction pass is typically in the same 1-3 second range per scanned page on modern GPU instances, and a Gemma-4 post-pass is usually sub-second for structured validation and routing.

The honest cost: GPU infrastructure has real operational overhead — deployment, monitoring, model updates, failover. For a team without existing MLOps capability, this is not free. The practical threshold: if you're processing more than a few hundred documents per day and the data residency concern is genuine, self-hosted pays for itself. Below that volume, commercial APIs with appropriate DPA agreements may be the pragmatic choice.

The composite architecture

These four layers compose naturally:

Incoming document or request (untrusted input)
                ↓
        [Layer 4] Self-hosted visual extraction (Qwen-3.6-VL + Gemma-4) — doc stays on-prem
                ↓
        [Layer 3] PII redaction — Presidio, ~15–30ms
                ↓
        AI inference — on anonymized, structured input
                ↓
        [Layer 1] Deterministic verifier — schema, state, permissions, ~2–5ms
                ↓
        [Layer 2] Tool executor — in ephemeral serverless context
                ↓
        Audit log + human review queue

Each layer addresses a distinct threat vector. Together they form a defense-in-depth stack that doesn't require trusting any single component.

The honest cost summary

None of these layers are free. There are real latency additions, real operational overhead items, and — as the nano benchmark shows — real accuracy tradeoffs if you push the model tier too low. The cost of the verification layer is measured in milliseconds. The cost of PII redaction is a small latency variance and a model tier requirement. The cost of serverless isolation is cold start management. The cost of self-hosted vision is MLOps overhead.

What these costs have in common: they are bounded, predictable, and manageable. The cost of a security incident in a regulated environment is none of those things.

The architectural insight worth keeping: security layers designed as first-class components of the system are cheap. Security controls retrofitted onto an insecure architecture are expensive. The time to make these choices is before you've shipped.

Patched builds structured workflow AI for regulated operations. If you're working on AI automation in regulated industries and want to compare notes, we're easy to find.