AI Security Architecture Is Not Just AI Governance

AI governance is now a boardroom topic. Organizations are writing policies, approving tools, mapping use cases, discussing the AI Act, reviewing GDPR exposure and deciding who is allowed to use which model.

For high-risk AI systems in the EU, this is not only a management preference. Article 15 of the AI Act explicitly points to technical robustness, cybersecurity and resilience against attempts to alter use, outputs or performance, including adversarial and data-related attacks. That makes security architecture part of the compliance discussion, not an optional engineering add-on.

That work matters. Without governance, AI quickly becomes a collection of shadow tools, private experiments and unmanaged data flows.

But governance is not enough.

An AI policy can say that sensitive data must not be exposed. It can say that human oversight is required. It can say that AI systems must be secure, transparent and accountable.

The harder question is whether the system actually enforces any of that.

Who can access the data? Which documents enter the prompt? What can an AI agent do through tools? How is prompt injection tested? Does the retrieval layer respect user permissions? What evidence remains after an incident?

Those questions are not answered by governance alone. They belong to AI security architecture.

Governance defines intent; architecture enforces it

AI governance usually defines the rules around AI use. It covers approved tools, data restrictions, risk ownership, legal review, transparency requirements, exception handling and reporting.

This is the right starting point. A company needs to know what it allows before it can secure it.

The mistake is treating the policy as the control.

A policy may state that an internal AI assistant must not reveal confidential documents to unauthorized users. That sentence is useful, but it does not protect anything by itself. The protection depends on the architecture:

• whether the user identity reaches the retrieval layer,
• whether document permissions are enforced at query time,
• whether sensitive chunks are minimized before they enter the prompt,
• whether answers contain source metadata,
• whether access failures are logged,
• whether negative permission tests exist.

Governance says what should happen. Architecture shows how it happens.

An AI system is more than a model

Many AI security discussions still focus too heavily on the model. The model matters, but in enterprise systems it is only one component.

A real AI system may include:

• a user interface,
• authentication and authorization,
• prompt templates,
• system instructions,
• retrieval pipelines,
• document stores,
• vector indexes,
• model providers,
• API integrations,
• agents and tool calls,
• logging and monitoring,
• administrative workflows,
• CI/CD pipelines,
• software dependencies,
• governance processes.

Each component can fail in a different way.

A retrieval system can expose documents the user should not see. An agent can call an API with broader permissions than the user has. Logs can retain sensitive prompts. A model provider boundary can be poorly understood. A dependency in the AI application can become a supply-chain risk.

If the review looks only at the model, it misses the system.

A basic AI security architecture view should trace the whole flow:

User
  ↓
Application layer
  ↓
Authentication and authorization
  ↓
Prompt and system instructions
  ↓
Retrieval, documents and vector index
  ↓
Model provider
  ↓
Tools, APIs and agent actions
  ↓
Output, logs and audit evidence

At every step, the architect should ask who is acting, what data is processed, which permissions apply, where the trust boundary is, what abuse path exists, which control reduces the risk and what evidence proves that the control works.

Prompt injection is not only a prompt problem

Prompt injection is often described as a user telling the model to ignore previous instructions. That is direct prompt injection: the user is the attacker and the prompt is the delivery channel.

The harder enterprise risk is often indirect prompt injection. In that case, the malicious instruction is hidden in a web page, PDF, ticket, email, document or knowledge-base article that the AI system later reads through RAG or an agent workflow. The user may be legitimate. The content source is not.

Prompt injection is a trust-boundary failure.

If untrusted content can influence a model that also has access to internal data, tools or actions, text becomes a control surface. The risk depends less on the sentence itself and more on what the model can influence after reading it.

The architecture must answer questions such as:

Area	Architecture question
Trusted instructions	Which instructions are system-controlled and which come from untrusted content?
RAG	Can a document, web page, email or PDF inject instructions into model context?
Tools	Can injected text trigger an API call or workflow action?
Data	Can the attack expose internal, confidential or personal data?
Output	Can model output become HTML, SQL, code or an executable workflow step?
Monitoring	Can the attempt be detected and investigated later?

A better system prompt may help, but it is not enough. Direct and indirect prompt injection require least privilege, trusted and untrusted context separation, tool control, output validation, abuse-case testing and logging.

This is why prompt injection is a useful maturity test. If an organization cannot explain what a model is allowed to do after receiving hostile instructions, the trust boundaries are not yet clear enough.

RAG turns the knowledge base into a security boundary

Retrieval-augmented generation is useful because it lets a model answer with additional context from documents, databases, search results or vector indexes.

It also changes the security model.

The knowledge base becomes part of the control plane. The organization must know which sources are indexed, who owns them, how they are classified, whether they are current, who can read them and whether permissions survive the retrieval process.

Important RAG security questions include:

• Which documents are indexed?
• Are documents classified by sensitivity?
• Are permissions enforced at document or chunk level?
• Does retrieval run under the user’s identity or a broad service identity?
• Is too much context sent into the prompt?
• Are citations and source metadata preserved?
• Can the organization reconstruct which sources shaped an answer?
• How are stale or restricted sources removed?

Without these controls, a RAG system can reveal information that the original application would never show to that user.

That is not a model hallucination. It is an architecture failure.

IAM for AI systems needs special attention

Classic IAM asks who the user is and what that user can do.

AI systems add more actors:

• human users,
• AI assistants,
• autonomous agents,
• service accounts,
• plugins,
• API integrations,
• retrieval services,
• model providers,
• automated workflows.

The central question is: under whose authority does the AI act?

If an agent uses one broad privileged service account, the organization has created a powerful technical user controlled through natural language. That may be convenient, but it is a high-risk design unless it is tightly scoped, logged and reviewed.

Better patterns exist. In many enterprise environments, the agent should use delegation or impersonation patterns such as OAuth2 on-behalf-of flow, short-lived scoped tokens or just-in-time authorization. Instead of giving the AI a static privileged account, the system can pass a limited token derived from the authenticated user and scoped to one action, one data set or one API call.

An AI identity model should clarify:

Area	Question
User identity	Is the human user’s identity carried into the AI workflow?
Agent identity	Does the agent act as the user or as a separate service?
Delegation	Is it clear when AI acts on behalf of a person, and is the delegated token short-lived and scoped?
Tool permissions	Which APIs and tools can the AI call?
RAG permissions	Does retrieval respect user access rights?
Secrets	Can prompts or tool calls expose credentials?
Audit	Can an action be traced to a user, agent and decision?

For AI systems, Zero Trust principles are not theoretical. Identity, least privilege, segmentation, continuous verification and telemetry become practical design requirements.

Governance without evidence is weak

Security and compliance depend on evidence. It is not enough to say that a control exists. The organization must be able to show how it works.

AI security architecture should therefore produce audit-ready artifacts:

• AI use-case inventory,
• architecture diagrams,
• data-flow maps,
• identity and access-control model,
• list of data sources,
• data classification,
• threat model,
• prompt-injection tests,
• RAG permission tests,
• exception records,
• logging and monitoring design,
• incident-response scenarios,
• risk-acceptance decisions.

These artifacts connect policy to implementation. Governance says that AI risk must be managed. Evidence shows how it is managed.

Without evidence, governance remains mostly declarative. With evidence, it becomes something that can be tested, audited and improved.

A minimal AI security architecture review

The first review does not need to be a large program. It can start with a structured assessment of the most important AI systems.

A practical minimum includes the following areas.

AI use-case inventory

The organization should know where AI is used, who owns each system and whether the use case is public, internal or sensitive.

Data flows

For each important system, the review should show where data comes from, where it goes, where it is stored and whether it leaves the organization.

Identity model

The review should identify users, agents, service accounts, tool permissions and delegation rules.

Retrieval and RAG layer

If the system uses RAG, the review should cover indexed sources, permissions, classification, provenance, freshness and logging.

Threat model

The threat model should include prompt injection, data leakage, unauthorized retrieval, tool misuse, model supply-chain risk and audit failure.

Controls and evidence

For each important risk, the review should identify the control and the evidence that proves it exists.

Incident response

AI incident response should cover wrong outputs, data leakage through prompts, agent misuse, contaminated retrieval sources and decisions that cannot be reconstructed.

Discussion

The distinction between AI governance and AI security architecture will become more important as AI systems move from experiments into business processes.

Governance is often the first visible layer because it is easier for management, legal teams and compliance teams to discuss. Architecture is where the organization finds out whether the rules can actually be enforced.

A company may look prepared because it has an AI policy. But the policy does not answer whether retrieval respects permissions, whether an agent can overreach, whether prompt injection is tested or whether logs can support an investigation.

AI security architecture is not a replacement for governance. It is the technical continuation of governance.

Without governance, architecture lacks business rules, ownership and risk decisions. Without architecture, governance lacks enforcement.

Conclusion

AI governance is necessary, but it is not sufficient for secure AI operation.

Organizations need AI security architecture to translate policies, risks and regulatory expectations into technical controls. That architecture must describe users, identities, data flows, retrieval layers, models, tools, agents, logs and audit evidence.

It must address prompt injection, RAG security, IAM, model supply chain, data governance and incident response. It must produce evidence, not only recommendations.

The question for organizations is therefore not only:

Do we have an AI policy?

A better question is:

Do we have an AI security architecture that technically enforces the policy and can prove it?

That difference separates AI security as a document from AI security as an operating capability.