Enterprise RAG Security and Evaluation: A Production Decision Framework

A production framework for governing sources, enforcing access, containing prompt injection, evaluating retrieval and answers, and operating enterprise RAG with evidence.

Executive decision brief

Decision: proceed only with a bounded RAG use case when source governance, authorization-before-retrieval, adversarial evaluation, operational ownership and rollback evidence all pass. Otherwise close named prerequisites, redesign the boundary or stop.

Retrieval-augmented generation can help a language model answer with current enterprise information. It can also create a new path through which users, documents, indexes, prompts, models, logs and business systems interact. The architecture is production-ready only when the organization can prove which sources may be used, which evidence each user may retrieve, how untrusted content is contained, how answer quality is measured, and how changes or incidents are reversed.

This whitepaper presents a decision framework rather than a product recipe. It separates four questions that are frequently collapsed into one demo:

  1. Is the source estate governable? Rights, ownership, authority, freshness, retention and deletion are known.
  2. Is retrieval authorized and useful? Access rules survive ingestion, indexing, retrieval, caching and evaluation, while representative questions find sufficient evidence.
  3. Is generation bounded? Retrieved content remains data rather than trusted instruction, outputs are supported and unsafe actions stay outside model authority.
  4. Can the system be operated? Releases are evaluated, traces are reviewable, incidents are contained, costs are attributable and rollback has been rehearsed.

The recommended decision is not “adopt RAG” at enterprise scale. Select one information domain, one user population and one bounded task. Build the evidence needed to decide whether that system may proceed to a controlled pilot, must close named prerequisites, requires redesign, or should stop.

Operational and security consequences must be reviewed together. A security rule that operators cannot observe or recover will fail under pressure, while an operational shortcut that bypasses identity or source controls creates a security defect.

1. Scope and assumptions

The framework applies to systems that retrieve enterprise content or records and provide them as context to a generative model. It covers internal assistants, customer help, research, policy search, document drafting and similar knowledge workflows. It does not assume a particular model, vector database, cloud or orchestration library.

The system may retrieve from documents, structured records, search indexes, graphs or APIs. The relevant control boundary includes every derived representation: extracted text, chunks, embeddings, metadata, caches, evaluation samples, prompts, responses and logs. A permission applied only to the original repository is not an end-to-end control.

The paper assumes that the organization will retain human authority over material legal, financial, employment, safety or customer decisions unless a separate, approved control framework establishes otherwise. It does not provide legal advice or claim that one architecture satisfies every privacy, copyright, contractual or sector-specific requirement.

2. Why a RAG demo is not production evidence

A demo usually proves that selected content can be embedded, retrieved and summarized for selected questions. Production introduces conflicting versions, changing permissions, failed ingestion, ambiguous language, malicious content, users with different roles, provider limits and questions for which the corpus has no sufficient answer.

Fluent output can hide a weak retrieval path. A response may sound correct while using the wrong version, missing the decisive clause, combining two incompatible sources or exposing information the user could not access directly. Conversely, a model may refuse even when the right evidence was available because the context was truncated or the instruction hierarchy was confused.

The decision unit must therefore be the complete application, not the model. NIST’s Generative AI Profile frames risk management across the AI lifecycle and the organization’s goals, risk tolerance and context. That lifecycle view is appropriate for RAG because many severe failures originate in data, identity, application logic or operations rather than model weights.

3. Define the task and consequence boundary

Write the task as an observable user outcome: “find the current approved policy and explain the relevant clause with citations” is testable. “Enterprise knowledge copilot” is not. Name the user groups, source collections, jurisdictions, expected decisions, prohibited uses, escalation path and existing workflow.

Classify consequences before designing the experience:

| Consequence class | Example behavior | Minimum control direction | | --- | --- | --- | | Informational | Locate and summarize internal guidance | Citation, access enforcement, no-answer path and feedback | | Advisory | Recommend a procedure or next step | Evidence sufficiency, limitations, human confirmation and decision record | | Transaction preparation | Draft a ticket, response or structured action | Schema validation, preview, user approval and stable operation identity | | External or irreversible action | Send, approve, change, purchase or delete | Application-enforced authority, policy checks, confirmation and readback |

Retrieval quality alone cannot justify expanding action authority. Keep the model’s ability to compose language separate from the application’s authority to perform a business operation.

4. Establish source authority and permitted use

For every source collection, record the owner, business purpose, system of record, audience, permitted AI use, geographic or contractual restrictions, retention, deletion, refresh mechanism and conflict rule. Publicly reachable content should not be assumed to be unrestricted for every derived use.

Authority is contextual. A published policy may override a draft, but a local operating procedure may be authoritative for one site. A support article may explain a product without being authoritative for a contract. Encode enough metadata to distinguish these cases and define which source wins when records disagree.

Reject or quarantine sources that lack an owner, contain uncontrolled secrets, have no reliable version identity, or cannot meet the required deletion and access behavior. More content is not automatically better. A smaller governed corpus can produce more defensible answers than a larger mixture of uncertain material.

5. Design the ingestion and lineage contract

Ingestion must preserve evidence about origin. Each derived record should retain a stable source identifier, version or revision, collection timestamp, content hash, authority class, access attributes, transformation version and deletion state. Preserve structural cues such as headings, tables and lists when they affect meaning.

Make the pipeline restartable and reconcilable. A failed batch should not leave half a document active without an observable state. Use idempotent operations or versioned replacement so retries do not create indistinguishable duplicates. Record rejected content and transformation errors separately from successful ingestion.

Chunking is a semantic decision, not merely a token setting. A chunk should retain the context needed to interpret its claim, while retrieval must remain precise enough to avoid flooding the model with irrelevant text. Tables, definitions, exceptions and cross-references may need specialized extraction. Test them directly instead of assuming one chunk size works for all formats.

"type":"svg-architecture", "title":"Governed enterprise RAG reference architecture", "nodes":[ ], "links":[ ], "caption":"Authorization precedes retrieval. Source lineage and policy metadata remain attached to evidence, while evaluation and operations control every release." }} />

6. Enforce authorization before retrieval

The application should derive identity and authorization from trusted session or service context, not from a role stated in the user’s prompt. Filter the searchable corpus before similarity or semantic retrieval where possible. Post-filtering an unauthorized result after retrieval can still expose content through logs, ranking behavior, caches or model context.

Carry the access dimensions required by the source system: tenant, organization, group, role, purpose, region, document-level grants and expiry. Avoid simplifying a complex source permission model into one broad index label without proving equivalence. If the derived system cannot represent the source rule, that source is not ready for ingestion into that architecture.

Test mixed-permission cases. Put similar language in an allowed document and a restricted document. Ask direct, indirect and comparative questions. Confirm that retrieval, snippets, citations, autocomplete, history and evaluator traces contain only authorized evidence.

The AWS Generative AI Security Reference Architecture describes RAG as access to external enterprise data and emphasizes defense in depth. Its secure access capability is a useful vendor reference, but the organization must map controls to its identity and data model.

7. Treat retrieved content as untrusted input

Documents can contain instructions, hidden text, manipulated metadata, links and examples that conflict with the application’s intent. Retrieved text must remain data. It should not gain authority merely because it was indexed from an approved repository.

OWASP lists prompt injection as a leading risk for LLM applications and notes that RAG does not fully remove it. See LLM01:2025 Prompt Injection. Apply controls at several layers:

  • separate trusted application instructions from user and retrieved content;
  • label source boundaries and citations explicitly;
  • limit retrieval to the task and user scope;
  • validate any structured output in application code;
  • keep credentials, permissions and policy enforcement outside prompts;
  • restrict tools independently of model text;
  • test direct, indirect, encoded and multi-document instruction conflicts;
  • design a safe refusal or escalation path.

Content scanning may identify known patterns but cannot prove that arbitrary natural language is safe. The important architectural control is that a successful injection cannot grant new data or action authority.

8. Prevent sensitive information disclosure

Least privilege should apply to source connectors, ingestion workers, indexes, model calls, evaluators, administrators and support tools. Avoid placing secrets in system prompts or traces. Classify prompt and response data according to the information they may contain, not according to the fact that they are “AI logs.”

OWASP’s Sensitive Information Disclosure guidance warns that system-prompt restrictions may be bypassed and should not replace controls. The application must prevent unauthorized evidence from reaching the model, minimize data sent to providers, and validate outputs before they enter another system.

Review provider data use, retention, regional processing, support access and deletion behavior for the selected endpoints and contract. These settings can change by product or account configuration. Retain a dated decision record and verify current provider documentation before launch.

9. Make retrieval evaluation independent from answer evaluation

An answer can fail because retrieval missed the evidence, retrieved the wrong version, supplied too much irrelevant context, or because generation ignored or distorted good evidence. Measure these stages separately.

Build an evaluation set from real tasks, support questions, search logs and expert interviews. Include:

  • ordinary questions with one clear source;
  • questions requiring several sources;
  • synonyms, acronyms and vague language;
  • recently changed information;
  • conflicting or superseded sources;
  • questions whose evidence is restricted for some roles;
  • questions with no sufficient evidence;
  • adversarial instructions in user input and retrieved content.

For retrieval, judge whether the required evidence appears within the context actually given to the model, whether unauthorized evidence is absent and whether decisive qualifications remain intact. Useful measures may include recall at a context limit, precision, rank, freshness and permission correctness. The measure must match the application’s evidence needs.

10. Evaluate grounded answer behavior

Answer evaluation should examine factual support, completeness, citation correctness, contradiction handling, uncertainty, refusal and task usefulness. Do not reward fluency separately from evidence. A concise refusal can be safer and more useful than a polished unsupported answer.

Use deterministic checks for permissions, schemas, required citation fields, source versions, operation limits and forbidden outputs. Use trained human reviewers for judgments that require context. Model-based grading can increase coverage, but validate the grader against human decisions and monitor disagreement.

OpenAI’s Evals API represents evaluation as testing criteria plus a data-source schema and supports comparable runs. The portable principle is to version test data, criteria, system configuration and results so a release decision can be reproduced, regardless of provider.

Do not collapse all scores into one average. Unauthorized disclosure, invented citations or an unsafe external action should remain blocking failures. Segment results by audience, task, source type, language and consequence class so aggregate performance does not hide a vulnerable group.

11. Define release gates and change evidence

Version the source snapshot, ingestion logic, chunking, embedding model, index configuration, retrieval policy, reranker, prompt, generation model, tool contract and evaluation set. A change to any one of these can alter the system’s behavior.

Use progressive release. Start with offline evaluation, then a controlled internal pilot, shadow or advisory use where appropriate, and a bounded production cohort. Preserve a stable rollback unit rather than attempting to reconstruct the previous state from memory.

"type":"svg-flow", "title":"RAG evaluation and release control loop", "steps":[ ], "caption":"A configuration reaches broader use only after staged evidence. Production traces and user outcomes return to the next evaluation cycle." }} />

Release criteria should include minimum task usefulness and retrieval support, plus zero tolerance for named blocking conditions. Set thresholds from business consequence and baseline performance rather than a generic benchmark. Record the owner who can approve an exception and the date on which it expires.

12. Design no-answer, fallback and escalation behavior

The system needs a useful response when evidence is missing, conflicting, stale or unauthorized. “I do not know” without a path forward can frustrate users, while an unsupported answer creates false confidence. The fallback may link to the source system, request clarification, show the conflicting documents, open a human review task or return the user to the current workflow.

Do not silently switch to a different corpus, model or region when a dependency fails. That can change permissions, retention, output behavior and cost. Treat fallback configuration as part of the governed release.

For high-consequence workflows, require human confirmation of the evidence and proposed action. Show the cited passages, source version and limitations needed for that review. Avoid presenting model confidence as a probability of correctness unless the metric has a defensible calibration for the specific task.

13. Operate freshness, deletion and correction

Define the acceptable lag for each collection and monitor source-to-index freshness. Use stable identities so a changed or removed document updates the correct derived records. Detect ingestion failures, stalled queues and partial replacements.

Deletion must cover extracted copies, chunks, embeddings, keyword indexes, caches and governed prompts or responses. Backups and immutable audit records may have separate obligations. Document those limits instead of promising instantaneous erasure the architecture cannot perform.

When an authoritative source is corrected, decide whether previous answers must be identified, users notified or downstream drafts re-reviewed. A RAG system can reproduce outdated content long after the source team believes it has been removed unless the derived estate is reconciled.

14. Build the production operating model

Name owners for source approval, ingestion, access policy, retrieval, prompts, models, evaluation, security response, cost, user support and the disable path. A prototype team may cover several roles temporarily, but the production record must make authority explicit.

Monitor user-visible task success, retrieval support, refusal and escalation, permission-denial behavior, ingestion lag, evaluation drift, latency, provider errors and cost. Sample traces according to risk and privacy requirements. Do not log unrestricted content by default merely because debugging may be easier.

Create an incident runbook for suspected disclosure, poisoned content, incorrect authoritative answers, provider failure and uncontrolled cost. The runbook should identify how to disable a source, index version, feature, user cohort or external action independently.

15. Model cost and capacity honestly

The production cost includes source connectors, extraction, embedding, indexes, reranking, model tokens, guardrails, evaluation, observability, human review, support and exception handling. Estimate per useful completed task, not only per model call.

Cache only where identity, freshness and deletion controls remain valid. A shared response cache can cross permission boundaries. Reducing context may lower cost while omitting decisive evidence. Using a smaller model may be appropriate after task-specific evaluation, not merely because a benchmark suggests it.

Test peak demand, long documents, large permission filters and evaluation load. Provider quotas and index latency can create a failure mode in which the system retrieves less evidence or times out silently. Make degraded behavior explicit.

16. Compare architecture options and tradeoffs

| Choice | Advantage | Risk and required evidence | | --- | --- | --- | | Shared index with permission filtering | Operational efficiency and cross-source retrieval | Filter correctness, tenant isolation, cache safety and leakage tests | | Separate index by tenant or domain | Stronger isolation boundary and simpler local policy | Provisioning, cost, lifecycle consistency and cross-domain questions | | Keyword plus semantic retrieval | Better coverage for identifiers and natural language | Fusion, ranking and duplicate-evidence behavior | | Precomputed chunks | Predictable latency and repeatable evaluation | Freshness, structure loss and transformation governance | | Live source retrieval | Current records and source-native authorization | Latency, availability, rate limits and query safety | | One general assistant | Familiar access point | Broad authority, ambiguous context and evaluation explosion | | Task-specific experiences | Clearer evidence and safer actions | More product surfaces and shared-platform requirements |

No option is universally best. Choose the boundary that makes identity, data ownership, task evaluation and failure containment understandable to the teams that must operate it.

17. Failure modes that should block release

Block or redesign the release when the team cannot prove source permission, derived-store deletion, authorization before retrieval, representative negative evaluation, or an owner for incident response. Also block when the model can convert untrusted content into external authority, when unsupported answers are presented as authoritative, or when the fallback silently changes the data-control boundary.

Other warning signs include evaluation written entirely from the demo set, identical questions used for tuning and release approval, missing source versions in citations, unrestricted administrator search, and logs that become a second ungoverned knowledge base.

The recovery path depends on the phase. Before launch, quarantine the affected source or configuration and rebuild the evaluation evidence. During a bounded pilot, disable the cohort or revert to the prior version. After possible disclosure, preserve evidence, revoke access where appropriate, identify affected users and content, follow the organization’s incident process, and do not resume until the control failure and derived copies are addressed.

18. A 12-week bounded implementation sequence

The timeline below is illustrative, not a delivery promise. Scope, data condition, approvals and platform readiness may change it.

| Period | Work | Exit evidence | | --- | --- | --- | | Weeks 1 to 2 | Define task, users, consequence, baseline, sources and owners | Approved scope and source register | | Weeks 3 to 4 | Prove ingestion, lineage, permissions, freshness and deletion in a small corpus | Reconciliation and access-test record | | Weeks 5 to 6 | Build representative retrieval and negative evaluation set | Expected evidence and reviewer guidance | | Weeks 7 to 8 | Implement bounded generation, citations, refusal and application controls | Offline evaluation and threat-test results | | Weeks 9 to 10 | Exercise incidents, rollback, cost, load and support | Operational acceptance evidence | | Weeks 11 to 12 | Run controlled pilot and compare with baseline | Proceed, remediate, redesign or stop decision |

Avoid expanding sources or users merely to keep the timeline busy. Close control and evidence gaps while they are small.

19. Decision record and acceptance checklist

Use case, user groups, task, baseline, and prohibited scope:
Source owners, permitted use, authority, access, retention, and deletion:
Ingestion version, lineage, reconciliation, and freshness evidence:
Identity source, authorization policy, tenant boundary, and cache behavior:
Retrieval set, expected evidence, negative cases, and segment results:
Answer criteria, blocking failures, reviewer guidance, and disagreements:
Prompt-injection tests, tool authority, output validation, and fallback:
Release version, pilot cohort, operational owners, cost, and rollback:
Known limitations, accepted risks, approver, expiry, and next review:

"One bounded task, audience, consequence class, and existing baseline are approved.", "Every source collection has an owner, permitted-use decision, authority rule, and lifecycle path.", "Lineage connects each retrieved passage to a stable source version and transformation.", "Authorization is derived from trusted context and enforced before retrieval.", "Mixed-permission, no-answer, conflicting-version, deletion, and prompt-injection tests pass.", "Retrieval quality is evaluated separately from grounded answer behavior.", "Blocking security and evidence failures cannot be averaged into a passing score.", "Fallback, provider failure, rollback, disable, correction, and incident paths are exercised.", "Production owners accept support, evaluation, access review, cost, and evidence retention.", "The pilot decision records limitations and does not imply enterprise-wide readiness." ]} />

20. Govern model and provider dependencies

The model provider is one dependency in the system, but its behavior can affect answer quality, latency, cost, data handling and release evidence. Record the selected model and endpoint, version policy, supported regions, quotas, content limits, data controls, retention, support access and change-notification mechanism. Verify current terms and technical documentation for the actual account and endpoint rather than relying on a general product statement.

Pin versions where the provider supports it and still run evaluations for each material release. A pinned identifier does not freeze every surrounding service behavior, and an automatically updated model can change instruction following, refusal, token use or citation behavior. Keep the prior supported configuration available for a bounded rollback period.

Design provider abstraction around the controls the application needs, not around a claim that all models are interchangeable. Models may differ in context handling, structured output, tool interfaces, safety behavior, regional availability and telemetry. The abstraction should make those differences visible to evaluation and operations.

Procurement should examine more than token price. Include service objectives, incident communication, audit material, sub-processors where applicable, data-use controls, retention, deletion, exportability and termination assistance. The technical team should show what data reaches the provider and which controls remain the organization’s responsibility.

If a fallback provider is required, evaluate it as a separate release. Confirm that identity, data boundary, regional processing, prompt format, output validation and refusal behavior remain acceptable. A switch that restores availability while weakening access or data controls is not a safe fallback.

21. Detect source poisoning and authority manipulation

An attacker or mistaken editor may place misleading content into an otherwise approved repository. The ingestion pipeline can faithfully preserve and retrieve it, so source approval alone does not guarantee content integrity. Separate permission to contribute from authority to define facts for the RAG task.

Protect high-authority collections with review, version history and restricted publication workflows. Use signed or verifiable source revisions where the environment supports them. Monitor sudden changes in document volume, authority labels, access metadata and retrieval share. A new document that begins dominating answers should be reviewable even when ingestion succeeded technically.

Evaluation should include poisoned and contradictory sources. Test whether the system recognizes authority, dates and scope, whether it cites the manipulated source, and whether a retrieved instruction can change system behavior. Keep the ability to disable one collection or revision without taking the entire assistant offline.

When poisoning is suspected, freeze the affected index version, preserve source and retrieval evidence, remove the collection from active retrieval, identify queries and responses that used it, and follow the organization’s incident process. Rebuild from a known source checkpoint after the content owner and security owner approve the recovery boundary.

This control is especially important when content comes from collaborative tools, external websites, customer submissions or automatically generated documents. Broad contribution rights can be appropriate for collaboration but inappropriate for an authoritative answer corpus without an additional publication step.

22. Design evaluation as a maintained product asset

An evaluation set decays when products, policies, language and user behavior change. Give it an owner, schema, source rationale, version history and review cadence. Store the expected evidence, not merely an expected sentence, so the system can improve its wording without losing the factual test.

Separate development, validation and release sets where repeated tuning could overfit known examples. Add production failures after investigation, but avoid making every incident example visible to the same workflow that optimizes prompts. For sensitive data, use controlled access, representative synthetic cases or approved de-identification while retaining the behavior that matters.

Reviewer guidance should define supported, partially supported, unsupported, harmless and blocking outcomes with examples. Measure inter-reviewer agreement and adjudicate important disagreements. If experts cannot agree on the authoritative answer or required evidence, the source domain may not be ready for automated evaluation.

Track results over time by configuration and segment. A release comparison should show which cases improved, regressed or became unevaluable, not only the new average. Retain enough trace information to distinguish retrieval failure, generation failure, policy failure and evaluator uncertainty.

Use production feedback as investigation input rather than direct ground truth. Users may reward a confident but wrong answer or reject a correct answer that conflicts with expectation. Sample high-consequence and low-confidence interactions for expert review according to privacy and retention requirements.

23. Establish governance that can make real decisions

Governance should connect policy to the people who can stop or change the system. A review board that only records recommendations cannot contain an active disclosure or unsafe release. Define decision rights for source approval, risk acceptance, production release, incident disablement, external action authority and expansion to new users or domains.

Use a tiered process. Low-consequence changes that pass established evaluation may follow normal engineering approval. New data classes, broader audiences, changed action authority or unresolved blocking failures should require review by the relevant data, security, legal, privacy, product or business owner. The required participants depend on the actual consequence, not on the label “AI.”

Record accepted risks with evidence, owner, affected scope, compensating control and expiry. Permanent exceptions tend to become undocumented architecture. An expiry date forces the organization to confirm whether the context and control still apply.

Governance should also retire systems. Define conditions for pausing or ending a use case when the value is not demonstrated, data obligations cannot be met, provider economics change, support demand exceeds benefit or a safer existing workflow performs better. Stopping a weak implementation is a valid outcome, not a failed innovation programme.

For each quarterly or material-change review, present the current scope, production outcomes, evaluation regressions, security events, access changes, source health, user complaints, cost per useful task, open exceptions and next expansion request. End with an explicit decision and accountable actions.

24. Limitations and responsible use

Evaluation demonstrates behavior on defined evidence, not universal safety or correctness. Attack techniques, models, provider controls and source collections change. Human review can also be inconsistent or biased, so reviewer guidance and disagreement analysis matter. Reference architectures must be adapted to actual identity, data, contractual and operating constraints.

NIST AI RMF resources, the NIST Generative AI Profile, the OWASP Top 10 for LLM and GenAI applications, the AWS Generative AI Lens security guidance, the AWS security reference architecture and provider evaluation documentation are inputs to local judgment. They do not certify an implementation.

The correct final artifact is an evidence-backed, bounded decision. A useful RAG system shows users where its answer came from, respects the same information boundaries they already rely on, fails safely when evidence is insufficient, and gives operators a practical way to change, disable and recover it.