AI Integration Patterns for Enterprise Systems

Integrate AI into an existing workflow with explicit authority, permission-aware retrieval, task-specific evaluation, bounded operating costs, and recovery gates.

trigger="A defined business workflow may benefit from model-generated predictions, content, or proposed actions." owner="The engineering owner accountable for the workflow's production outcome." participants={["Product and process owner", "AI engineer", "Data owner", "Security and privacy reviewers", "Operations lead", "Domain reviewer"]} prerequisites={[ "A documented current workflow, baseline performance, and a non-AI operating path.", "Approved data sources and provider terms, with permission, retention, and deletion responsibilities.", "Representative evaluation cases, consequential failure examples, and available human-review capacity." ]} outputs={[ "An integration contract defining permitted inputs, outputs, actions, and escalation.", "A versioned evaluation report and operating-cost comparison against the existing workflow.", "A staged rollout, disable control, and reconciliation runbook." ]} doneWhen={[ "The process owner accepts measured quality and failure behavior for the intended task.", "Permission, injection, deletion, tool-authorization, and fallback tests pass their defined gates.", "Reviewers can reject or correct outputs before any action that requires their approval.", "Operators can disable model use, preserve pending work, and reconcile completed effects." ]} />

Start with the workflow and its authority boundary

Choose one workflow with a clear input, useful output, and accountable owner. “Add an assistant” is not an implementation contract. “Draft a support response using documents this employee may access, then require approval before sending” is specific enough to test.

Record whether the system advises, drafts, classifies, or executes. These are different authority levels. A model that produces a plausible answer has not established permission to read another customer's record, update a system of record, or send a message.

This playbook addresses integration and operation, not model training from scratch. It presents engineering procedures, not client outcomes. The acceptance thresholds must come from the task's costs and risks; there is no general accuracy percentage or confidence score that makes an AI workflow ready for production.

1. Establish the baseline and release gates

The process owner documents current completion quality, elapsed time, review effort, exception rate, and operating cost. Include rejected and abandoned cases so the baseline does not contain only successful work. The AI owner defines how those measures will be collected for the proposed integration.

Build an evaluation set that covers routine cases, difficult cases, missing information, unanswerable requests, permission boundaries, and foreseeable misuse. Use approved or appropriately de-identified data. Keep development examples separate from the held-out release set, and track dataset versions and known coverage gaps.

| Gate | Owner | Evidence required | | --- | --- | --- | | Task value | Product owner | Comparison with the current process, including review and rework | | Output quality | Domain reviewer | Task-specific rubric, error categories, and reviewed examples | | Access and actions | Security owner | Denied-read and denied-action tests across roles and tenants | | Data handling | Data and privacy owners | Source rights, provider terms, retention, and deletion procedure | | Reliability | Service owner | Deadline, quota, outage, queue, and disable-path tests | | Operational readiness | Operations lead | Monitoring, escalation, reviewer capacity, and recovery exercise |

The NIST AI Risk Management Framework provides a voluntary risk-management structure and links to its generative-AI profile. Use it to organize responsibilities and evidence, not as a certification or a substitute for applicable law.

2. Select the narrowest useful integration pattern

| Pattern | Use when | Condition that must hold | | --- | --- | --- | | Synchronous model API | A user needs a result within the interaction | End-to-end deadline and explicit unavailable behavior are acceptable | | Asynchronous job | Processing can complete after the initial request | Durable state, progress, cancellation, expiry, and deduplication are defined | | Assisted drafting | A person can assess output before release | The reviewer has the context, authority, and time to make a real decision | | Retrieval-augmented generation | The task needs approved external knowledge | Retrieval permissions, freshness, provenance, and deletion are controlled | | Tool-assisted workflow | The model proposes calls to business capabilities | The application independently validates and authorizes every action | | Shadow evaluation | Outputs can be compared without changing the live decision | Shadow execution cannot send messages or create duplicate side effects |

A separately deployed inference service is not necessarily a sidecar. Reserve “sidecar” for a component co-located with an application instance under the selected deployment model. Choose placement from isolation, resource, latency, and scaling requirements rather than treating a label as an architectural advantage.

Shadow mode still processes data and consumes capacity. Confirm the provider may receive that data, bound the extra load, and disable real writes in the shadow path. It is not automatically safe because its answer is hidden from the user.

3. Keep model output outside the trusted execution boundary

Place orchestration behind the existing application's authentication and authorization controls. The model receives only the context needed for the task. A retrieved document, user message, or tool response is data, not authority to change system policy.

For tool use, allowlist capabilities and validate typed arguments, target resource, tenant, caller authority, and business limits outside the model. Use scoped credentials and stable operation identifiers. A model-generated claim that approval exists is not an approval record.

OWASP's prompt-injection guidance includes attacks delivered through retrieved content. Test hostile instructions in documents and tool responses as well as direct prompts. A second model or output filter can add a check, but it does not make untrusted instructions harmless or replace access controls.

4. Make retrieval freshness and deletion observable

Retrieval-augmented generation, or RAG, can provide task-relevant evidence. It does not ensure that the evidence is current, authorized, complete, or correctly used in the answer.

The data owner records source identifiers, versions, update times, permission metadata, and ingestion status. Define the largest acceptable freshness gap for the task and the behavior when the index is behind. Deny access or withhold an answer when permission evidence is missing or too old for the approved policy.

Azure AI Search's document-access documentation illustrates an important condition: query-time enforcement relies on permission metadata synchronized to the index. Check the specific source connector, feature status, and API version rather than assuming source permission changes take effect immediately.

Test the whole deletion path: source record, extracted text, chunks, embeddings, derived caches, and any retained evaluation or trace data within the approved retention policy. Track failed deletions and prevent affected content from being retrieved while repair is pending. Backup handling and legal retention exceptions need their own documented policy.

Require citations or source references when the task needs verifiable factual support. Evaluate whether each reference actually supports the answer. A cited document does not prove that the generated statement follows from it.

5. Evaluate quality and cost together

The AI owner compares candidate configurations on the same task set, with the same data permissions and acceptance rubric. Record model identifier, prompt version, retrieval configuration, tool schema, and evaluation method. Separate schema validity, factual support, task completion, safety failures, and human correction effort.

Do not treat self-reported confidence or token probability as a calibrated probability of business correctness. If a router uses a score, validate its relationship to the actual acceptance decision on held-out cases, including relevant task and user segments. Monitor coverage and escalation, not only the score among accepted answers.

An automated model judge can help organize evaluation, but compare its decisions with domain-reviewed labels and investigate disagreements. A larger judge model or a single correlation threshold does not establish reliability for the task.

A cascade may reduce model charges or may add latency, retries, and review. Use a workload calculation instead of a promised savings range:

Illustrative comparison structure, not a measured result:
Total operating cost =
  inference + retrieval + infrastructure + retries
  + human review + correction + ongoing operation

Cost per accepted outcome =
  total operating cost / outcomes meeting the same acceptance definition

For a cascade, include every attempted stage.
Report unresolved and rejected cases separately.

Use measured traffic proportions and current contracted prices. A cheap answer that requires correction is not equivalent to an accepted outcome. Semantic caching also needs permission and meaning-equivalence tests; similar wording can belong to different users or require different current facts.

6. Design failure paths and a usable review queue

| Failure | Expected behavior | Recovery evidence | | --- | --- | --- | | Provider timeout or quota limit | Respect the deadline; use an approved fallback or pending state | No unbounded retries or silent loss of accepted jobs | | Invalid or unsupported output | Reject, escalate, or perform a bounded repair attempt | Final output passes the task checks | | Missing or unauthorized context | Withhold the affected answer | Access and data freshness are restored before reuse | | Human review backlog | Limit admissions or use the existing workflow | Queue age and reviewer capacity remain within the accepted policy | | Tool result is unknown | Preserve the operation identity and reconcile | No duplicate external effect | | Model or prompt regression | Disable the affected configuration | Known-good workflow and compatibility are verified | | Data exposure or unsafe action | Contain the integration and invoke incident response | Scope, affected records, and required follow-up are established |

A secondary provider is a separate deployment candidate. Before failover, verify its data terms, regional processing, output behavior, tool compatibility, and task quality. Do not silently move sensitive work to an unevaluated provider.

Human review must be operationally real. Show the reviewer the proposed output, relevant evidence, missing information, and permitted actions. Support rejection and correction. For consequential actions, keep the approval attached to the exact action and parameters executed, and invalidate it when those parameters change.

7. Log observable evidence, not hidden reasoning

The service owner defines an audit record containing a request or operation identifier, permitted input references, retrieved-source versions, configuration identifiers, validation results, tool calls and outcomes, approvals, and final disposition. Store only the content needed for the approved purpose, with access and retention controls.

Do not rely on internal chain-of-thought as a faithful explanation or dependable audit artifact. Use concise decision explanations supported by observable evidence and the business rules that were applied. When an explanation is model-generated, label its role and verify consequential factual claims.

Monitor quality through an owned sampling and evaluation plan. Reassess after model, prompt, source, permission, tool, or workflow changes. A shift in input distribution warrants investigation; it does not by itself prove quality degraded or prescribe retraining. Hosted-model integrations may need prompt, retrieval, or workflow repair instead.

8. Resolve governance before expanding use

Maintain a use-case record for the jurisdictions, affected people, organization role, decision consequence, data categories, and intended purpose. A qualified legal or compliance reviewer determines applicable obligations, dates, required notices, assessment needs, and evidence retention. This engineering playbook is not legal advice or a compliance determination.

For EU-related use cases, the reviewer should consult the official AI Act text and current applicable amendments and guidance. Do not apply a single penalty percentage or label every enterprise use case “high risk.” Distinguish laws and contractual requirements from assurance reports. AICPA's SOC resources concern examinations and reporting on controls; SOC 2 is not an AI regulation.

A provider agreement, security report, or human reviewer does not by itself resolve the complete governance question. Record remaining issues and who may accept them. Keep unresolved domain and legal approval separate from a technically successful pilot.

9. Roll out with a reversible operating path

Pin the complete configuration where the platform allows it and document dependencies that cannot be pinned. Keep independent controls for model use, retrieval sources, and external actions. Test the non-AI path at expected fallback load.

Rollback stops future use of a bad configuration; it does not undo sent messages or changed records. Preserve affected operation identifiers, reconcile completed effects, and obtain the required approval for correction or compensation. Verify that a previous model configuration can still read the current tool and data schemas before restoring it.

10. Rehearse one draft from retrieval through release

Use the support-response workflow from the opening as a bounded exercise. The application owner creates an authorized test ticket, a permitted knowledge document, an inaccessible document, and an outdated revision. These are fixtures, not real customer messages. The expected output is a draft with traceable evidence, not a sent response. The domain reviewer defines which statements the draft must support and when it must ask for missing information.

Run the happy path first. Record the ticket revision, allowed source revisions, model configuration, draft revision, and proposed recipient. The reviewer accepts or edits the draft against the rubric. The release handler then verifies current authority and the exact approved content before attempting delivery. Approval of draft revision A must not authorize revision B, a different recipient, or a newly attached file.

Next, make the exercise fail deliberately. These acceptance checks are proposed controls for this workflow, not a claim that a particular model or product implements them:

| Injected change | Required decision | Evidence to retain | | --- | --- | --- | | Source permission is revoked after drafting | Block release and reassess affected content under current permissions | Permission revision and denied release record | | Document contains an instruction to export account data | Treat it as untrusted content; refuse the unauthorized tool action | Sanitized attempted action and policy decision | | Ticket changes while review is open | Mark the draft stale and require review of the new context | Ticket and draft revision comparison | | Reviewer approves twice or delivery times out | Keep one stable delivery operation; reconcile its status | Approval identity, operation key, and delivery evidence | | Model endpoint is disabled | Continue through the documented human workflow | Queue transfer and successful non-AI completion |

An application should enforce its permission rule at the last authoritative action boundary. If a downstream system cannot bind authorization and execution together, document the residual race and restrict the capability accordingly. A check performed when the draft was created does not protect a later send. Similarly, the reviewer should not be expected to reconstruct unseen retrieval failures from fluent prose. Display missing or stale evidence explicitly.

The operator closes the exercise by accounting for every ticket: drafted, rejected, superseded, sent, or pending reconciliation. A successful model response with no durable workflow disposition is unfinished work. Keep test evidence free of secrets and retain only the content needed to reproduce the decision under the approved policy.

11. Make the pilot's economic decision reproducible

The following arithmetic is hypothetical and uses invented planning inputs, not market prices or Ampity results. Suppose a pilot admits 100 tasks. Model, retrieval, and infrastructure charges total $12. Review and correction take 120 minutes at an assumed fully loaded rate of $30 per hour, adding $60. Allocated operational support adds $8. Total cost is therefore $80. If 80 outcomes satisfy the same acceptance rubric as the baseline, cost per accepted outcome is $1. The other 20 tasks remain visible as rejected or unresolved; they do not disappear from the cost numerator.

Compare that result with the existing workflow's measured cost for equivalent accepted work, not with inference charges alone. If a cascade lowers provider charges by $5 but adds 20 review minutes at the assumed rate, it adds $10 of labor and is more expensive overall. A different workload mix could reverse that decision. Keep task mix, rejected work, and review time in the comparison so a router cannot appear cheaper merely by declining difficult cases.

The process owner also checks capacity. In a second hypothetical scenario, 60 drafts arrive per hour and average review takes two minutes. That demands 120 reviewer-minutes per hour before breaks, escalations, and variability. One reviewer cannot sustain that arrival rate. The appropriate response may be admission control, a narrower task, additional approved review capacity, or no rollout. Removing required review to make the economics attractive changes the authority boundary and requires a fresh decision.

Attach observed inputs and sensitivity ranges to the pilot record. Test what happens when difficult cases become more common, retrieval misses increase, or reviewers need more time. The release meeting should produce an accepted operating range and a stop rule, not a universal savings claim. This calculation does not replace domain, privacy, security, or legal approval.

Integration record and completion checklist

Workflow / accountable owner / current baseline:
Permitted inputs, outputs, and action authority:
Sources / permission freshness / deletion owner:
Model, prompt, retrieval, and tool versions:
Acceptance rubric / evaluation evidence / known gaps:
Cost per accepted outcome / review effort:
Deadlines / quotas / queue policy / fallback:
Approval and audit record:
Disable control / rollback compatibility:
Affected-effect reconciliation procedure:
Domain, security, privacy, and legal decisions:

"The task and authority boundary are explicit, with an existing-workflow fallback.", "Evaluation covers routine, difficult, unauthorized, and unanswerable cases.", "Retrieval freshness, permission changes, and deletion propagation have tested behavior.", "Tool execution is independently authorized and duplicate-safe where required.", "Quality, review capacity, and total operating cost meet the accepted gates.", "Observable evidence, retention, incident response, and reconciliation have owners.", "Unresolved domain and legal approval remains visible before broader use." ]} />

Ampity's AI product integration is the related service for implementing an accepted workflow and operating controls. For external side effects, use the operation contract in Building Reliable Distributed Systems. The official references above support design checks, not a claim that this playbook certifies a deployment.