Onboard an AI Employee: Role Pack, Tests and Safe Operations

Onboard an AI employee with a reusable role pack, scoped access, isolated tests, acceptance evidence, routine authority and owned recovery procedures.

Outcome and operating scope

Use this playbook to turn an agreed business job into a role that can be evaluated, granted limited production authority and operated by named people. An AI employee here is an automated software role. It does not inherit an employee's unrestricted access or decision-making authority.

The working example is an illustrative delivery-brief coordinator. It reads permitted project records, prepares a sourced internal brief and raises exceptions for missing or conflicting information. After acceptance, routine internal posting can run independently. Changing customer commitments remains protected. Substitute another job only after rewriting the inputs, acceptance criteria and action policy for its consequences.

The AI agent development playbook covers implementation mechanisms in greater engineering detail. This procedure supplies the business and operating evidence those mechanisms must satisfy. Business acceptance does not replace security or integration tests, and technical test success does not establish that the output is useful.

The actions below are proposed checks. This document does not claim that a customer rollout, connector test or recovery drill has already been executed.

1. Assign owners and the first decision gate

The business owner identifies the intended result and the work that remains outside the role. The data owner approves the purpose and handling of each source. The authorized technical grantor configures the actual identity and platform permissions. The implementer builds and tests the role. An acceptance reviewer judges evidence against criteria agreed before the final evaluation. The operator handles routine operation and the exception route.

These responsibilities can overlap in a small team. Record that overlap rather than assigning fictional independence. Where a failure has high consequences, arrange a reviewer who can challenge the implementation and protect a held-out acceptance set.

The first gate is a scoped commissioning decision. The owner accepts the job definition and evidence plan, not a production release. If required data cannot be used lawfully or under the company's policies, or nobody can judge the result, pause commissioning and resolve that gap. A vendor demonstration cannot substitute for those decisions.

The exception owner needs a documented way to pause work and reach the right administrator or engineer. Do not require the business owner to hold broad technical privileges merely to make a stop decision.

2. Complete the reusable role pack

Keep the pack in customer-controlled storage available to the people responsible for operating and ending the role. Use links to secret-manager entries and approved evidence stores, never raw credentials or confidential examples pasted into the pack.

For the example role, the output could be an internal brief listing verified commitments, source references and unresolved blockers. The launch authority could permit reading approved project records and posting only to the agreed internal destination. The specification should state what happens when a commitment has no owner. It must not invent an owner or quietly omit the item.

Replace arbitrary counts with the condition that matters. “Flag an unverifiable commitment” is more defensible than “produce a brief if three commitments were found.” An account with no open commitments may still have a valid result, while one missing critical commitment may invalidate an otherwise long brief.

3. Map data and separate staging from production grants

Create an access record for each environment. Staging and production identities, destinations and credentials should be distinct where the platform supports that separation. A successful staging login does not establish the production identity's effective permissions.

Identity and consent requirements differ by platform. For example, Microsoft's permissions and consent overview distinguishes app-only permissions from delegated access. This is why a business approval, tenant consent and runtime authorization should not be collapsed into one checkbox.

Start the isolated evaluation with the permissions it needs. Production authority is a later decision. If a test needs a write, grant that operation to a safe test target, not a broad production identity. The access-boundary article provides a separate platform-focused discussion.

4. Build fixtures and protect acceptance evidence

The acceptance reviewer and business owner select representative cases from permitted redacted history and clearly labeled synthetic or adversarial scenarios. Historical records establish actual work variation. Synthetic cases can exercise rare failures without exposing sensitive data or waiting for a live incident.

Keep fixture provenance and handling explicit. Record whether a case is historical, adapted or invented; what was removed; whether it may be sent to the chosen provider; and when its local copies must be deleted. Treat generated outputs and logs as data copies subject to the same handling review.

Use development fixtures to refine prompts, tool rules and formatting. Keep a separate held-out acceptance set, or holdout, controlled by the reviewer. Do not repeatedly tune against that set and then report its passing result as independent evidence. If a failed acceptance case becomes a development case, record the change and replenish the missing coverage. Google's dataset division guidance explains the underlying evaluation-separation issue; the appropriate coverage for an operational role remains workload-specific.

No fixed number of examples establishes sufficiency. Describe coverage gaps, including meaningful workload changes and rare high-consequence cases.

5. Confirm isolation and safe test sinks

The implementer verifies the test environment before enabling triggers. Use synthetic identifiers and test records. Inspect every outbound path, including email, chat, webhooks, workflow automations and scheduled follow-up processes that a test write might activate.

A private channel is still a delivering destination. It can notify members, trigger integrations or retain data. A non-delivering sink means outbound effects are intercepted or disabled and that behavior has been verified. If you use a dedicated test channel instead, explicitly authorize its testers and confirm that notifications, forwarding and downstream automations have an accepted containment boundary.

Where a vendor has no suitable sandbox, use a mock connector or controlled test double for the dangerous operations and document the fidelity limitation. A later production check needs its own authorization and a minimally consequential target. Do not quietly describe a local simulation as proof of the live integration.

The gate requires evidence that test actions cannot reach an unintended recipient or production record, that the operator can stop the test, and that generated artifacts remain in the approved environment. If the isolation boundary is uncertain, do not run an external action merely to see where it goes.

Reference handoffs: grant limited production authority only after the relevant evidence is accepted. If operation must stop, reconcile actions already started or completed. Removing access alone cannot undo their effects.

6. Exercise routine, refusal and uncertain outcomes

Run the agreed development cases in isolation, recording the configuration version, fixture identifier, source evidence, expected action, observed action and result. Confirm output correctness and authorization separately. A polished brief sent to the wrong destination fails even when every sentence is correct.

Test explicit refusals as well as successful work. A role must not treat a retrieved note, message or document as permission to widen its scope. The implementer should enforce resource and action checks outside free-form model judgment. OWASP's excessive-agency guidance supports minimizing tool capabilities and checking authorization at the execution boundary.

For write-capable roles, test the ambiguous case where the target may have accepted a request but the caller lost the response. A timeout does not prove the action failed. Check the destination using a safe request identifier or reconciliation method before retrying. The Amazon Builders' Library discussion of idempotent APIs explains why repeat attempts can create duplicate effects when intent is not tracked.

The connector's actual retry contract matters. Record whether it supports an idempotency key, how that key is scoped and retained, and what evidence distinguishes a new intended action from a retry. Do not invent an idempotency field when the destination does not support one. If the outcome remains unknown, hold dependent work and give the exception owner an explicit reconciliation task.

7. Run held-out business and technical acceptance

After development findings are resolved, the acceptance reviewer runs the protected cases against the release candidate. Freeze the relevant configuration for that evaluation. A material prompt, policy, model or connector change after the run invalidates the affected evidence and requires appropriate retesting.

Evaluate output usefulness, source fidelity, privacy, action authority, exception handling, operational capacity and recoverability. Use the criteria agreed for this job. A reduced correction rate can support the usefulness decision, but it cannot compensate for an unauthorized action, omitted critical commitment or disclosure to the wrong recipient.

For the briefing example, acceptance might require accurate due dates and owners, explicit treatment of conflicting records, source references the reviewer can inspect, delivery only to the allowed destination, and successful stop/escalation checks. These are proposed criteria to tailor, not a universal certification test.

Record failed and inconclusive cases alongside passing ones. A test that could not reach the destination is not a successful negative-authorization test unless the reason for denial is established. Similarly, an alert shown in an application log is not evidence that its intended owner received it. Check the actual authorized notification path during the drill, using test content.

NIST's Generative AI Profile discusses pre-deployment testing and ongoing risk management. The decision record below applies that lifecycle perspective to one bounded role without claiming that a finite test suite proves complete safety.

8. Decide limited production authority

The business owner, authorized grantor and operating owner review the candidate's evidence and explicitly decide what it may do in production. Record allowed resources, operations, recipients, volume or workload bounds where appropriate, protected decisions and conditions that suspend authority.

Shadow or read-first observation can help compare results before enabling consequential actions. Treat it as evidence about the observed task, not a write-path test. If the role will post or update records, test that action within a safe boundary before releasing it. Routine independent execution is appropriate only for the actions the owners have accepted; protected actions continue to require their defined approval.

A proposed authority record is reusable:

Do not add a universal observation period or percentage target. Match observation and acceptance to workload variation and consequences. Group related permission changes only if their combined behavior can be tested and attributed. A calendar milestone does not authorize new access.

9. Observe production without making review the hidden job

The operator checks execution receipts and agreed quality samples, watches the exception queue, and verifies that alert delivery is working. Sampling is a monitoring choice, not a substitute for enforced authorization on every action. Preserve oversight for protected decisions and escalate evidence of boundary failure promptly.

Measure accepted results, corrections, incomplete work, unknown external outcomes and the time people spend resolving them. Define denominators. A “successful run” that produced an unusable brief should not be counted as accepted work. Do not hide review and support labor when comparing the role with the previous process.

Match operating coverage to expected activity. A role scheduled outside office hours needs an agreed response arrangement for the events that cannot wait. If an owner is unavailable or the queue exceeds the accepted handling capacity, the fallback may be a controlled pause or a narrower task rather than unattended accumulation.

The accepted-result cost tool helps examine the same workload across sourcing options. Replace estimates with observed values when available, and retain what remains uncertain. No productivity or cost improvement is established by this playbook itself.

10. Stop promptly and reconcile effects

Use this procedure when the agreed stop condition is met. A single high-consequence event can be sufficient. Do not wait for a trend when there is evidence of unauthorized access, harmful disclosure or continuing unsafe actions.

Do not universally let in-progress runs finish before restricting access. For an active compromise or harmful write path, prompt containment may take priority. For an otherwise safe transaction, controlled draining may be appropriate. The incident owner and technical responder choose the containment method based on what is running, cancellation support and consequences.

Stopping dispatch does not necessarily cancel an already running request. Credential revocation may have platform-specific propagation and cached-session behavior. Verify its effect rather than claiming an immediate universal authentication failure. Revocation does not reverse a completed write or unsend a message. Stopping dispatch cannot undo those effects either.

If the destination cannot establish whether an action completed, record its state as unknown, hold dependent actions and assign investigation. Compensation is an owned business and technical decision. Do not automatically delete a record as an “undo” when another person or process may have changed it since the role acted.

11. Handover, change and exit

Handover includes the current role pack, production access inventory, acceptance record, configuration versions, operating limits, support contacts, recovery procedure and known gaps. Ensure the customer can access these materials without depending on one supplier's private workspace.

Walk the operator through a routine receipt, an exception, an unknown action outcome and the stop route. The operator may invoke containment through named administrators rather than personally performing every technical step. Record who actually demonstrates each control and what evidence was checked.

Define which changes require review: a new destination, expanded data scope, a changed model or tool contract, a new business policy, or different workload consequences. Not every text correction requires the same review depth, but changes that affect authorization or accepted output need relevant regression evidence.

At exit, stop triggers, settle or assign outstanding work, export agreed records, remove access through authorized owners, and apply retention and deletion policies to prompts, logs, summaries and cached content. Verify completion for each system. Keep necessary incident or contractual records according to the approved policy rather than promising that all copies disappear instantly.

12. Close the onboarding record

The acceptance owner records the decision as accepted within scope, rejected, or not yet established. Keep the source versions, evidence references and limitations with that decision. A passing demonstration with an unresolved production grant or notification path is not completed onboarding.

• The role pack defines an accepted result, routine authority, protected decisions and prohibited actions
• Business/data approvals and technical staging/production grants are recorded separately
• Fixtures are permitted and labeled, with held-out acceptance evidence protected from repeated tuning
• Outbound test sinks and test records have a verified isolation boundary
• Routine, negative, adversarial, duplicate and uncertain-response cases have owned outcomes
• Production authority is tied to the accepted candidate and job-specific evidence
• The operator and backup can invoke stop, reconciliation and technical escalation
• Completed effects and unknown outcomes have named compensation or investigation owners
• Customer-controlled handover, change and exit records are available

For preparation before commissioning, use Prepare your company for an AI workforce in 2027. For the delivery-model decision, use the platform, internal build and managed-role comparison. Start the next review with the first unresolved entry in this onboarding record, rather than adding authority to a role whose operating evidence is still incomplete.

Primary references

Checked on 16 October 2026. These sources inform particular principles; they do not prescribe this exact role pack or certify a deployment.

• NIST AI 600-1, Generative Artificial Intelligence Profile: pre-deployment testing and lifecycle risk management.
• OWASP LLM06:2025, excessive agency: restricted capabilities, permissions and independently enforced authorization.
• Amazon Builders' Library, making retries safe with idempotent APIs: ambiguous responses, duplicate effects and explicit request intent.
• Microsoft identity platform, permissions and consent overview: delegated versus app-only access and consent.
• Google Machine Learning Crash Course, dividing datasets: keeping development and evaluation evidence separate, adapted here to application acceptance.