Should AI investigate a production incident?

Architecture decisions

An alert fires after a release. The team needs a useful hypothesis quickly, not an assistant that confidently changes production. Where should the AI boundary sit?

Illustrative worked decision

What makes the obvious answer unsafe?

Imagine checkout errors increase five minutes after a release. Logs show connection timeouts, traces show a slow dependency, and a configuration change landed at roughly the same time. An assistant can correlate those signals, but correlation is not a rollback instruction.

The useful decision is whether AI should shorten evidence gathering during the first minutes of an incident. That is different from asking it to run the incident. The commander still owns impact, severity, communications, mitigation, and recovery.

Three credible approaches.

The most automated option is not the default. Choose it only if its additional value can be measured against the simpler flow.

Approach	Fit	Cost and risk
Human-only investigation	Best when incidents are rare or telemetry is sparse	Few new failure modes, but responders manually assemble changes, traces, and timelines.
Deterministic evidence bundle	The reliable foundation	Collects bounded facts quickly. It does not explain ambiguous relationships.
Evidence bundle plus AI hypotheses	Conditional extension	May accelerate triage, but adds model latency, inaccurate synthesis risk, access controls, and evaluation work.

Where the authority changes hands.

Start with a deterministic evidence bundle: affected service and time window, current impact, relevant metrics, representative trace IDs, deploy and configuration history, and links to the original records. This remains useful when AI is unavailable.

If incident rehearsals show that interpreting this bundle is the bottleneck, add a read-only assistant. Give it a short-lived incident-scoped identity. It should return a small set of hypotheses, the evidence for and against each, what it could not inspect, and the next safe observation to make. Treat generated text as a lead to test, not a finding to publish.

Keep the recovery channel separate. The assistant has no credential that can run a rollback, change flags, edit cloud resources, or send customer communications. A human uses the existing, logged runbook and approval path. This matters most when a model’s answer sounds more certain than the evidence warrants.

Follow the decision on a phone

Alert and service impact establish the incident scope.
A read-only collector captures bounded telemetry and recent changes.
The assistant returns cited hypotheses with contradictions and missing evidence.
The commander tests a hypothesis and authorizes any recovery action through existing controls.

Illustrative investigation boundary. The assistant can propose and cite. It cannot deploy, roll back, change configuration, or close the incident.

What must remain true?

Evidence boundary

Query only named services and a bounded time window. Redact secrets and sensitive payloads. Preserve source links, timestamps, and collection errors.

Reasoning boundary

Require citations for material claims and a visible “unknown” state. Contradictory signals stay in the response instead of being averaged into a neat story.

Authority boundary

The assistant cannot invoke write-capable tools. A commander chooses the mitigation and records the decision through existing incident controls.

Fallback boundary

If retrieval or model service fails, show the raw evidence bundle and continue the ordinary incident process.

Test the decision, not just the diagram.

False lead

Rehearse a deployment that coincides with, but did not cause, the outage. The assistant must surface conflicting evidence and the commander must be able to reject its lead.

Missing telemetry

Make one log or tracing source unavailable. The response must name the gap rather than imply a complete search.

Malicious log content

Insert an instruction-like payload into a log entry. It must remain untrusted data and must never become a tool instruction.

Permission and latency

Verify denied write attempts, bounded retrieval scope, an incident-time latency target, and a human-only fallback.

Use the complete incident-response playbook

This is an illustrative design for education and review, not a verified Ampity client deployment.

Technical guidance: AWS Agentic AI Lens: distinct agent identities and least privilege; AWS Agentic AI Lens: risk-tiered human approvals; Google SRE: incident learning and evidence. These sources inform principles; they do not validate this exact design.

Which boundary is difficult in your system?

Bring the current workflow, its failure cases, and the decision your team must make. We can help compare safe options and define the evidence for production.

Discuss your architecture