Infrastructure as Code Maturity: A Change-Safety Assessment
Assess infrastructure-as-code maturity through plan integrity, state protection, change authority, drift handling and partial-apply recovery.
A mature infrastructure-as-code workflow lets a team explain what will change, authorize that exact change and recover when only part of it succeeds. A repository of configuration is useful evidence, but it does not answer those questions on its own.
Assess the workflow used for a real production change. Include the cloud account, state backend, credentials, approval process and people who can change the system outside the pipeline. A small system with a careful manual approval step can be operated safely. It does not need a self-service platform to earn that description.
This guide uses Terraform examples. Check the behavior of your selected tool, provider and backend versions before applying the same controls elsewhere.
Replace a maturity score with observable evidence
For each capability, mark demonstrated, untested or missing. Link the evidence and name the person responsible for the gap. Avoid combining the results into a score that hides a critical weakness.
| Capability | Evidence to request | | --- | --- | | Known ownership | Each managed resource has one declared source of configuration and an accountable owner | | Reproducible inputs | The change identifies the commit, module and provider versions, variables and target environment | | Plan integrity | The approved change is tied to the artifact that will be applied | | Protected state | Access, encryption, recovery and concurrent-writer behavior have been tested | | Controlled execution | Production credentials and approval authority are limited to the intended workflow | | Drift handling | Out-of-band changes are detected, explained and reconciled deliberately | | Recovery | A partial failure exercise shows how the team finds and repairs the actual state |
A successful low-risk deployment proves less than a successful exercise involving a failed apply or an emergency change. Collect both. An untested recovery process should remain visibly untested even if normal deployments are fast.
Define the change boundary before splitting repositories
State boundaries should follow ownership, lifecycle and acceptable impact. A single state that includes unrelated products makes every apply a broader decision. Splitting every resource into its own state can create coordination and dependency problems.
For a production service, identify which resources must change together and which are shared. Keep dependency outputs explicit and protect them from accidental changes. Assign responsibility for shared DNS, networking and identity resources rather than letting several teams manage the same object.
Separate environments through the relevant account, credentials and backend controls. A different workspace name is not a complete security boundary if the same identity can modify every environment.
For brownfield resources, inspect and import them under an approved adoption procedure. Confirm that the resulting plan represents the intended ownership and does not propose unexpected replacements. Do not use the first production apply as a discovery tool.
Approve the plan that will be executed
A pull request can show a speculative plan that differs from the final plan. Terraform's plan documentation distinguishes speculative plans from saved execution plans and warns that plan files can contain sensitive values.
A controlled workflow should record the commit, dependency versions, variable source, backend and target account. Produce the execution plan under the intended identity. Review its creates, updates, replacements and deletions, including indirect effects such as changed routes or access.
Bind approval to an immutable artifact identifier or digest and its execution context. Limit who can replace or download it. If the plan must be regenerated because inputs, state or circumstances changed, obtain approval for the new plan. A screenshot from an earlier run is not approval for a later one.
When a saved plan is supplied, the apply command executes that plan without the normal interactive approval prompt. The approval gate therefore belongs before the authorized runner invokes it. Treat the artifact store and runner as part of the production change boundary.
An ordinary plan is not proof that a cloud provider will accept every operation or that the application will remain healthy. Add policy checks, integration tests and service-specific acceptance checks where they address a real failure mode.
Protect state and serialize writers
Remote storage does not automatically provide locking, limited access or usable recovery. Terraform's locking documentation makes support dependent on the backend. Test how a second run behaves while another run owns the lock.
For the S3 backend, the current HashiCorp documentation describes opt-in lockfiles through use_lockfile and marks DynamoDB locking as deprecated. Match configuration and permissions to the Terraform versions your runners use. Do not assume that storing state in a bucket enables a lock.
Keep state and saved plans out of public repositories and unrestricted logs. Protect backend credentials, encryption keys and version history. The person who can read state may be able to read sensitive values even if the terminal output hides them.
A stale lock requires investigation. Confirm that no active process is writing before using an authorized unlock procedure. Forcing an unlock to make a queue move can permit conflicting changes.
A state backup helps recover bookkeeping. Restoring an old state file does not undo changes already made in the cloud, and it can leave Terraform with an incorrect view of current resources.
Worked exercise: a routing change fails halfway
Consider a fictional service moving traffic to a new worker group. The old group is healthy. The approved plan creates the new group, configures its alarm and updates routing only after health checks pass.
The provider creates the group and alarm, but rejects the routing update because the runner lacks a required permission. The apply exits unsuccessfully. Some resources now exist even though the overall run failed.
| Observation | Operator decision | | --- | --- | | Old workers still serve traffic | Preserve that known-working path while investigating | | New group exists | Check health, cost and whether it received any work | | Routing update failed | Confirm the live route, rather than assuming no change occurred | | State and provider records disagree | Identify resource IDs and use an approved reconciliation procedure | | Cause is understood | Produce a fresh plan for completion or cleanup and review its effects |
Record successful and failed operations, resource identifiers and the current state version. Prevent another pipeline from racing the investigation. Check the provider's actual state because a request can succeed remotely even if the client loses its response.
The team might approve the missing permission and finish the change. It might instead remove the unused group. Either choice requires a new assessment of the live system. A Git revert alone is not a tested rollback, particularly when an earlier operation changed data or deleted a resource.
If state repair or import is required, use the documented tool workflow under restricted access. Do not invent entries or restore old state merely to make a plan look clean. End the exercise by verifying traffic, alarms, resource ownership and the absence of unintended capacity.
Distinguish CI execution from GitOps reconciliation
A pipeline that runs after a merge is CI-triggered execution. The OpenGitOps principles also describe desired state that agents pull and continuously reconcile. Calling every merge-and-apply workflow GitOps obscures operational differences.
For a one-shot apply, define how drift is detected between runs. For a reconciling controller, define how emergency changes interact with its loop. A responder's manual mitigation can be immediately reversed if the controller still sees a different desired state.
Document the approved pause, emergency-change and resume procedure. After an incident, decide whether the changed live configuration should become the new desired state or be reverted. Reconciliation should follow that decision rather than erase it.
Keep policy exceptions specific and temporary
Policy checks can identify broad access, unsafe deletion, unapproved regions or missing ownership. They only enforce what they express and what the pipeline cannot bypass.
An exception should name the resource, risk, approving owner, compensating control and expiry condition. Keep it linked to the change. Avoid a permanent global skip because one provider resource needs special treatment.
Test failure behavior for the policy service itself. Decide which operations must stop if validation is unavailable and how urgent recovery is authorized. A security gate that nobody can explain during an outage invites informal bypasses.
Self-service becomes useful when recurring requests have stable interfaces and tested boundaries. Provide a narrow catalog with ownership, quotas, lifecycle and failure reporting. Keep unusual or high-impact changes on a reviewed path. Measure whether the catalog removes real waiting rather than treating its existence as the final maturity level.
Choose the next improvement from a real gap
Review one recent change and one failed or interrupted run. Can a second engineer reconstruct the approved intent, actual mutations and recovery decision without the original operator?
If not, improve that missing capability first. A plan-artifact boundary, a locking test or a recovery exercise may be more valuable than a new platform interface.
Ampity's infrastructure-as-code service is a relevant next step when you need to assess or implement that workflow. Bring the state boundaries, an anonymized plan, runner permissions and a failed-change timeline. Keep credentials and sensitive state values out of the initial material.