A Missing S3 Replica: Classify the Object Version Before Replaying It

Separate live-replication eligibility, pending or failed status and destination-version evidence before proposing an S3 Batch Replication job or another upload.

Primary sources checked

When an S3 destination appears to be missing an object, identify the exact source version and its replication status before changing the rule or uploading it again. A version created before live replication was configured, a currently pending version and a failed version require different investigations. Looking only at the latest destination key can also hide an older replica that still exists.

This guide produces a version-bound diagnosis and an owned next action for one live-replication rule between two versioning-enabled general purpose buckets in the same AWS account. The destination may be in another Region. Worked records use ordinary data versions and SSE-S3. Multiple destinations, replica modification sync, cross-account ownership, delete-marker propagation, Object Lock, directory buckets and actual replication jobs need separate review. SSE-KMS is an explicit handoff branch below, not an implied equivalent of the worked encryption path. All examples are constructed; no AWS requests, uploads or recovery jobs were executed.

Resolve which version is supposedly missing

Keep source bucket owner, Region, key, version ID and observation time together. A key name is not a version. A later writer can change the current source or destination object while the incident record still refers to an earlier upload. Capture source creation time and the historical replication configuration applicable to that creation, if available.

Identify the intended destination bucket and Region from the actual rule. A console tab, nearby bucket name or environment variable can point to another account or deployment. Require an approved collector and restricted evidence location; bucket/key names and tags can themselves reveal customer information. This procedure does not require downloading payloads.

For the destination, inspect existing version-aware evidence for the selected key. S3 replication retains source version IDs as metadata. Match the selected version in the intended destination rather than choosing whichever version looks closest in time. An ordinary object copy or a new upload is a different operation and must not be substituted for replication evidence. If the required version-aware read is unavailable, keep the destination finding unknown. AWS replication use cases.

A successful current-object read cannot establish whether all prior versions were replicated. Conversely, a current delete marker or later write can change what an unversioned read returns without proving that an earlier replica never existed. Hand marker and retention questions to the appropriate storage owner before proposing to copy or delete anything.

Read replication state in its applicable context

S3 documents source replication states PENDING, COMPLETED and FAILED; a normal replica reports REPLICA. Eligible source-object metadata can include the replication-status header. Missing status is not a returned FAILED value. Replication-status semantics.

Obtain existing version-specific metadata or an approved bounded metadata read. A denied read supplies an authorization gap, not evidence that the object or replica is absent. Preserve the request version and response capture time. A later status change should be a new observation, not an overwrite of the earlier incident row.

The single-destination scope is deliberate. With multiple destinations, the source status aggregates their progress; it does not identify one particular failing destination by itself. Replica modification sync also changes which states a replica can report. Stop this worksheet at either configuration mismatch and use a destination-specific review rather than applying the simple interpretation to a different topology.

COMPLETED describes replication progress, not application readability, legal recovery sufficiency or every retained copy. The destination consumer still needs its intended version, storage-class availability and access. This guide stops before payload or application recovery testing. Those need the data owner's separate scope and permission decision.

Check eligibility before treating absence as failed replication

Read the enabled rule's filter, destination, priority and relevant encryption selection. Confirm versioning on both buckets and the actual replication role. AWS's replication requirements distinguish these prerequisites. A valid present configuration is not proof of the configuration that applied when the object version was created.

An object outside the rule's prefix or tag scope should not be treated as a broken in-scope copy. Historical tags or filters may be unavailable. Mark the eligibility question unresolved when a current tag snapshot cannot reconstruct the relevant history. Do not modify tags as a diagnostic shortcut; doing so changes the case being investigated.

Live replication applies to new eligible objects after configuration. Existing objects can be handled through a separately authorized S3 Batch Replication operation. The Batch Replication guide covers pre-existing, previously replicated and failed objects. A new live rule is not evidence that an earlier population was backfilled.

If the rule was enabled at 10:00 UTC and the selected version was created at 09:30, preserve that timing rather than waiting indefinitely for a live-replication promise the configuration did not make. Confirm that the version was not already selected by an earlier applicable rule. The useful disposition is a historical-population review, not an invented service failure.

Distinguish pending work from a failed version

AWS's replication troubleshooting guidance directs pending and failed objects to different checks. Temporary resource unavailability can leave a version pending; permission problems can produce a failed status. Neither state supplies a complete root-cause diagnosis without applicable configuration and error evidence.

For pending work, retain status over time, destination availability evidence and any existing replication telemetry. State the time since the relevant observation, not an unsupported universal completion deadline. A dashboard sample does not establish that every object is pending or that the oldest selected version is represented.

For a failed version, inspect existing replication failure events or other approved error records. Replication failure events can add failure-reason and replication-rule context when configured. Enabling notifications now cannot reconstruct an event that was never captured. Do not call missing notifications proof that no failure occurred.

Correcting a permission problem does not by itself retry a version already marked failed. AWS identifies Batch Replication or a new upload as ways to initiate work for failed objects. Either is a mutation with its own consequences. This diagnosis prepares the owner to choose a recovery route; it does not start one or promise that the next attempt will succeed.

Route encrypted-object findings to the right owner

The worked scope uses SSE-S3 to keep status classification separate from customer-managed key behavior. If the selected source version uses SSE-KMS or DSSE-KMS, capture that fact and the exact key references without collecting key material. The rule must explicitly select the relevant encrypted objects and specify an applicable destination key. Encrypted-object replication requirements.

A destination's default encryption setting is not sufficient proof that a customer-key-encrypted source version will replicate using that setting. Ask the storage and key owners to review the version's encryption mode, replication selection, destination Region/key validity and effective role/key/bucket authorization. S3 Bucket Keys and cross-account keys introduce additional context requirements. Do not turn a short action list into a universal deployable policy.

AWS also documents that accepting a replication configuration request does not validate the supplied KMS key's usability. Preserve an accepted configuration and a failed object as separate observations. Replacing encryption or widening a key policy merely to get a green status can change the data-protection boundary and belongs to a new approved change.

The KMS change guide addresses rotation, ciphertext migration and key retirement. Replication is not proof that the source key can be retired: earlier versions, retained backups and other consumers may still depend on it. Keep this diagnosis bounded to the selected version and destination.

Worked scenarios: pre-existing, pending and failed versions

Assume one fictional source prefix exports/, one destination and an enabled rule first established at 10:00 UTC. All three rows are stipulated teaching inputs, not S3 exports. No previous rule selected the pre-existing version.

Version recordStipulated evidenceDisposition
A, vA created 09:30Correct prefix; no replication status; predates first ruleExisting-object population; review a bounded backfill, not a failed live attempt
B, vB created 10:10Rule selected version; status PENDING at 10:12 and 10:20Pending observation; collect destination/replication evidence, no completion-time promise
C, vC created 10:15Rule selected version; status FAILED; matched failure record identifies role denialResolve the scoped authorization issue, then review an explicit retry mechanism
A, vA created 09:30
Stipulated evidence: Correct prefix; no replication status; predates first rule
Disposition: Existing-object population; review a bounded backfill, not a failed live attempt
B, vB created 10:10
Stipulated evidence: Rule selected version; status PENDING at 10:12 and 10:20
Disposition: Pending observation; collect destination/replication evidence, no completion-time promise
C, vC created 10:15
Stipulated evidence: Rule selected version; status FAILED; matched failure record identifies role denial
Disposition: Resolve the scoped authorization issue, then review an explicit retry mechanism

For A, changing the rule name or waiting another hour does not establish that vA entered live replication. The owner needs a version-aware candidate population for a possible backfill. Other versions under the same key may belong to another population; a current-key-only manifest would lose that distinction.

For B, the invented eight-minute observation interval is a duration, not a service breach. There is no RTC commitment in this fixture. If the destination owner has no matching metadata, keep both facts. Do not reset the source version's history with a fresh upload just to see a new status.

For C, stipulate that the role evidence confirms the reported restriction applied during the attempt. A reviewed correction would address that restriction, but the failed version remains a separate recovery item. If the evidence instead names a different role or rule revision, the causal join fails and C must return to unknown cause.

A fourth counterexample is useful when reviewing the packet: the selected destination replica exists as a noncurrent version beneath a newer object. An unversioned lookup could support “not the current value,” but not “never replicated.” The next action changes from initiating a copy to resolving which version the consumer is supposed to use.

Prepare a recovery choice without initiating it

Waiting with observation preserves the original version and may be appropriate for pending temporary conditions, but cannot solve a known failed historical attempt. Batch Replication can address an explicit existing-version population, but requires a reviewed manifest, existing replication configuration, appropriate job role and completion-report handling. A new upload creates a new source version and may trigger downstream processing; it is not an identity-preserving retry of the old version.

Compare those alternatives against business meaning. Does the application refer to a version, the latest key or a separate catalogue? Can another upload duplicate notifications or activate a consumer? Is the source version still available? Who can approve request/storage/transfer cost and destination effects? If those answers are missing, stop at the recovery proposal.

Batch Replication also has limits. The cited Batch guide excludes re-replicating versions permanently deleted from the destination through a version-specific delete; its suggested copy-based alternative creates new source versions. Do not label Batch Replication a universal recovery tool. Lifecycle interaction and deletion policy require separate owner review, not an instruction to disable retention during diagnosis.

Keep the version and recovery worksheet readable

Decision fieldEvidence or explicit unknown
Exact sourceOwner/account/Region, bucket/key/version, creation time and observation time
Applicable ruleHistorical enabled revision, filter, one destination, versioning and replication role
Observed stateVersion-specific status and captures; existing failure/telemetry reference
Destination comparisonIntended bucket, returned version identity and current/noncurrent distinction
Recovery proposalWait, bounded Batch review, new-version review or unsupported-path handoff, with costs and downstream effects
ClosureFacts established, cause still unknown, assigned owner, next read-only evidence and change approval still required
Exact source
Evidence or explicit unknown: Owner/account/Region, bucket/key/version, creation time and observation time
Applicable rule
Evidence or explicit unknown: Historical enabled revision, filter, one destination, versioning and replication role
Observed state
Evidence or explicit unknown: Version-specific status and captures; existing failure/telemetry reference
Destination comparison
Evidence or explicit unknown: Intended bucket, returned version identity and current/noncurrent distinction
Recovery proposal
Evidence or explicit unknown: Wait, bounded Batch review, new-version review or unsupported-path handoff, with costs and downstream effects
Closure
Evidence or explicit unknown: Facts established, cause still unknown, assigned owner, next read-only evidence and change approval still required

An approved later rehearsal needs disposable versioned buckets, synthetic content, a narrowly scoped role, inert consumers and a cost/retention boundary. Record expected version relationships before any mutation. Preserve the failed attempt and compare only the approved population after any recovery operation. A job's overall completion should not hide per-object failures or omitted versions.

Stop if evidence identifies another account, multiple destinations, ambiguous version history, a denied required read or an unexpected downstream effect. Reverting a rule cannot remove every replica or undo external processing. Do not clean up historical versions to make the comparison pass. Retain the original and its access controls until the owner decides what can be retired.

Next action: classify one version before selecting a job

Have the storage owner complete the exact source, rule and status fields for one missing-version report. Ask a second operator to state whether it is pre-existing/out-of-scope, pending, failed or unresolved without guessing from the object name. Then assign the smallest missing evidence request and, only where justified, a separate version-specific recovery proposal.

For NFS round-trip recovery, use the archive semantics guide. For retained versions awaiting deletion, use the noncurrent-expiration diagnosis. Those questions concern filesystem meaning and lifecycle eligibility respectively; a replication status does not settle either.

Related services