Upgrade the OpenSearch Domain or Rebuild the Index?

A per-index evidence register for choosing a managed OpenSearch domain upgrade, parallel-domain migration or hold without mistaking a compatible engine or copied...

Decision brief: decide per index before deciding per domain

A managed domain running OpenSearch 2.19 can still contain an index created by Elasticsearch 7.10. The current engine version does not establish that every retained index can enter OpenSearch 3.x. Moving that index to cold storage does not remove its lineage. A parallel domain avoids changing the current endpoint immediately, but it creates another problem: proving that the target represents the required source state when consumers switch.

Authorize one of three outcomes: remediate and upgrade the existing domain, build a separately validated compatible target, or hold the change. Do not let a supported upgrade path decide the data-transfer method, and do not let a completed copy decide consumer acceptance.

This paper covers Amazon OpenSearch Service managed domains, concentrating on a proposed 2.19-to-3.x transition. It excludes OpenSearch Serverless, self-managed cluster procedures, search-relevance optimization and a general disaster-recovery drill. Exact target minor version, Region availability and domain configuration remain inputs, not universal defaults. Vendor contracts were checked on October 7, 2026. All populated examples are fictional and NOT EXECUTED. No AWS account was inspected, domain changed, snapshot restored, workload benchmarked or customer outcome measured.

The deliverable is a per-index transition register and a domain-level decision signed through the reader's actual change process. A missing creation-version record or unproven replay window is a hold with an owner, not an invitation to choose the most convenient migration tool.

Separate engine progress from index lineage

AWS distinguishes engine upgrades from service-software updates. Its managed-domain upgrade guide currently requires domains on 1.3 or 2.x to reach 2.19 before entering 3.x. It also requires reindexing indexes originally created in OpenSearch 1.3, Elasticsearch 7.10 or earlier before 3.x, including hot, UltraWarm and cold indexes. For incompatible warm or cold indexes, the documented path moves them to hot storage for reindexing before returning them to the intended tier.

*Engine history and index history are different evidence tracks. The figure describes admission requirements, not an executed upgrade or an automatic index-format conversion. Reindexing creates a new index whose recorded lineage must be inspected.*

That is a compatibility requirement, not proof that the reader has capacity or authority to perform it. Inventory every retained index, including inactive and archived indexes, before scheduling the change. Record index UUID, name, captured creation-version evidence, current tier and capture time. Do not substitute an index-name date, a restored-domain engine version or a client-compatible version response for creation evidence. When the evidence cannot be obtained, preserve UNKNOWN and have the platform owner identify a supported inspection method.

For an authorized inspection, the upstream Get Index API documents index settings, mappings and aliases. Its flat response includes index.uuid and the encoded index.version.created. Preserve the raw value and a verified version interpretation together; do not decode it using guessed decimal slicing. Record the exact requested index set. Default wildcard expansion covers open indexes, so a default response is not proof that hidden or closed indexes, or indexes in every managed storage tier, were inventoried. Reconcile those inventories separately with supported tier inspection evidence. A denied read is UNKNOWN, not an absent index.

Keep mapping, analysis configuration, relevant settings and installed/custom packages alongside lineage. AWS identifies four settings that can block the 2.19-to-3.x precheck: index.knn.algo_param.ef_construction, index.knn.algo_param.m, index.knn.space_type and index.store.hybrid.mmap.extensions. Their appearance requires version-specific remediation and validation, not blind deletion. Capture the before state, proposed replacement behavior and query fixtures before changing a setting. A newly created index can still have incompatible settings or an unsupported dependency.

The GetCompatibleVersions API provides source-to-target upgrade mappings and accepts an optional domain name. Retain the actual authorized response for the intended Region and domain, with capture time. Documentation examples are not an availability result for the reader's account. If the exact proposed target is absent, hold that proposal rather than infer availability from the phrase “3.x.”

Existing cross-cluster connections are a separate dependency from a proposed remote-reindex transfer. AWS requires maintained source/destination compatibility after an upgrade and warns that active replication cannot resume after its connection is deleted. Inventory connection identity, direction, use, remote/current/proposed versions and compatibility evidence with the dependency owner. An evidenced empty inventory can be not applicable; a denied read cannot. Connection deletion is outside this paper's authority, not a default way to clear an eligibility failure.

Admit the archived-index remediation record

For fictional I1, “move to hot and reindex” is a goal, not a single cold-to-hot request. The cold-storage guide documents attachment from cold to UltraWarm; the UltraWarm guide then documents returning warm indexes to hot. Keep separate evidence for each transition. A queued or acknowledged request is not proof that the index reached the intended tier. Retain the cold inventory's identifier alongside the later index UUID and names so a handoff does not confuse cold metadata with engine creation-version evidence.

Build the remediation record around the peak simultaneous footprint, not merely I1's cold-storage size. Hot admission must cover the returned source, its actual replica configuration, the newly reindexed destination, temporary work and the live domain's required reserve. UltraWarm's documented hot return has one replica; inspect the resulting settings instead of assuming the former warm footprint predicts hot allocation. Do not attach every archived index at once because the domain has enough cold storage.

The following is a filled fictional planning review, not a migration result. No capacities, lookup results or snapshot contents have been measured. It shows what would reject one batch even before a data operation is authorized.

Snapshot coverage needs particular care here. A default manual snapshot excludes warm indexes; AWS documents separate restrictions for explicit warm-index snapshots, including one warm index per snapshot and no hot/warm mixing. Manual snapshots restore to hot, whereas cs-ultrawarm restoration targets warm. The name “full snapshot” is therefore insufficient recovery evidence. Record repository, included index, snapshot engine/status and the actual restoration tier before budgeting recovery capacity. The same index name in two records does not prove they contain equivalent data.

The remediation packet's output is an admitted batch envelope: enumerated indexes, transition order, maximum simultaneous hot occupancy, snapshot coverage, lookup acceptance and authorized disposition. If R-I1 remains FAIL or UNKNOWN, do not begin its tier transition to discover whether capacity happens to suffice. Reconsider a smaller verified batch, a funded capacity change or a separately compatible parallel path. A batch change requires its own resource and authority evidence; it does not automatically cure the recovery-copy or retention gaps.

Compare four paths against the same retained obligations

Choose a method for each index, then reconcile the methods into a domain decision. Some indexes may be suitable for snapshot transfer while others need reconstruction. A mixed plan is not inherently wrong, but all indexes required by a consumer must meet the same acceptance boundary before that consumer switches.

Defer is a fifth decision, not a transfer mechanism. State which obligation blocks the change, who owns it, the next evidence date and the operational exposure accepted meanwhile. Do not claim indefinite version support or a free extended-support period. Any support/calendar/cost decision requires its own current service evidence.

Avoid selecting parallel migration solely because it sounds reversible. A still-running old domain can be stale after new writes reach the target. Repointing consumers without handling those writes is a different data-loss decision. Conversely, in-place remediation may consume substantial temporary hot-tier headroom even without a second domain. Both need measured inputs and a spend owner.

A snapshot preserves a baseline, not a compatible future

AWS separates automated and manual snapshots. Automated snapshots are for cluster recovery in service-owned storage; manual snapshots support owned recovery or migration in an S3 repository. A normal cross-domain migration plan should not assume it can directly use a service-owned automated snapshot. Preserve snapshot identity, creation engine, included indexes, status and capture window.

Use the managed restore guide to qualify the target, including custom analysis packages, allocation settings, system-index limitations and replica settings for the actual deployment configuration. PARTIAL is not complete data evidence. An index-name collision is not permission to delete the target index. A separately named isolated restore can avoid destructive replacement, subject to an approved plan and supported rename parameters.

Upstream snapshot compatibility guidance describes one-major-version forward compatibility and intermediate restore/reindex steps. Do not treat that general rule as a managed-service admission matrix. AWS explicitly identifies OpenSearch 1.x and Elasticsearch 7.10-or-earlier snapshots as incompatible with 3.x. The selected snapshot's constituent indexes still require lineage evidence. A snapshot taken on a newer engine must not be assumed to have rewritten an inherited index.

AWS's upgrade page also instructs removal of incompatible manual-repository snapshots before 3.x. This paper does not make deletion a default step. Record the affected snapshots, retention obligations and approved recovery replacement. If preserving them cannot be reconciled with the proposed path, hold and obtain a separately authorized retention/repository decision. Do not destroy the only recoverable copy to satisfy a precheck.

Repository access is a separate control plane. Registration guidance requires signed requests; with fine-grained access control, the relevant IAM role mapping also matters. The caller needs scoped iam:PassRole and es:ESHttpPut; the delegated snapshot role needs the intended S3 access and trust relationship. Reader, writer and delegated-role authority are not interchangeable. Register a new target as read-only when another domain writes the repository. Preserve one-writer ownership, network reachability, bucket encryption configuration and applicable key permissions. Never copy broad policy examples into production unchanged.

The registration page's encryption wording distinguishes repository options from bucket-side encryption, but can be read inconsistently around KMS. Record the actual bucket encryption mode and key-policy evidence rather than asserting that all manual-snapshot KMS encryption is unsupported. If the precise selected configuration is unresolved, keep its restore gate UNKNOWN and seek a bounded authorized compatibility check.

Remote reindex recreates documents, not the whole operating contract

The managed remote-reindex guide calls the receiving domain local and the source remote. Its local-version minimum is OpenSearch 1.0 or Elasticsearch 6.7; the remote source cannot have a higher major version than the local target, with Elasticsearch treated as lower than OpenSearch. Within the same major, a source minor can be higher. These are remote-reindex rules, not snapshot or in-place upgrade rules. T2/T3 data nodes do not support remote reindex. For its native VPC connection method, the documented conditions include local OpenSearch 1.0 or later, same-Region domains and a VPC source. Other documented connection methods have different constraints. Do not turn one method's prerequisites into universal cross-Region or cross-network support.

The target initiates the copy from the source. Record the exact chosen connection method, endpoint identity, authenticated principals and separately proven source-read/target-write privileges. AWS's page contains directionally confusing permission and VPC prose. Do not infer a production policy from it alone. Have the security owner resolve the exact managed authentication path using current supported controls and a restricted rehearsal. A successful connection approval does not prove document privileges, and a successful source read does not prove target creation/write permission.

OpenSearch's Reindex API requires enabled _source and a deliberately prepared destination; it does not copy mappings, settings or shard configuration. It copies a source snapshot, not a continuous change stream. Default internal version handling overwrites matching IDs. An asynchronous task ID proves submission, not completion. Preserve terminal status, failures, conflicts and the actual selected scope before accepting a baseline.

Define the destination's mapping, analysis, shard/replica configuration and pipeline behavior before transfer. Record any intentional divergence from the source and the fixtures that exercise it. Do not use generic performance advice to disable replicas or refresh behavior as an unconditional safe default. Any temporary change needs capacity evidence, an authorized vulnerability window and a recorded restoration gate before consumer acceptance.

Do not equate the reindex response's deleted field with propagation of all source deletions during the copy interval. It can reflect script-directed deletes during the operation; it is not proof of a continuing tombstone stream. A query-filtered or max_docs-limited operation is a subset. Its count must not be presented as the complete source index.

Qualify the target index and producer contract

Make the destination specification a reviewable difference record before copying. For each required behavior, record source evidence, intended target behavior, candidate evidence, acceptance fixture and disposition. “Copy the mapping” is insufficient when analysis depends on a domain package or a producer uses an endpoint/version compatibility mode. A deliberate change is acceptable only if its consumer owner accepts the changed contract; an accidental change is not a modernization benefit.

Consider fictional contract C-I2 for catalog-current. These rows compare invented configuration records and requirements, not actual OpenSearch responses. PASS means only the stated planning comparison passes; application execution remains UNKNOWN.

The upstream keyword field contract explains why a no-normalizer keyword identity field is not interchangeable with analyzed full text. In C-I2's fictional acceptance fixture, document c501 has sku: Rail-Kit; an exact Rail-Kit term must find c501, while a lowercase rail-kit term must not under the stipulated no-normalizer contract. That is an expected result to test, not a recorded result or a ranking claim. Capture the exact query body, selected field and returned IDs; a dashboard screenshot saying “one hit” loses the contract.

For analysis, retain package ID, revision/content identity, target association and whether the dictionary serves indexing or search. AWS package guidance states that uploading a replacement file to S3 does not by itself update the service package or every associated domain. Index-analyzer versus eligible updateable search-analyzer behavior differs. A familiar package name cannot establish the dictionary actually used to index the candidate or answer a query. Optional-plugin support also depends on the exact target engine, not the source domain's installed list.

Give mutation fixtures equally explicit outputs. For C-I2, an authorized future rehearsal would update c501.price from 4 to 5, verify a numeric filter at the accepted cut, delete fictional document c502, and verify that the exact required query no longer returns it. Record request identity/order, terminal result, refresh/visibility boundary, source position and read principal. Define these conditions before execution; a 200 response alone cannot establish visible query state or replay completeness. No such requests were issued here.

The producer row must retain the deployed artifact/version, request shapes, endpoint/signing configuration, enabled compatibility options and primary support evidence for the selected target. Where AWS specifically calls out Firehose or CloudWatch Logs compatibility, obtain that producer's current evidence rather than assume SDK success covers it. The packet rejects C-I2 while the price row fails or package/client rows remain UNKNOWN. Correcting the mapping declaration clears only that declaration gap, not query, dictionary, mutation or producer qualification.

Close the source-state gap before consumer admission

Every transfer path needs two boundaries: the baseline it represents and the later source changes it must incorporate. Record source position or another reproducible cut, its relationship to the copied baseline, and the acceptance cut for switching consumers. If the tool cannot associate its read snapshot with the source's change log, that linkage is unresolved. Do not simply invent a timestamp cut that makes the evidence appear complete.

For reconstruction, identify the authoritative records and transformation version. For snapshot or remote copy, identify how updates and deletes since baseline will be captured and applied in order. If the retained source is only current state, establish whether it can correctly reconstruct deleted records, access removals, late updates and historical projections required by consumers. An index that is the only retained authority cannot be rebuilt from a nonexistent upstream source.

*These are alternative evidence paths, not three simultaneous transfers. Arrows describe admission dependencies, not continuous replication. A completed baseline cannot skip the replay and application gates.*

Assign the source owner responsibility for the reconstruction window and tombstones. Assign the search owner responsibility for mapping/analysis equivalence and expected query behavior. Assign the application owner responsibility for actual producers and consumers. A platform owner can attest a healthy domain without attesting that a product's required query remains correct. A security owner must separately check access and protected-data obligations; see search permission-change consistency.

Use data reconciliation for the general paired-cut foundation. Here the additional obligation is to connect that cut to index-origin compatibility and the managed transfer actually selected. Equal document counts can hide one missing ID and one extra ID, stale values, undeleted documents or changed analyzers. Counts are a diagnostic, not the acceptance contract.

A populated fictional register exposes two different holds

Assume fictional domain catalog-example, OpenSearch 2.19, with a proposed target minor version still UNKNOWN. These rows are invented evidence-planning examples, not collected settings or AWS responses.

For I3, assume a fictional plan needs 45 minutes from its earliest required source position to the final reconciliation cut. A 30-minute retained window leaves a 15-minute coverage shortfall. This subtraction is a planning fixture, not a throughput estimate or evidence that either duration can be achieved. Faster nodes do not prove away a retention gap. A source owner must supply a complete alternative baseline/change capture or an approved bounded writer-fence plan. Otherwise hold.

Suppose I2's copied target contains IDs a and b, while the authoritative acceptance cut contains a and c. Both counts are two. The ID set fails. In another fictional fixture, both sides contain a and c, but a.price remains the old value. ID equality still fails update acceptance. If a deleted b remains searchable, the tombstone gate fails. These fixtures do not claim a real restore or replay result.

No row can lend its passing evidence to another index. I1's documented lineage requirement does not establish I2's settings, and I2's successful hypothetical query cannot establish I3's delete handling. The domain decision remains hold until every required row has an admissible method and all cross-index consumer dependencies pass.

Assemble a register that can survive a handoff

Use one row per index UUID and one append-only evidence reference per claim. Preserve previous names and tier transitions instead of overwriting history. Record a conclusion as PASS, FAIL or UNKNOWN, with the evidence timestamp and reviewer responsibility. PASS means the defined requirement was demonstrated for the selected scope, not that all migration risks vanished.

The blank record below is intended to be copied and filled. Every UNKNOWN on a required gate holds the proposed action. Evidence references should point to approved internal records, not public credentials, sensitive documents or raw customer data.

Decision packet ID: UNKNOWN
Account / Region / domain ARN / capture time: UNKNOWN
Current engine / exact proposed target / compatibility-response reference: UNKNOWN
Service-software version (separate from engine): UNKNOWN
Index UUID / current name / previous names / tier: UNKNOWN
Original creation version / inspection evidence / inspector: UNKNOWN
Mappings / analysis / settings / packages / intended differences: UNKNOWN
Required consumer and producer identities / supported-version evidence: UNKNOWN
Existing cross-cluster connection identity/direction/use / active replication: UNKNOWN
Remote/current/proposed versions / post-upgrade compatibility / dependency owner: UNKNOWN
Authoritative source / retained history / delete evidence / source owner: UNKNOWN
Selected path and rejected alternatives with reasons: UNKNOWN
Snapshot identity / engine / included indexes / status / restore compatibility: UNKNOWN
Reindex scope / enabled _source / destination contract / terminal task evidence: UNKNOWN
Repository writer / caller / delegated role / key and network evidence: UNKNOWN
Baseline-to-source position linkage / acceptance cut / replay ordering: UNKNOWN
Required replay window / retained window / safety margin: UNKNOWN
ID / update / delete / mapping-analysis / required-query fixtures: UNKNOWN
Application acceptance results and evidence references: UNKNOWN
Hot-tier headroom / parallel capacity / dated price inputs / cost ceiling: UNKNOWN
Switch authority / freeze conditions / post-switch write authority: UNKNOWN
Return or forward-recovery method / later-write handling / recovery evidence: UNKNOWN
Data-retention and cleanup authority / governed-copy inventory: UNKNOWN
Remaining FAIL or UNKNOWN gates / owner / next evidence date: UNKNOWN
Index decision / domain decision / actual approving authority: UNKNOWN

Do not label the packet approved because the worksheet has names in every field. The named owners must provide the actual evidence and the authorized approver must accept the residual limits. Maintain exact scope: one account/Region/domain/target version, an enumerated index set and identified producer/consumer versions.

Run a bounded rehearsal only after its inputs are admitted

This is a future rehearsal design, NOT EXECUTED. There are no write, delete or upgrade commands here. Before any account operation, obtain a separately approved plan with exact resources, principals, permitted actions, data classification and spend cap. Use an isolated target and an agreed representative data set; do not assume a production snapshot is harmless test data.

1. Platform owner collects the control-plane packet. Output: exact current and proposed engine evidence, dated compatibility response, domain health/configuration and per-index lineage/tier manifest. Stop on denied reads, absent target support or an incomplete retained-index inventory.
2. Search and data owners select the path. Output: index-level compatibility/reconstruction decisions and a complete baseline/change-linkage design. Stop if old lineage has no approved remediation, _source is unavailable for a reindex plan, or source retention cannot cover the defined interval.
3. Security and capacity owners admit the sandbox. Output: scoped caller/delegated-role/network/key evidence, target isolation, governed-copy register and approved headroom/cost ceiling. Stop on ambiguous privilege direction, repository multi-writer exposure or unknown capacity. Do not open a private domain publicly to bypass an unresolved connection.
4. Authorized operators establish the candidate baseline. Output: snapshot/restore or reindex/rebuild identity, scope, timestamps and terminal error/partial/conflict evidence. Stop on a partial snapshot, unexplained task failures, an unintended subset or target settings that differ without acceptance criteria. Preserve failed evidence; submission is not completion.
5. Data and application owners reconcile to the acceptance cut. Output: explicit IDs, update/delete fixtures, analysis/query outputs and producer/client results tied to that cut. Stop on unknown history coverage, divergence, authorization failure or missing required consumers. Capture measured durations only if the rehearsal actually occurs.
6. Change authority decides, independently of tool completion. Output: admit a specifically bounded change or hold with unresolved rows. For an in-place path, an authorized UpgradeDomain request with PerformCheckOnly: true is only an eligibility check, not an engine upgrade or application acceptance. A later real upgrade needs its own approval and request record.

Use GetUpgradeStatus with the captured operation context. Its step is PRE_UPGRADE_CHECK, SNAPSHOT or UPGRADE, with a separate status. A successful precheck is not a successful upgrade. Treat SUCCEEDED_WITH_ISSUES as evidence requiring issue review, not a blanket green application gate. The endpoint describes the last upgrade or check, so associate its result with the intended change rather than silently using a later operation's status.

Recovery after success is not the service's failed-upgrade restoration

AWS's downgrade guidance states that an in-place upgrade is irreversible. Its support-assisted recovery can restore an automatic pre-upgrade snapshot to a new older-version domain; an owned compatible manual snapshot provides a separate reader-managed restoration path. Neither is an in-place downgrade. Neither contains writes that occurred after its baseline.

Distinguish three outcomes in the change packet. Before an admitted real upgrade begins, hold preserves the current domain without claiming nothing else changed. If the service upgrade itself fails, the documented workflow uses its pre-upgrade snapshot for restoration; record actual resulting state, not assumed availability. After an upgrade succeeds but the application fails acceptance, choose the separately owned new-domain recovery or forward-remediation plan. Do not promise cancellation of an already successful engine transition.

For a parallel target, the old endpoint is a return candidate only while the accepted write-authority arrangement makes it complete enough for the agreed obligations. Identify the first target-only accepted write and how it can be reconciled back, or explicitly declare that return becomes unavailable beyond that point. An endpoint switch cannot manufacture missing updates/deletes. Unknown later-write handling holds cutover even when queries on the target look correct.

Specify abort triggers before the rehearsal: evidence gaps, unexpected source/target privilege, lost isolation, incomplete data, incompatible settings, query divergence, headroom breach or the approved spend threshold. The response is to stop authorizing further changes, preserve evidence and invoke the preapproved recovery decision. Do not issue ad hoc index deletion, snapshot deletion or broad permission changes under the label “rollback.”

Fund the transition and close governed copies deliberately

Estimate overlap using dated actual Region/node/storage/transfer inputs, not savings claims. Include both domains during parallel validation, hot-tier capacity to remediate archived indexes, repository storage, retained recovery copies and engineering time. A simple planning expression is overlap compute + added storage + applicable transfer + rehearsal/reconciliation effort. The expression is not a billing calculator; owners must identify which charges actually apply and the observation/alert method for the ceiling.

Separate cleanup from acceptance. List temporary indexes/domains, repositories, credentials/mappings, connection objects, retained snapshots and copied sensitive data. Assign retention authority and permitted cleanup time to each. Do not delete the old domain or sole recovery snapshot merely because a query fixture passes. Preserve the agreed recovery window and required evidence first. No cleanup actions have been performed for this paper.

Decision checklist and next steps

The paper's packet is complete when every required index has evidenced lineage, an admissible exact-target path and an owned data-state boundary; required producers/consumers pass defined acceptance; capacity, access, retention and spend are admitted; and the successful-change recovery limits are understood by the actual approver. A completed tool operation alone does not satisfy these conditions.

If any required field remains UNKNOWN, publish an internal hold decision with a specific owner and next evidence action. The useful outcome may be “do not upgrade this domain yet.” That is more defensible than calling an inherited index compatible because its domain is newer, or calling a copied target ready because its document count matches.

No live service availability, migration duration, zero-downtime result, charge estimate or recovered customer state is established here. Independent technical/source/visual review and actual authorized environment evidence remain separate gates. This paper is an evidence design for making the decision, not proof that a transition has been performed.