Secrets Manager Rotation: Which Credential Does the App Use?

Separate Secrets Manager rotation completion from cache adoption and fresh database authentication. Compare single-user and alternating-user evidence.

Primary sources checked

Check the consumer before closing the rotation record

A successful Secrets Manager rotation does not establish that every application process has adopted the new credential. Compare the secret's version labels, the credential supplied to a particular connection factory, a newly authenticated database session and the operation the application needs. A successful query on an older pooled session cannot answer all four questions.

The reader output is one consumer-adoption record that separates completed rotation, observed adoption, known mismatch and missing evidence. It supports a decision to close that consumer's routine rotation check, hold expansion or investigate a specific boundary. It does not authorize a rotation, restart, permission change or incident closure.

The supported teaching context is a password-authenticated application user on Amazon RDS for PostgreSQL, accessed directly by an application, with Secrets Manager rotation through Lambda. Single-user and alternating-user strategies are compared. RDS-managed master-password rotation, IAM database authentication, RDS Proxy, replicated secrets, other engines and non-database API keys need their own lifecycle review. An application using a proxy must first separate its client and backend authentication paths.

All example events are constructed. No AWS account, database, credential, cache package or driver was exercised. The default procedure reviews existing authorized records without retrieving secret values. A later rehearsal needs its own isolated authorization and effects boundary.

Read version labels as service state

AWS's Lambda rotation description separates creating a pending version, changing the target credential, testing it and moving the current label. These steps cross two systems. A database password can have changed before the service publishes the replacement as current.

The documented template test uses read access. That can support its rotation check without proving that every application operation, connection initializer or scheduled worker works. Record which test the deployed function actually performs. A custom function's successful return is not permission to invent the template's behavior, much less the application's result.

At completion, identify the exact version carrying AWSCURRENT and the earlier version carrying AWSPREVIOUS. Do not classify the cycle solely from the presence of AWSPENDING: AWS documents successful cases where it shares the current version or is absent. A pending label on a different version warrants investigation rather than an application-adoption conclusion.

An authorized DescribeSecret read provides metadata, including the labelled version map, without the secret value. It requires secretsmanager:DescribeSecret. Keep collection time and resource binding with the result. Missing fields, inaccessible metadata and observations from another Region or similarly named secret remain gaps. A current map alone cannot reconstruct an unrecorded historical map.

AWS also documents eventual consistency for Secrets Manager changes. Dated metadata, retrieval and database observations are not an atomic cross-system snapshot. Record conflicting observations and collect the missing evidence; do not invent a fixed propagation deadline to make them agree.

Choose what single-user or alternating-user success means

With single-user rotation, one database user's password changes. AWS documents that existing database connections are not dropped by rotation, while a brief database-versus-secret update gap can affect authentication. A process holding the earlier password can therefore behave differently from an already authenticated session in that same process. AWS strategy reference.

This produces a useful diagnostic contrast: an old pooled session may still read successfully while a fresh connection using stale configuration fails. The contrast points toward credential acquisition or connection construction, but a real incident still needs the actual authentication error and target identity. A timeout is not evidence of a bad password.

Alternating users changes the comparison. The function alternates updates between the original and cloned database users; AWS documents that both users' credentials are valid after rotation. Success with the earlier user can be compatible with the strategy while failing a requirement to adopt the current secret. A successful connection is therefore weaker evidence of adoption in this case.

Alternation also introduces another database identity. AWS calls out permission maintenance for the clone when the original user's permissions change. Compare the application's required access and relevant user settings, rather than assuming two accepted passwords mean interchangeable behavior. The alternating-user tutorial also explains the separate privileged secret used for cloning and password updates. That rotation authority does not belong in ordinary application credentials.

Choose the simpler single-user path when its continuity behavior fits the workload. Consider alternation when continuity justifies managing both identities and the permission model supports it. Neither choice establishes that previously exposed credentials, retained copies or existing sessions have been contained.

Follow the cached value into the connection factory

Three retained objects can matter: a secret-cache entry, a value copied into application configuration, and an authenticated database session. Refreshing one does not, by itself, establish what happened to the other two.

AWS documents an hourly default refresh for its Python caching component. Its configuration reference names secret_refresh_interval and the default AWSCURRENT stage. These are that component's documented settings, not a universal one-hour convergence deadline for all SDKs, extensions or applications. Capture the actual package version, configuration and refresh/error path before using timing to explain an incident.

The Python cache documentation also says its implementation does not include cache invalidation or force garbage collection after references are freed. A refresh policy is not proof that earlier values disappeared from process memory. Keep credential adoption, retained-material handling and exposure containment as separate review questions.

A process can retrieve a string once, build a pool with that string and never ask the cache again when it creates later connections. Another implementation might obtain credentials through a reviewed supplier at physical connection creation. Read the real connection-construction code before choosing a repair. Restarting a worker changes several observations at once and does not identify which retained object caused the problem.

An application pool's checkout is also different from establishing a new database session. Identify whether the test borrowed an existing connection or completed fresh authentication. Returning a client to an idle list does not prove that its next borrower will authenticate again. The existing fleet-pool article owns connection inventory and capacity; this guide uses that inventory to identify credential consumers.

Do not assume an API exposes provenance it does not return. The documented Python get_secret_string method returns a string. It does not supply a public VersionId response alongside that string. If the deployed wrapper has no reviewed version binding, record the consumed version as UNKNOWN. A separate metadata read cannot fill that field.

Bind observations without retaining passwords

The GetSecretValue API returns a version identifier and stage metadata with sensitive secret contents. When neither version selector is supplied, its documented selection is current. An explicitly pinned version changes that expectation. Where a reviewed application retrieval path retains only safe provenance, connect that observation to the value supplied to the connection factory without logging the value itself.

Do not add unrestricted secret retrieval for a dashboard. Review existing instrumentation first. Any new retrieval needs scoped GetSecretValue authority and, for a customer-managed encryption key, the documented decrypt permission. A metadata-only reviewer does not need the application's secret-access privileges. Never print SecretString, connection URIs, tokens, process environments or password hashes into this record.

Use sanitized aliases in broadly shared material. Keep actual resource identities and restricted evidence references in the approved operational system. A password digest can still expose a guessable secret; it is not an acceptable substitute for a non-secret version identifier.

Bind each observation to the same application revision, process, pool, connection factory, target and time window. A shell that fetches current credentials and logs in successfully does not establish what a warm worker used. A successful credential test also does not establish the permissions needed for the application's chosen operation. Keep authentication, authorization and correct result separate.

Worked example: a single-user adoption record

The following record is constructed teaching data. Every timestamp is stipulated UTC on 17 October 2026. Version aliases V17 and V18 are not valid API identifiers. No password or credential material is represented. The event intervals illustrate ordering, not an AWS rotation-duration promise.

Scope and required result

FieldConstructed value
Review and targetADOPT-17 revision 1; one fictional direct RDS PostgreSQL target DB-A and secret S-A in account A, Region R
Rotation and functionSingle-user application-password rotation; function revision F1; user alias U-A
Consumer inventoryapi-A/process-A/pool-A and report-B/process-B/pool-B; each has one reviewed connection factory
Application requirementEvery listed consumer must create a fresh session using V18 and read fixture key test-record with exact value fixture-r1
Runtime qualificationExact engine, driver and cache package versions are UNKNOWN; this is an interpretation example, not a compatibility test
Owner and scopeApplication owner owns adoption; database owner owns target/password reconciliation; security owner owns permitted evidence and any exposure response
Excluded effectsNo customer data, writes, notifications, funding, credentials or AWS calls; all observations below are stipulated

Event and observation timeline

UTC timeConstructed event or observationConclusion supported
09:58:00Session SA1 authenticated earlier for process-A using V17; its binding is stipulatedNames an existing session, not a later authentication
10:02:00F1 changes U-A's database password to the V18 value; the current-label readback still shows V17Target and current label are temporarily out of step
10:03:00F1's pending-version read test succeeds with V18 against DB-ASupports this stipulated rotation test only
10:04:00Recorded label transition makes V18 current and V17 previousService publication milestone
10:04:20process-A borrows SA1 and reads test-record = fixture-r1Existing-session operation succeeds; current adoption remains unproved
10:05:00process-B's reviewed factory supplies its retained V17 value to fresh session attempt SB2; DB-A reports password-authentication failureKnown stale-version mismatch in this constructed record, not a generic network diagnosis
10:06:00process-A's reviewed acquisition/factory binding supplies V18 to new session SA2; test-record returns fixture-r1process-A meets the stated fresh-session/read requirement
10:07:00Review closes its observation window; process-B has no successful V18 fresh-session evidenceOverall adoption remains HELD; process-B has a known failed path, not merely missing telemetry
In constructed ADOPT-17 single-user rotation, V18 becomes current while a reused V17 session still reads successfully. A fresh report-B attempt bound to V17 fails. A later fresh V18 api-A session succeeds, but aggregate adoption stays held because report-B has not met the requirement.

Constructed ADOPT-17 sequence, not captured AWS behavior. All times are stipulated UTC on 17 October 2026; spacing shows relationships, not elapsed duration or a service guarantee. V17 and V18 are version aliases, not valid API identifiers or secret values. Exact engine, driver and cache versions remain UNKNOWN. The target update and service publication do not establish consumer adoption. SA1 success uses an existing session. Only api-A has the stipulated fresh V18 authentication and required fixture result; report-B remains unresolved at 10:07.

The constructed single-user record begins with SA1 authenticated for process-A using V17 at 09:58:00. F1 changes the target U-A credential to the V18 value at 10:02:00 while the current-label readback still shows V17. At 10:03:00 its pending-V18 read test succeeds against DB-A. At 10:04:00 V18 becomes current and V17 previous. At 10:04:20 api-A borrows SA1 and reads test-record = fixture-r1; that is reused-session success, not fresh authentication. At 10:05:00 report-B supplies its stipulated retained V17 to fresh attempt SB2 and password authentication fails. At 10:06:00 api-A supplies stipulated V18 to new SA2 and reads test-record = fixture-r1. At 10:07:00 the review window ends with adoption HELD: api-A met the required fresh-session/read path, but report-B has no successful V18 fresh-session evidence. Without the supplied version binding, an authentication failure alone would leave its consumed version UNKNOWN.

The final label observation is compatible with completed rotation, while the consumer result is mixed. The record does not become green because process-A recovered, because a rotation log contains a successful step or because a generic health check passed. process-B's owner must reconcile its retained configuration and demonstrate the required path under a separately approved test.

The known V17-to-SB2 binding is a stipulated input to this example. Remove it and the same authentication error no longer identifies the credential version. That variant is UNKNOWN provenance with an observed authentication failure. It does not justify pretending V18 was rejected, or assuming an unobserved cache defect. Similarly, remove the fresh-session evidence from SA2 and a successful read no longer proves fresh authentication.

Reinterpret the same symptom under alternating users

Change the strategy assumption before interpreting the observations. At the first alternating cycle, suppose V17 names U-A and the new current V18 names U-B, the clone. A fresh connection using V17/U-A can still succeed after that cycle. The result establishes neither V18 adoption nor rejection of the new credential.

At a later cycle the rotation function updates U-A again and publishes V19 for that user. A retained V17 value is not indefinitely protected by alternation. Keep the cycle, user and version in the record; “old credentials work” without those identities is too ambiguous to support a lifecycle decision.

Suppose U-B authenticates but cannot perform the required isolated read, while U-A can. That is a different failure from a stale password. Record the authenticated identity and authorization reason, then have the database owner compare the required access and actual clone settings. Do not give the application the rotation function's privileged credentials to make the symptom disappear.

The closure requirement may be narrower than adopting current everywhere. A team could deliberately accept temporary use of the prior alternating user for continuity. That needs a named owner, permitted consumer set, next-cycle handling and a separate security decision. It is not evidence that current was adopted, and it must not be reported as credential revocation.

Retain a reusable consumer record

Use one row or labelled record per consumer, not one service-wide checkbox. Preserve the collection method and its limitations when handing the record to another owner.

Record groupFields to retain
Resource and configurationApproved secret/target/Region aliases; rotation strategy and function revision; expected stage or explicit version; engine, driver, cache and application revisions
Consumer bindingProcess and pool identities; construction/supplier path; acquisition event and non-secret version binding, or UNKNOWN
Service observationVersion-to-stage map, collector/time and rotation-step references; distinguish current readback from historical evidence
Session and operationFresh versus reused session; authentication result and identity evidence; harmless required operation and exact expected result
Coverage and dispositionAll named consumers, unobserved schedules/recovery paths, known mismatches, observation window, requirement met/failed/unverified, owner and next action
Recovery and safetyAuthorization, isolated destination, effects prohibited, retry/connection limits, stop owner and separately reviewed recovery path

For the filled record, api-A is requirement met at 10:06. report-B has an observed stale-credential failure at 10:05 and lacks the required V18 success. The aggregate requirement stays HELD. UNKNOWN runtime versions prevent this teaching record from certifying any actual driver/cache combination, even though its supplied observations can be interpreted coherently.

A rarely used scheduled consumer needs a deliberate disposition. Await its next approved operation, rehearse a representative isolated invocation or retain it as unverified with an owner. No recent errors is not a replacement for observation. A recovery tool that only runs during an incident can be the consumer that invalidates an otherwise reassuring rotation summary.

Compare repairs against the failed boundary

If the cache holds the intended version but the factory retains a copied value, changing a cache interval alone cannot establish repair. Review how replacement connections receive credentials. If the application requests an explicit older version, evaluate that pin deliberately instead of shortening a default-stage refresh setting that the code bypasses.

If fresh authentication succeeds but the required read fails, keep the credential valid and the operation failed as separate facts. The owner should investigate grants, user settings, target and application expectations within the approved scope. Avoid widening access merely because authentication was successful.

For a service using a proxy, record both authentication boundaries before applying this guide. The RDS Proxy pinning article explains why a client session and a backend session have distinct reuse boundaries. A fresh client-to-proxy connection alone is not this direct-database example's fresh backend authentication evidence.

Before scheduling process or pool replacement, use the fleet connection budget. Simultaneously recycling every consumer can create connection pressure or interrupt borrowed work. A controlled replacement can be a legitimate recovery choice, but its successful result does not retroactively prove why the old process failed.

Specify a bounded rehearsal only when evidence is missing

Review existing evidence first. If a rehearsal is needed, obtain explicit environment, identity, data, cost, duration and action authorization. Use isolated synthetic or approved sanitized data and receivers that cannot send customer messages or create external effects. Rotation itself mutates credentials, so a read-only diagnostic approval does not authorize triggering it, changing stages or editing database users.

The proposed acceptance set should distinguish a warm reused session, a new session from the same process, a separately started process, a scheduled consumer and the next alternating cycle where applicable. Include a cache-refresh failure, a factory retaining the earlier value after cache refresh, an explicit-version pin and a clone permission mismatch. These are fixture requirements, not performed tests.

For each case, the reviewer must identify which boundary was exercised and which result was expected in advance. Missing provenance should remain UNKNOWN; a supplied mismatch should fail the corresponding requirement. A network failure must not count as an old-credential authorization denial. A known-good new session should not erase unresolved borrowed work.

Bound fresh connection attempts and acquisition retries across the fleet, including SDK, application and worker schedules. Keep one task deadline rather than granting each refresh failure an unlimited new budget. Authentication recovery must not blindly replay a business operation whose earlier effect is uncertain. The separate failover-recovery article owns that reconciliation problem.

Stop expansion and reconcile before reversing state

Stop the proposed cohort expansion and intentional connection cycling when the target binding changes, the wrong identity appears, the fixture result is incorrect, secret material enters diagnostics or agreed retry/cost limits are exceeded. The responsible operator must also stop new outward test attempts, including queued retries that wake after Stop. Already submitted requests and existing sessions retain their separately recorded uncertainty.

A partly executed rotation needs the rotation and database owners to reconcile the target credential and all relevant version labels. Do not mechanically remove a pending label or launch another rotation to clear a dashboard. AWS's troubleshooting guidance includes different failed-step causes; it does not establish what happened in this workload without readback.

Moving a label with UpdateSecretVersionStage changes Secrets Manager's version selection. It is not a command to restore a database password, update a process's copied value or close an authenticated session. An application rollback can therefore require the current credential path rather than the earlier secret. Never install a generic fallback to previous merely because that label exists.

If exposure is suspected, invoke the security incident owner. A continuity choice that retains both alternating users is not automatically acceptable containment. Account for target credentials, active sessions and retained copies under that incident scope. Repair consumer adoption or issue an approved safe replacement rather than routinely restoring a known compromised password. This guide does not determine a regulatory revocation deadline.

Next action: close one consumer's evidence gap

Choose one actual consumer and state its closure requirement before changing anything. Obtain the approved label snapshot and that consumer's construction, version-binding and session evidence. Identify whether the first gap is metadata, credential provenance, fresh authentication, authorization or a correct application result. Assign the next observation to the owner who can collect it without exposing credentials.

Use the existing security-hardening playbook for the broader access, exposure and rotation-rehearsal programme. Use the EKS credential-source guide when the unresolved question is which AWS credential source the application selected, rather than which database password version it supplied. Neither replaces this consumer's adoption record.

Related services