API Gateway vs Service Mesh

Decide where gateway, mesh, network, and application controls belong, then test identity, bypass paths, certificate rotation, traffic behavior, and safe rollout.

trigger="A workload needs clearer control of API entry traffic, service-to-service identity, or shared traffic policy." owner="The platform lead accountable for the selected traffic boundary and its operation." participants={["Application service owners", "Security architect", "Network engineer", "SRE or operations lead", "API owner"]} prerequisites={[ "An inventory of traffic paths, protocols, identities, and required access decisions.", "Measured latency, capacity, and incident baselines for representative service calls.", "A test environment, supported product versions, and a reversible deployment procedure." ]} outputs={[ "A control-ownership matrix and a decision to use existing controls, a gateway, a mesh, or a justified combination.", "Representative performance, authorization, bypass, and certificate-lifecycle test results.", "A bounded rollout and recovery runbook with explicit security constraints." ]} doneWhen={[ "Every required control has one accountable owner and a tested enforcement point.", "Approved callers work while unauthorized and bypass paths are denied as intended.", "Certificate rotation, unavailable control-plane, and traffic-failure scenarios have documented behavior.", "Operators can recover without silently weakening the accepted security boundary." ]} />

Choose by the traffic boundary, not the service count

An API gateway commonly manages an API entry boundary: routing, consumer authentication, quotas, and API-facing policy. A service mesh commonly supplies identity and traffic controls among participating workloads. Their capabilities overlap, and particular products can also handle ingress or egress.

Neither label proves what is enforced in your deployment. A gateway can serve internal consumers; a mesh can include gateways. A load balancer, application library, managed platform, or existing identity layer may already satisfy the requirement without another product.

Use this playbook to locate the missing control and evaluate how to operate it. There is no universal service count at which a mesh becomes worthwhile, and no vendor is consistently fastest or lowest effort across every configuration.

1. Inventory the real paths and trust decisions

The platform owner maps entry traffic, internal calls, egress, administration, health checks, and scheduled jobs. Include paths that do not pass through the usual proxy. Record protocols, long-lived connections, caller identity, destination identity, and the consequence of a denied or delayed call.

Distinguish three questions:

  • Authentication: who is the user or workload making the call?
  • Authorization: may that identity perform this action on this resource?
  • Network reachability: can the caller reach a path that avoids the expected enforcement point?

A client-facing gateway can verify a token, but downstream services still need resource and business authorization, or an equivalently enforced reviewed mechanism. Workload mutual TLS identifies peers under a trust configuration; it does not decide whether a user can refund a particular order.

2. Allocate controls before selecting products

| Concern | Candidate enforcement point | Responsibility that remains | | --- | --- | --- | | Public API authentication and consumer quota | Gateway or API platform | API owner defines identity and consumer policy | | Service identity and transport encryption | Mesh or existing workload identity system | Security owner defines trust and verifies coverage | | Resource, tenant, and field authorization | Resource-owning service or reviewed authorization layer | Application owner supplies business context | | Network ingress and egress restriction | Network controls and supported enforcement components | Network owner proves bypass paths are closed | | End-to-end deadline and duplicate-safe retries | Application, with coordinated proxy policy | Operation owner protects business effects | | Traffic selection and rollout | One designated gateway, mesh, or deployment layer | Release owner defines gates and rollback | | User outcome and business telemetry | Application plus infrastructure telemetry | Service owner verifies meaningful observability |

Avoid configuring independent retry policies at every layer. The combined number of attempts and deadlines must fit the operation contract. Similarly, pick one owner for a canary routing decision so weights in two proxies do not create an unexplained traffic distribution.

Use both gateway and mesh only when the control matrix justifies both. Define how identities, error responses, trace context, and timeouts pass between them. Duplicate features are not automatically redundant, but duplicated ownership is a predictable operational problem.

"type": "svg-architecture", "title": "Different boundaries retain different responsibilities", "nodes": [ ], "links": [ ], "caption": "An intended permitted path, not proof of enforcement. Mesh identity and transport controls apply only where configured; services retain business authorization and bypass routes need separate tests." }} />

3. Prove identity propagation and deny bypass

If the gateway adds identity headers, remove or reject untrusted inbound copies and ensure services accept identity assertions only from the approved trusted path. Use a verifiable mechanism appropriate to the architecture; a header name is not evidence of authenticity.

Test direct access to service addresses, alternate listeners, internal load balancers, debug ports, and unmeshed workloads. Confirm the policy holds when an attacker or misconfigured workload can reach an internal network. Document deliberate exceptions rather than relying on the assumption that internal traffic is trusted.

Istio's security documentation separates authentication from authorization and notes that workloads without applicable authorization policies allow requests by default. Installation alone therefore does not establish the desired access policy. Check the selected data-plane mode and deployed version when applying a policy example.

For Kubernetes, NetworkPolicy documentation describes network-level controls and their dependence on a supporting network implementation. Verify actual enforcement and limitations. Network policy is not a substitute for object-level authorization, and proxy routing configuration is not automatically an egress security boundary.

4. Evaluate implementations with the workload

The platform owner produces a shortlist from required capabilities and support constraints, then tests representative configurations. Avoid rankings based on vendor descriptions, an isolated benchmark, or unqualified per-hop latency numbers.

| Evaluation area | Test or evidence | | --- | --- | | Protocol fit | Actual HTTP, RPC, streaming, upgrade, and connection behavior used by the workload | | Latency and capacity | Baseline comparison under representative requests, concurrency, and TLS settings | | Resource overhead | Proxy and control-plane CPU, memory, scheduling, and failure headroom | | Identity integration | Trust domains, issuance, rotation, revocation strategy, and policy distribution | | Workload coverage | Supported cluster, VM, runtime, and data-plane modes | | Operational fit | Upgrade, configuration validation, diagnostics, access, and on-call competence | | Commercial lifecycle | Supported release policy, licensing, service limits, and current contracted costs | | Exit path | Configuration portability, observability continuity, and migration evidence |

Measure cold connections, certificate handshakes, large payloads, and persistent streams where they matter. A proxy-only benchmark does not establish end-to-end user latency. Include the failure path and recovery load, not just peak successful throughput.

As checked on September 21, 2026, AWS's App Mesh notice states that support ends on September 30, 2026 and console and resource access will no longer be available afterward. Do not select App Mesh as a normal new-build option. An existing deployment needs a time-bound migration plan using current AWS guidance and workload-specific validation.

5. Design traffic policy without hiding business semantics

A proxy can route, reject, retry, or delay a call. It cannot infer whether repeating that call will charge a customer twice. Coordinate retries, circuit breaking, admission control, and application deadlines with the service owner. Use the operation contract in Building Reliable Distributed Systems.

Traffic mirroring can produce real effects if the shadow target writes data or calls an external system. Isolate side effects, use approved data, and bound the extra load before copying requests. A hidden response is not equivalent to an isolated test.

For canary routing, identify the cohort and stickiness requirements. Verify how connection reuse, sessions, hashing, and long-lived streams affect the observed distribution. Choose exposure and observation periods from risk and workload cycles, not fixed percentages and minutes.

Infrastructure metrics do not replace application instrumentation. End-to-end traces may need context propagation and instrumented boundaries, while business success needs application-level signals. Confirm what the selected proxy and deployment mode can actually observe, including encrypted or unsupported protocols.

6. Test certificate issuance and rotation

The security and platform owners record trust roots, identity mapping, certificate lifetime, intended rotation timing, and recovery access. Protect signing authority and configuration privileges. An authenticated workload can still misuse its own permitted identity, so least privilege remains necessary.

Do not alert on a fixed “less than 48 hours remaining” threshold when the configured certificate lifetime is shorter than that. Derive alerts from the actual lifecycle: failed or overdue renewal, approaching expiry relative to repair time, issuer errors, and failed validation. Check both new connections and already established sessions where relevant.

| Failure scenario | Evidence to gather | | --- | --- | | Issuer or control plane unavailable | Existing traffic behavior and whether new workloads can obtain identity | | Rotation delayed | Detection occurs with enough time for the approved recovery | | Trust bundle changes | Expected peers remain trusted and removed trust is handled as designed | | Wrong service identity | Authentication or authorization rejects the caller | | Clock discrepancy | Handshake behavior and time-synchronization monitoring are understood | | Proxy restart or node replacement | Identity and policy are restored before the workload becomes usable |

Automatic rotation is a feature to verify, not a guarantee of no downtime. Record the deployed version and configuration with the test. Avoid copying lifetime defaults from one product or release into another's alert rules.

7. Roll out across a bounded boundary

"type": "flow", "title": "Adopt controls with evidence at each boundary", "steps": [ ], "caption": "Choose the cohort from dependency and risk boundaries. Do not weaken existing production protection merely to simplify adoption." }} />

Start with a representative service pair or entry route that has an accountable owner and a tested fallback. Validate configuration before distribution, confirm that the intended workloads received it, and observe both successes and denials.

A permissive or audit phase can expose dependencies where supported and approved, but it is not a mandatory step and should not reduce existing protection. Before enforcing stricter policy, inventory expected callers, health checks, jobs, and administration. Keep a clear stop condition for unexpected denials or latency changes.

Record configuration revisions, control-plane versions, data-plane versions, and workload rollout status. A configuration accepted by an API is not proof every proxy has applied it.

8. Recover without silently opening the boundary

| Failure | Immediate response | Recovery limit | | --- | --- | --- | | Bad route or policy | Stop expansion; restore a verified configuration if safe | Confirm all relevant enforcement points converge | | Legitimate traffic denied | Identify the specific caller and policy mismatch | Do not replace the policy with unrestricted access | | Certificate or trust failure | Invoke the approved identity recovery procedure | Preserve authentication and record emergency exceptions | | Proxy overhead exceeds budget | Reduce exposure or restore the compatible data path | Maintain equivalent required security controls | | Control plane unavailable | Follow observed data-plane behavior and change restrictions | Do not assume new workloads or updates will succeed | | Retry amplification | Bound attempts and admission | Reconcile uncertain business effects before repeating work |

Removing a sidecar, unenrolling a workload, or bypassing a gateway may remove security controls as well as traffic overhead. Such a rollback needs an equivalent accepted control or an explicit security decision. Do not describe it as a routine availability fix.

Routing changes do not reverse committed application writes. If canary or retry behavior caused incorrect effects, identify affected operations and use the application reconciliation procedure. Preserve telemetry needed to investigate while respecting the data-retention policy.

9. Rehearse a service-pair migration

The platform engineer and both service owners run this exercise before expanding a new traffic layer. Consider a hypothetical order API calling an inventory service. The existing gateway validates the external consumer. The proposed mesh supplies workload identity between the two services. Inventory authorization must still restrict which operations the caller may perform, and the order service must still enforce the user's tenant and business permissions.

Write the expected path for one allowed request and one denied request. Include the original consumer identity, the order workload identity, the inventory destination and the resource being accessed. Decide which identity evidence the downstream service needs. Forward only approved claims through an authenticated mechanism, and reject a caller-supplied assertion that impersonates the gateway or a privileged service.

The network owner then inventories paths outside that drawing: direct service addresses, secondary ports, a batch inventory importer, operational probes and a legacy deployment that is not enrolled in the mesh. Decide which must work and which must be denied. A blanket denial introduced without the importer can cause a delayed business failure that a request-only smoke test will miss.

Use the following acceptance record for the service pair. The examples are test cases, not suggested permissions for an actual order system.

| Exercise | Required evidence before expansion | | --- | --- | | Gateway-to-order request | An approved consumer reaches the correct route; forged identity headers do not survive as trusted claims | | Order-to-inventory request | The intended workload identity is authenticated and the requested operation is authorized | | Unrelated workload-to-inventory request | Reachability and authorization behave as specified; a shared network does not grant access | | Importer and health-check traffic | Approved operational callers continue to work with their own restricted identities or documented controls | | Policy update | The exact revision is visible at the enforcement points that serve the cohort, including newly started instances | | Enrollment reversal | The previous path retains equivalent required protection and does not reopen direct access |

The SRE runs these cases with connection reuse and representative concurrency. Include a new connection after credential rotation and an already established connection across the change. Record the observed session behavior rather than assuming that changing a trust bundle immediately terminates every existing connection.

Keep the gateway and mesh traffic-selection rules stable while measuring the first cohort. If both layers independently select a canary, the product of their decisions may create a smaller or different population than intended. The release owner chooses one routing authority and records what the other layer must preserve. Observed routing should be checked using request or operation identifiers with appropriate redaction.

Gate: the service pair passes the allowed, denied and operational cases, and operators can identify the applied configuration without guessing from the control plane's desired state. An unexplained denial, unprotected bypass or unknown policy revision prevents expansion.

10. Test control-plane loss and policy recovery

The incident owner defines a bounded exercise for control-plane unavailability in an authorized test environment. Existing proxies may continue using prior configuration, while new workloads, identity renewal or policy updates behave differently. Treat those as separate cases. Do not turn the control plane off in production merely to demonstrate a general claim about mesh resilience.

Before the fault, record the currently accepted routing and authorization revisions and the expected credential lifecycle. During the exercise, observe existing calls, new connections, workload restart, a requested policy change and identity renewal. Measure which actions remain possible and which become unavailable. The operator must know whether the environment is serving a last-known policy or failing to establish a required decision.

After recovery, confirm convergence by checking affected enforcement points and repeating both positive and negative access tests. A reconnecting proxy can be healthy at the process level while still using an old policy. Preserve the evidence that distinguishes configuration receipt, application and actual enforcement. If the product cannot expose a necessary state, record that observability limitation in the decision.

Use a reviewed configuration rollback when the new policy is wrong, but inspect the permissions restored by the previous version. Restoring an older allow rule can also restore access that was intentionally removed. If no safe prior version exists, hold the affected operation while the owners create and verify a forward fix.

The handover names the platform responder, service responder and security approver for this recovery path. It includes emergency access to the management plane, the method for identifying a partial rollout, and the evidence required to resume changes. Completion requires an operator other than the implementer to follow the runbook successfully in rehearsal. Retire temporary test identities and exceptions afterward, while retaining the decision and failure evidence under the agreed policy.

Decision record and completion checklist

Workload and traffic boundary:
Missing control and business consequence:
Current paths, identities, protocols, and bypass routes:
Selected enforcement point and accountable owner:
Gateway / mesh / network / application responsibility split:
Supported implementation versions and lifecycle:
Performance, authorization, and certificate evidence:
Retry, timeout, quota, and traffic-selection ownership:
Rollout cohort and stop conditions:
Safe rollback and equivalent security protection:
Incident recovery and application reconciliation:
Remaining risks and next review trigger:

"The decision is tied to a missing control, not a generic service-count threshold.", "Authentication, resource authorization, and network reachability have separate tested owners.", "Trusted identity propagation rejects forged assertions and bypass tests pass.", "Gateway and mesh retries, deadlines, routing, and telemetry are coordinated.", "Certificate alerts match configured lifetimes, renewal behavior, and repair needs.", "Representative performance and failure tests justify the selected implementation.", "Rollback preserves required protection and accounts for committed application effects.", "Product support lifecycle and migration responsibilities have been checked." ]} />

Ampity's cloud platform engineering is the related service for designing and operating these boundaries. For workload recovery objectives, continue with Cloud Reliability Engineering.

Primary references and deployment checks

Use documentation for the actual deployed release and mode. These references establish relevant checks, not a claim that a particular gateway or mesh automatically provides zero trust, complete tracing, or interruption-free migration.