API Gateway Patterns: Routing, Aggregation and Failure Boundaries

Choose gateway responsibilities, work through fan-out latency and partial failure, and evaluate deployment options with security and recovery tests.

Choose the responsibility before the gateway product

An API gateway gives callers a controlled entry point for routing and shared request policies. It can simplify authentication checks, traffic limits and protocol handling. It also becomes another dependency on the request path.

Use one when several APIs need a consistent external boundary or when consumers should not know the internal service topology. A small system with one backend may already have sufficient routing and protection in its load balancer and application. Microservices alone do not establish that another gateway is necessary.

The design decision is which responsibilities belong at that boundary, which belong in a client-specific backend, and which remain with the service that owns the data. This guide uses a hypothetical order dashboard to work through those choices. The timings are illustrative, not a benchmark or an Ampity customer result.

Separate routing, client composition and domain rules

| Responsibility | Likely owner | Boundary to preserve | |---|---|---| | Route by host, path or supported API version | Gateway | Reject unsupported routes and versions predictably | | Validate external credentials and coarse scopes | Gateway or shared authentication layer | Services still enforce resource and action authorization | | Shape data for an operations dashboard | A backend for frontend (BFF), when needed | Keep client-specific presentation separate from domain decisions | | Decide whether an order can be cancelled | Order service | Enforce the rule at the write boundary, including non-gateway callers | | Limit aggregate traffic | Gateway | Services also bound expensive work, concurrency and downstream calls | | Execute a long-running business process | Application or workflow service | Persist state, permissions and recovery outside transient proxy execution |

A BFF can be useful when an operations dashboard and a partner integration need materially different response shapes or release cycles. It adds a deployable service, tests and an owner. If both clients need the same contract, a shared endpoint may be enough.

Lightweight aggregation can live in a gateway policy. When composition needs substantial application code, different scaling behavior or domain-specific failure handling, put an aggregation service behind the gateway. Microsoft's Gateway Aggregation pattern discusses this separation and its availability and resource tradeoffs.

Worked example: calculate the dependent fan-out

An order dashboard needs the authorized order record, the customer's display details and shipment status. In this example, the order lookup establishes access and returns the identifiers needed for the other two calls.

Assume the following durations for one hypothetical request, including each downstream call's network time:

| Work | Duration | Dependency | |---|---|---| | Retrieve and authorize the order | 40 ms | Must complete first | | Retrieve customer display details | 60 ms | Needs the authorized order's customer identifier | | Retrieve shipment status | 90 ms | Needs the authorized order's shipment identifier | | Assemble and serialize the response | 10 ms | Runs after the selected results are available |

Making all three calls sequentially takes approximately 40 + 60 + 90 + 10 = 200 ms. Running the two independent enrichment calls in parallel after the order lookup takes approximately 40 + max(60, 90) + 10 = 140 ms.

The complete request is not just the slowest call because the first lookup is a dependency. These calculations exclude additional gateway admission time, contention and other overhead. Production latency needs traces and load tests; adding calls can increase tail latency even when they run concurrently.

Do not launch enrichment using arbitrary identifiers from the client before establishing access to the order. Each receiving service also needs the appropriate identity context and authorization check.

Define partial results before a dependency fails

For this dashboard, suppose order access and order state are required. Customer display details and shipment tracking are optional for viewing, but may be required for a later business action.

| Failure | Dashboard response | What must not happen | |---|---|---| | Order access is denied | Return the documented denial | Fetch or reveal enrichment for that order | | Order service is unavailable | Fail the summary request clearly | Invent an empty order or report it as successfully cancelled | | Shipment lookup exceeds its budget | Show tracking as unavailable with an explicit section status | Present missing tracking as “not shipped” | | Customer display details are unavailable | Omit or mark that section under the response contract | Reuse another customer's cached response | | Client disconnects or the total deadline expires | Propagate cancellation and stop unnecessary work where supported | Keep launching new fan-out calls or retries |

Give the aggregation request a total deadline and each call a budget that fits inside it. Account for response assembly. Do not reset the original budget at each hop. Cancellation must reach the code doing the work, and it does not undo a completed write.

A partial dashboard is acceptable only if consumers can distinguish “unavailable” from a real empty value. A cancellation or refund command needs its own current-state checks; the dashboard's degraded read must not authorize that action.

Keep resource authorization in the service

A gateway can validate a token and permit an orders route. It usually lacks enough current domain information to decide whether this caller may access this order.

The order service should verify tenant membership, object access and the requested action. Test direct service access and alternate routes so that bypassing the gateway cannot bypass the intended controls. If the gateway forwards identity headers, remove caller-supplied versions and authenticate the forwarding component.

OWASP's object-level authorization guidance applies even when the endpoint already requires authentication. A gateway policy, opaque identifier or web application firewall does not replace that check.

Cache policy follows the same boundary. For a personalized order response, start without shared caching unless the access partitioning, cache key, freshness and invalidation behavior are explicitly designed and tested. Include the API representation and relevant request variants in that design. A fast cross-tenant cache hit is a security failure.

Make asynchronous acceptance a separate contract

“Queue writes when the backend is unavailable” changes the meaning of success. It is appropriate only for operations whose consumers can accept delayed completion.

For example, an export request can return an operation identifier after the application has durably recorded the accepted job. The worker later rechecks required conditions, produces the authorized artifact and updates the operation status. Repeated submissions need a defined idempotency policy.

Do not acknowledge durable acceptance while the job exists only in gateway memory. If recording the job fails, return an explicit failure. If the acknowledgement is lost, let the caller find or retry the same operation safely.

HTTP 202 Accepted indicates that processing is not complete; it does not guarantee eventual success. The HTTP semantics specification recommends providing status information. Your application must define retention, cancellation, failure visibility and how a caller follows the operation.

An immediate order cancellation cannot silently become an asynchronous best-effort request. That requires a deliberate API and business-policy change.

Compare deployment options with a workload worksheet

Managed services reduce some platform operations. Self-managed gateways give the team more control over placement and extension behavior, along with responsibility for upgrades, configuration distribution and recovery. Neither description establishes the right product for a particular workload.

Evaluate the exact product, API type, tier and deployment mode. For example, AWS documents different features for API Gateway REST APIs and HTTP APIs. Its comparison lists private API endpoints and direct AWS WAF integration for REST APIs but not HTTP APIs. A private backend integration is a different requirement from a private client-facing endpoint. Recheck current documentation before choosing.

| Constraint | Test or question | Evidence for selection | |---|---|---| | Protocol and connection lifetime | Does the selected mode support the required streaming, payload and timeout behavior? | Current limits plus a representative end-to-end test | | Identity and network placement | Can clients and backends reach only the intended paths? | Authentication tests, network design and bypass checks | | Extensions and transformations | Can the required policy run safely without domain logic spreading into configuration? | A bounded implementation and its failure behavior | | Traffic and cost | What happens at normal load, a burst and a slow backend? | Latency, rejection, resource and cost measurements for that traffic mix | | Availability and recovery | What happens when an instance, zone or configuration dependency fails? | Recovery test and remaining-capacity evidence | | Ownership | Who patches, deploys, rotates credentials and responds to incidents? | Named responsibilities and an exercised rollback process |

Do not rank products as “good” or “excellent” without a workload and measurements. Include backend capacity in the test: scaling the gateway can send more traffic to a service that is already overloaded.

Operate the gateway as a shared dependency

Version routing and security policies, validate them before rollout, and retain the last working configuration. A redundant data plane can still fail everywhere after one bad configuration change.

Observe gateway processing time separately from upstream time. Track rejected requests, downstream concurrency, partial responses, cancellation, retry attempts and configuration version. Use correlation or trace identifiers across the calls, with controls for untrusted incoming trace data. Avoid logging tokens or unnecessary request bodies.

A backend slowdown should not consume every connection or worker slot. Bound fan-out and pending work per dependency, coordinate retries with application clients, and test whether health checks reflect the ability to serve real requests. Keep recovery capacity rather than relying only on scaling after saturation.

Next action: verify one operation end to end

Before expanding traffic, verify that:

• Unauthorized object access fails through every reachable path.
• A slow optional dependency produces the documented partial response.
• Required-data failure is visible and does not become false success.
• Deadline expiry stops new work and triggers supported cancellation.
• Duplicate asynchronous submissions resolve to one logical operation.
• Configuration rollback restores expected routing and authentication.
• A gateway failure leaves the documented remaining capacity.

Start with one client operation and fill in its ownership, timing and failure tables. Use the API design for longevity guide to connect those runtime controls to a durable compatibility contract. For gateway and service implementation work, see Ampity's backend systems and API service.