Separating Edge Gateway and Internal Traffic Responsibilities
An anonymous account describing Kong at the API edge and Envoy for internal traffic, with configuration, retry ownership and migration effects kept distinct.
A custom gateway carried several traffic responsibilities
The platform used a custom Java reverse proxy for authentication and routing, with session data, counters and routing state held within the gateway. Scaling concerns and limited routing and customer-tier controls prompted a review of that arrangement.
Ampity designed a split: Kong at the external API boundary and Envoy sidecars for internal service communication. Existing API contracts, custom authentication, AWS deployment, Kubernetes, REST and gRPC traffic all constrained the change.
Separate gateway processing from its dependencies
Before attributing a slow request to the gateway, distinguish proxy processing, external policy dependencies and upstream service time. Adding instances may not resolve a shared dependency, but stateful design alone does not prove that replacement is necessary.
The engineering choice was to separate edge and internal-call responsibilities. Tuning the existing gateway and using a managed gateway were alternatives. Compare the actual authentication and traffic paths rather than assuming that every managed gateway has the same latency, cost or customization limits.
A second traffic layer must earn its operating cost
Kong was assigned external routing, authentication and rate limiting. Envoy was assigned internal circuit breaking, retries and observability. The design included cached JWT validation, Redis-backed limiting, request transformation and declarative configuration.
Adding internal proxies also adds configuration distribution, upgrades and incident ownership. Use that layer for a required control between services. A single entry gateway may be sufficient when the needed internal identity and traffic controls already exist elsewhere.
Trace the policy that each request actually receives
The exact JWT cache behavior, Redis failure policy, retry settings and configuration-distribution mechanism are unspecified. These details determine whether a request uses current authority, what happens when a shared dependency fails and how much repeat work is permitted.
Trace the deadline and attempt allowance through Kong, Envoy and the application. Independent retries can multiply attempts across layers. Authentication at the edge still leaves resource and tenant authorization with the application or another explicitly reviewed enforcement point.
Review existing traffic, newly starting workloads and policy changes separately during a control-plane outage. A separation between control and data planes does not define all three behaviors.
Shadow traffic can still change the upstream system
The migration approach combined shadow requests, staged sidecar adoption and gradual selection of the production traffic path. A duplicate request can change data or contact an external system even when nobody sees its response.
Choose cohorts and stopping conditions from connection behavior, dependencies and business effects, rather than fixed rollout percentages. Changing a traffic path limits new exposure; it does not cancel every dispatched request or reverse an upstream write. Reconcile an uncertain operation before allowing another attempt.
Assign one policy owner at each boundary
Use permitted configuration and existing traces for an authorized read-only review. This worksheet is proposed guidance for a comparable deployment, not an executed test from this engagement.
| Boundary to inspect | Evidence to retain | Question before expanding traffic |
|---|---|---|
| External entry | Effective Kong route, caller identity and response contract | Which gateway checks apply, and which application decisions remain? |
| Internal call | Envoy route and applied policy, upstream identity and remaining deadline | Which layer owns attempts and how is the overall deadline preserved? |
| Shared policy dependency | Effective Redis or authentication failure behavior | Does unavailability deny, defer or permit work, and is that acceptable? |
| Migration effects | Selected executor, connection cohort and in-flight operation record | Could shadowing, retries or recovery repeat a real effect? |
Any later comparison needs an authorized isolated environment, synthetic or approved sanitized requests, restricted identities and sinks that cannot mutate production or contact real recipients. Bound duplicate traffic and dependency pressure. Do not retain live production credentials in a comparison target merely because its responses are discarded.
Stop on an unauthorized path, unexpected effect or unexplained policy revision. Preserve the operation evidence before retrying or restoring traffic.
The scope is a responsibility split, not a benchmark
This case describes Kong and Envoy responsibilities and a staged migration approach. It does not publish measured throughput, latency, operating-effort improvements or an uninterrupted-cutover result.
Choose the smallest arrangement that covers the missing control
Start with one service call. Identify its external API policy, internal-call policy and application authorization separately, then decide which layer should own the missing control.
The API gateway versus service mesh playbook provides the broader control-ownership and failure-testing process. The choice depends on that work, not on adding Envoy to every gateway deployment.
Technical reference
Envoy's HTTP routing documentation describes configurable retry conditions and request-timeout limits. Its circuit-breaking documentation describes bounded upstream resources and active retries. These mechanisms require a workload-specific policy; neither establishes that repeating a business operation is safe. The account does not identify its deployed Envoy release or effective settings.