ECS Exec TargetNotConnected: Find the Failing Task, Agent or Channel Boundary
Trace an ECS Exec failure through the exact running task, caller, managed agent, task role and network path before proposing a scoped change.
Primary sources checked
Start with the exact failed attempt
For an ECS Exec TargetNotConnectedException, check the task and container that received the request, then separate the operator's invocation authority from the managed agent's channel permissions and connectivity. The error alone cannot identify the broken boundary. AWS lists permission, agent and Session Manager endpoint problems as possible causes in its exception reference. Adding permissions everywhere or repeatedly opening a shell will not distinguish them.
This guide helps an operator turn an existing failed attempt into an evidence-backed handoff: a supported configuration mismatch, a documented access or transport failure, or a named unknown. Begin with retained configuration and previously authorized observations. No AWS session, command, permission change or customer diagnosis was performed for the examples here.
The worked scope is Linux tasks on Fargate, or Linux tasks on ECS-optimized EC2 instances using awsvpc, with an explicit task IAM role and a known container target. Windows is a separate branch, not an unsupported service: its platform and image requirements need their own review. ECS Anywhere, ECS Managed Instances, custom EC2 images, other EC2 network modes and tasks without an explicit role are outside this procedure. Do not apply the examples until the launch type, operating system and task identity are confirmed.
Check applicability before following a Linux diagnosis
Keep the failed request's account, Region, explicit cluster, task identity, container name, timestamp and sanitized error together. The ExecuteCommand request names the task and can name a container; omitting a cluster selects the default. A nearby healthy task or the currently selected console Region may describe a different target. For multi-container tasks, identify the intended application container rather than assuming the first listed container is correct.
Use this table as an applicability screen, not evidence that a session will work. The version floors come from AWS's Exec prerequisites; capture the actual running version rather than the word “latest.”
| Target | Documented prerequisite to check | This guide's disposition |
|---|---|---|
| Fargate Linux | Platform version at least 1.4.0 | Continue after confirming the exact task |
| EC2 Linux | ECS-optimized AMI released after 20 January 2021; ECS agent at least 1.50.2 | Continue only within the explicit-role, awsvpc scope |
| Fargate Windows | Windows platform version 1.0.0 | Hand off to a Windows-specific review |
| EC2 Windows | Supported Windows ECS-optimized image; container agent at least 1.56 | Hand off; do not reuse the Linux floor |
The Windows image restrictions and exclusion of IPv6-only tasks are in AWS's Exec considerations. Route either unsupported or unknown applicability to the platform owner before interpreting a network or role worksheet. A failed prerequisite is a useful finding, but it need not explain every recorded error code.
Compare the running task with the configuration that launched it
Retain an existing DescribeTasks result for the requested task, including collection time and any per-item failures. Its schema includes task-definition identity, launch type, platform version, enableExecuteCommand, containers and managed-agent records. Record task and container lastStatus separately; a stopped target is not a current running-container session candidate. A successful HTTP response with a task in failures leaves that task unresolved. A missing record is not equivalent to enableExecuteCommand=false. DescribeTasks response fields.
Compare task creation and start evidence with the service or standalone launch configuration. Enabling Exec on a service does not retrofit an already running task; AWS limits enablement to new tasks. A service screenshot taken after a configuration change cannot establish the setting that applied to an older task. Preserve both revisions. Exec task enablement.
Inspect the specific container's ExecuteCommandAgent, recording lastStatus, reason and lastStartedAt when supplied. The ManagedAgent schema describes these as container-level, last-known observations. An application container being RUNNING does not supply the managed agent's status. A different sidecar's agent status does not supply it either.
If enablement is false or the correct managed-agent record is absent or stopped, keep that finding ahead of speculative IAM edits. The platform owner should inspect the relevant image, task configuration and retained agent evidence. A later RUNNING observation belongs to a later collection time; it does not prove that the agent was running during the failed attempt. Conversely, RUNNING is a prerequisite signal, not an end-to-end session test.
Use the actual task definition and any launch overrides to identify the task role. The TaskOverride schema distinguishes taskRoleArn from executionRoleArn. A newer family revision in source control may not be the revision that launched this task. If an override or relevant historical configuration is unavailable, leave the binding unknown instead of attaching whichever role has the most convenient policy.
Separate the caller from the role used inside the task
The invoking identity needs appropriate ecs:ExecuteCommand authority for the request. Resource and condition restrictions can matter: the API documents AccessDeniedException for a mismatch between a policy condition and the request parameter. Record the actual invoking principal and applicable policy review, not a successful ECS read by someone else. An explicit invocation denial is a different observation from a session that could not connect. ExecuteCommand authorization context.
The managed agent needs its own communication permissions. AWS's task-role Exec section identifies ssmmessages:CreateControlChannel, ssmmessages:CreateDataChannel, ssmmessages:OpenControlChannel and ssmmessages:OpenDataChannel. Review all four against the task's actual role and applicable restrictions. An allow excerpt is not a full effective-authorization result; missing policies or organization controls leave an unresolved boundary.
The task execution role serves ECS agent operations such as image retrieval and configured container logging. A successful image pull therefore does not establish the managed agent's channel authority. Keep the two role identities and their evidence separate. Task execution role responsibilities.
This procedure deliberately requires an explicit task role. The current Exec considerations describe EC2 instance-role fallback when no task role exists. That is a separate credential-path review, not permission to move channel actions onto every EC2 instance profile. Do not infer the agent's effective credentials from an application SDK's caller or from another container's environment. Request owner-approved role and credential-path evidence if the observed configuration conflicts with the expected path.
Review the task's channel path, including DNS and proxy handling
Trace outbound access from the affected task's network context to Session Manager's ssmmessages service in the correct Region. An ECS interface endpoint alone does not provide that channel. In a private-endpoint design, record the selected service, endpoint state, VPC, endpoint interfaces and actual DNS behavior. In a permitted outbound-internet design, record the owned egress path instead. The ECS troubleshooting page identifies the missing Session Manager endpoint as a possible failure.
For an interface path, ask the network owner for existing evidence about private DNS, the endpoint security group's HTTPS ingress from the task's source, task egress and relevant network ACLs. AWS's interface-endpoint prerequisites describe DNS attributes, security groups and ACL requirements. Merely finding an endpoint in an inventory does not show that this task resolves or reaches it. A laptop's DNS answer belongs to the laptop's resolver path.
General Systems Manager documentation also covers managed-node operations beyond ECS Exec and lists other service endpoints. Do not turn that broader list into a universal ECS minimum or create endpoints indiscriminately. Start with the Exec-specific dependency and the environment's actual configuration. Optional encryption and logging have additional dependencies to evaluate separately. Session Manager endpoint context.
Where an HTTP proxy exists, inspect the reviewed configuration for bypass of 169.254.169.254 and 169.254.170.2, as AWS's Exec considerations specify for metadata and IAM-role traffic. This is a configuration review, not permission to request either metadata endpoint or print credentials. Proxy values can contain secrets; retain sanitized references and bypass entries only.
A rejected flow, a DNS failure and a policy denial support different hypotheses. Keep the timestamp, source, destination and observation method beside each. If network evidence was never collected, “no flow seen” cannot establish that the agent made no connection. Do not open inbound container ports, remove an endpoint policy or disable controls as a way to obtain a passing session.
Check session constraints without treating them as repair instructions
Record the target's existing root-filesystem configuration and PID-namespace arrangement. ECS Exec requires writable agent paths; AWS does not support a read-only root filesystem for this feature. It also permits only one Exec session per PID namespace. Shared-namespace contention needs the session owner's evidence, not termination of another operator's session. AWS session constraints.
Keep the local client separate from the task. The documented CLI path needs the Session Manager plugin and AWS CLI at least 1.22.3 or 2.3.6. ECS starts interactive sessions only, so retain the requested interactive mode and permitted command reference as well as the target. These prerequisites do not prove target readiness. If the recorded failure is a plugin installation or local credential error rather than the named exception, route it to the client owner before applying task-side conclusions. Existing approved client-version evidence is sufficient; do not download or execute a troubleshooting script automatically. CLI and interactive-session prerequisites.
If a customer-managed KMS key protects Exec session data, the documented roles differ again: the invoking user needs kms:GenerateDataKey, and the task role needs kms:Decrypt. Have the key owner establish the actual key and applicable authorization, and the network owner establish any required KMS path. Session-data encryption, log-destination encryption and application encryption are separate contexts. A generic “KMS enabled” field cannot answer them. Exec encryption configuration.
Do not bypass ECS Exec by starting a direct Systems Manager session. AWS warns that sessions outside execute-command can escape the intended Exec logging path and still consume session quota. Existing logging settings also need their own review: an audit event identifying access is not the command-output transcript. A failed transcript lookup does not prove that no command ran. These are reasons to preserve approved controls, not a logging or encryption reconfiguration procedure.
Work through three synthetic scenarios before choosing the next owner
The following records are fictional supplied evidence, not AWS responses or Ampity client results. Their configuration versions and relative order are stipulated. They teach how to limit a conclusion; they do not emulate ECS, authenticate a policy review or calculate IAM authorization.
Record A: the enabled service and the old task
The failed request identifies Linux Fargate task task-a, application container web, platform 1.4.0. A retained task snapshot says enableExecuteCommand=false. A separate service snapshot says Exec is enabled, but its collection occurred after task-a started. Task-a is the same identity in the request and task snapshot. Its managed-agent evidence is missing.
The supported finding is a task-level prerequisite mismatch. The later service setting cannot establish that task-a was enabled. Missing agent evidence remains missing, rather than being converted to STOPPED. This record does not assert that disabled Exec uniquely produces TargetNotConnected. Preserve the actual error as observed and the enablement mismatch as a separate fact.
The service owner should compare the intended rollout with the actual task population and propose any replacement through the release process. This record authorizes no forced deployment, stopped task or desired-count change. A replacement would be a new observation with its own task identity and recovery checks.
Record B: the wrong role received the channel policy
The supplied EC2 Linux record identifies an ECS-optimized instance with agent 1.80.0, an awsvpc task task-b and an explicit role task-role-b. Exec is enabled and the selected container's managed agent was RUNNING at collection. The operator's request-specific authorization review is supplied. A policy change was attached to execution-role-b, while the task-role review lacks ssmmessages:OpenDataChannel and includes a supplied applicable explicit deny of that action.
The supported finding concerns the task-role channel boundary. The execution-role change does not resolve it. RUNNING status and operator authority do not override the supplied deny. Because these are teaching inputs, the conclusion means the records identify a concrete access conflict, not that a real AWS policy evaluator was exercised or that every other prerequisite passed.
The IAM owner should review the intended task-role scope and the deny's owner before any change proposal. Do not delete the deny or copy the policy onto the instance role. A separately authorized validation would still need to establish the session path and retain the intended restricted-access boundary.
Record C: an ECS endpoint and an unresolved channel route
The supplied Linux Fargate record identifies platform 1.4.0, task task-c, enabled Exec, a RUNNING managed agent and a matching explicit task role. The available role review covers all four channel actions. The network inventory lists an ECS endpoint, no ssmmessages endpoint, and no approved outbound-internet path. However, current route, DNS and traffic observations for task-c are unavailable.
The supported finding is an unresolved Session Manager transport path. The inventory does not supply an evidenced usable route, but it also cannot establish the sole cause of the original failure. It would be wrong to mark all networking healthy because the ECS endpoint exists, or to infer an observed packet drop from missing traffic records.
The network owner should supply the intended channel path and task-scoped configuration evidence first. If a private endpoint is proposed, its DNS and access boundaries need review; if approved egress already exists, establish that path instead. Neither alternative is selected automatically by this worksheet. The agent status and permission review remain retained prerequisites, not substituted connectivity tests.
Reuse a task-bound diagnostic worksheet
Keep one worksheet per failed attempt and target container. An incident involving two tasks needs two bindings, even when both use the same task definition. Share aliases in broad discussion and retain actual identifiers only in the approved operational record.
Failure and applicability
- Attempt: timestamp, exact error code/message, client evidence reference and collector. Exclude session tokens, stream URLs and command output containing private data.
- Target binding: account, Region, cluster, task identity and container name from the failed request and matching task evidence. Record any mismatch before continuing.
- Workload: launch type, operating system, actual platform or EC2 AMI/agent, network mode and explicit task role. Mark excluded or unknown branches.
- Lifecycle: task creation/start, task-definition revision, relevant overrides, service/launch configuration and collection times. Keep old and replacement task records separate.
Prerequisite and channel evidence
- Task and agent: separate task/container lastStatus; enableExecuteCommand value; exact container's ExecuteCommandAgent lastStatus, reason and lastStartedAt; missing or failed item reads.
- Caller authority: invoking identity and owned review for this request, including applicable conditions. Do not substitute permission to describe tasks.
- Agent authority: effective task-role binding, all four channel actions and owned review of applicable restrictions. Record execution role separately.
- Network: intended ssmmessages path, correct Region, DNS/configuration evidence, endpoint or egress context, security boundaries and observation gaps.
- Additional context: proxy-bypass configuration, filesystem/PID constraints, local client prerequisites, and customer-key or logging dependencies where configured.
Decision and next action
- Finding: supported mismatch or observation, its source and time. State which boundary it concerns and what it does not establish.
- Unknown: first unresolved field, evidence owner and smallest safe evidence request. A missing snapshot is not an absent permission or absent resource.
- Next action: retain the evidence-only disposition or raise a separately reviewed test/change proposal. Record approval scope, stop conditions and validation required before using the word fixed.
Choose an investigation that preserves the application
Configuration comparison is the first option when task identity, enablement or historical role binding is uncertain. It can identify an impossible prerequisite without opening a session, but cannot prove channel reachability. Existing agent, flow, DNS and authorization evidence can narrow a remaining fault if it belongs to the same target and incident window. Evidence from a replacement task is a comparator, not a rewrite of the original failure.
The AWS-linked Exec Checker is another option under separate approval. AWS describes it as calling APIs on the user's behalf. Read and pin its source, requested permissions and output handling before running it against an account. A green checker cannot authenticate a missing historical record or demonstrate that an intended command completed. This guide did not run the checker. AWS checker scope.
Consider a separately approved isolated reproduction only when existing evidence cannot distinguish the remaining hypotheses. Specify a non-sensitive target, scoped identity, known image and container, permitted command, bounded session and costs, inert downstream behavior, audit coverage and cleanup owner. For Linux, Exec commands run as root, so even a shell intended for observation can change application state or expose secrets. Do not experiment against customer traffic to finish the worksheet.
Any proposed replacement needs normal release and recovery authority. The separate ECS rollback eligibility article addresses whether an automatic completed-deployment target is available; an Exec connection says nothing about that recovery boundary. If a test creates external effects, ending its session does not reverse them.
Next action: assign the first unresolved boundary
Stop when diagnosis would require unapproved account access, credential exposure, a production command, broader permissions, a task replacement or a new network path. Retain the original attempt, intermediate findings and missing evidence. Do not turn a read-only investigation into a repair by changing several controls and calling the next successful shell a root-cause explanation.
Fill the worksheet's failure and applicability fields for one attempt. Ask the owner of its first unresolved boundary for the smallest existing configuration or observation record that would distinguish the remaining hypotheses. Close the diagnosis only when the evidence supports its finding. Any later claim of repair needs separately authorized validation against the intended task/container, the approved command and the required access and audit boundaries.