SDK Behavioral Contract
This document is the normative cross-SDK contract for App Mesh client SDKs (Python, Go, Rust, Java, JavaScript). Each SDK implements these guarantees independently; when changing one SDK, update the others (or this document) in the same change. Per-SDK code comments describe implementation detail — this document defines the behavior.
Applies to the TCP and WSS transports (AppMeshClientTCP / AppMeshClientWSS
and equivalents). The HTTP transport has no demuxer; its per-request semantics
come from the HTTP library.
TCP Wire Framing
| Constant | Value |
|---|---|
| Frame header | 8 bytes: 4-byte magic + 4-byte body length, both big-endian |
| Magic number | 0x07C707F8 |
| Max body length | 1 GiB (1024 * 1024 * 1024) — reject larger frames |
The length field excludes the 8-byte header. Bodies are msgpack-encoded
Request/Response messages. These constants are part of the daemon wire
protocol and must never change.
Message Demuxer
Each subscription-capable transport runs a demuxer: one background reader that
routes every incoming Response either to a pending request waiter (matched by
request uuid) or to an event subscription callback.
Event push identification
Server-push events are Response messages with request_uri = "/appmesh/event"
(see EventSubscription.md for the body format). The
subscription ID is taken from the body’s subscription_id, falling back to the
X-Subscription-Id header.
Register pending before send
A request’s uuid MUST be registered with the demuxer before the request
bytes are written to the socket. Otherwise a fast response can arrive before
the waiter exists and be dropped (conformance scenario S7).
Request timeout policy
Requests routed through the demuxer have no artificial client-side wait cap: a request slower than the transport receive timeout (e.g. > 60s) must still complete (S1). An empty/None/closed-channel result from the demuxer means the demuxer stopped — i.e. the transport disconnected — never “slow request”. On that signal the client closes the transport and raises/returns a connection error (not a timeout).
Event ordering
Guaranteed (normative): events of one subscription are delivered to its callback serially, in arrival order (the server’s per-subscription monotonic
sequenceorder).Not guaranteed: cross-subscription ordering. Some implementations happen to provide it; callers MUST NOT rely on it.
| SDK | Dispatch mechanism | Incidental cross-subscription FIFO? |
|---|---|---|
| Python | single global dispatch thread + FIFO queue | yes |
| Go | single dispatch goroutine + FIFO channel | yes |
| JavaScript | synchronous dispatch on the event loop | yes |
| Java | one single-threaded executor per subscription | no |
| Rust | one worker task per subscription | no |
A slow callback may delay later events of its subscription but MUST never block the socket reader.
Synthetic __disconnected__ event
When the demuxer stops (transport error, close()), it pushes a synthetic
event with event_type = "__disconnected__" to every registered callback,
carrying only subscription_id and event_type. This is client-local — the
daemon never sends it — and exists so long-running waits (e.g.
wait_for_async_run) unblock instead of hanging (S2). Pending request waiters
are woken with the empty/disconnect result at the same time.
Constant name per SDK: EVENT_TYPE_DISCONNECTED (Python/Rust/JS/Go as
EventTypeDisconnected, Java MessageDemuxer.EVENT_TYPE_DISCONNECTED).
Pre-registration event buffering (atomic-subscribe race)
Events can arrive between server-side subscription creation and the client
registering its callback (e.g. atomic add_app(subscribe_events=...) on a fast
app whose output is pushed before add_app returns). The demuxer buffers such
events per subscription ID and flushes them — under the same lock, so buffered
events precede any later live event — when the callback registers (S4).
Bounds (identical in all SDKs, S5):
| Cap | Value | Overflow policy |
|---|---|---|
| Distinct unregistered subscription IDs | 64 | events for further IDs dropped |
| Buffered events per subscription ID | 1000 | drop oldest |
Buffers are discarded on demuxer stop and on unsubscribe.
wait_for_async_run (subscribe-based wait)
Contract: subscribe to STDOUT, EXIT, REMOVED for the run’s app; then
backfill output already emitted (get_app_output from position 0, which also
detects an already-exited process); deduplicate stdout by byte position to
bridge backfill and live events; wait for EXIT / REMOVED /
__disconnected__ / caller timeout.
Termination signaling per SDK
| Outcome | Python | Go | Rust | JavaScript | Java |
|---|---|---|---|---|---|
| Process exited | returns exit code | (&code, nil) |
Ok(Some(code)) |
resolves exit code | returns exit code |
| Caller timeout | returns None |
(nil, nil) |
Ok(None) |
resolves null |
returns null |
| App removed before exit | raises AppMeshAppRemovedError |
(nil, ErrAppRemoved) |
Err(AppMeshError::AppRemoved) |
throws AppRemovedError |
throws AppRemovedException |
| Transport disconnected | raises AppMeshConnectionError |
(nil, ErrTransportDisconnected) |
Err(AppMeshError::TransportDisconnected) |
throws TransportDisconnectedError |
throws TransportDisconnectedException |
Exit codes may be negative (signal kills, e.g. -2 = SIGINT) and must
round-trip as exit codes, not be conflated with error sentinels (S6).
Cleanup policy
Unsubscribe: best-effort after the wait ends — except when the transport is already dead (disconnect observed), where sending an unsubscribe request would register a waiter that never gets a response.
Delete-on-exit: best-effort
delete_appof the transient run app only after a real observed exit. Never afterREMOVED(the app is already gone) and never after a disconnect (the daemon is unreachable and may still be running the process) (S8).
Auth Token Synchronization (TCP/WSS)
The HTTP transport relies on Set-Cookie and the HTTP library’s cookie jar.
TCP/WSS transports must extract the new access_token from auth endpoint JSON
bodies themselves — only on HTTP 200, and only for these paths:
| Path | When to apply the body's access_token |
|---|---|
/appmesh/login, /appmesh/auth, /appmesh/totp/validate |
only when the request carried X-Set-Cookie: true |
/appmesh/token/renew, /appmesh/totp/setup |
always (client already has an active session) |
/appmesh/self/logoff |
clear the cached token |
This list is duplicated in Python (transport_mixin.py), Go (requester.go),
Rust (requester.rs), and JavaScript (appmesh_tcp.js); keep all of them —
and this table — in sync.
Worker Task Loop (fetch_task / send_task_result)
The worker half of the client/worker model: an App Mesh-managed application process polls the daemon for task payloads and returns results. Applies to all transports (HTTP included — this loop has no demuxer dependency).
Canonical type name per language
The task-loop helper’s canonical name is the Worker form.
| Language | Canonical |
|---|---|
| Python | AppMeshWorker (+ TCP/WSS) |
| Go | WorkerHTTPContext (+ TCP/WSS) |
| Rust | AppMeshWorker/AppMeshWorkerTCP/AppMeshWorkerWSS |
| Java | AppMeshWorker (+ TCP/WSS) |
| JavaScript | AppMeshWorker/AppMeshWorkerTCP |
Canonical method names per language
The two task-loop methods:
| Language | Canonical |
|---|---|
| Python | fetch_task / send_task_result |
| Go | FetchTask(Context) / SendTaskResult |
| Rust | fetch_task / send_task_result |
| Java | fetchTask / sendTaskResult |
| JavaScript | fetch_task / send_task_result |
| Operation | Endpoint |
|---|---|
| Fetch task | GET /appmesh/app/{app_name}/task?process_key=... |
| Return result | PUT /appmesh/app/{app_name}/task?process_key=... |
APP_MESH_PROCESS_KEY and APP_MESH_APPLICATION_NAME are injected by the
daemon; a missing variable is an immediate error, never retried.
Retry policy
Normative: the fetch loop retries indefinitely with a fixed 100 ms floor per attempt — if an attempt (request + failure handling) took less than 100 ms, sleep the remainder; otherwise retry immediately. No backoff.
Python additionally accepts an optional max_retries cap (exhaustion raises
AppMeshError); the other SDKs retry forever.
Superseded process (HTTP 412)
HTTP 412 on fetch means this process key was superseded by a newer process
instance; the loop MUST stop immediately (no retry) and surface a typed
error — never call exit() from library code:
| SDK | 412 signal |
|---|---|
| Python | raises AppMeshProcessSupersededError |
| Go | returns ErrProcessSuperseded |
| Rust | Err(AppMeshError::ProcessSuperseded) |
| Java | throws ProcessSupersededException |
| JavaScript | throws ProcessSupersededError |
Cancellation signaling per SDK
Cancellation is checked at least once per retry iteration; SDKs whose
mechanism allows it (Python stop_event.wait, Go/Rust select) also abort
the retry sleep or the in-flight request. Java and JavaScript check a flag per
iteration, so worst-case cancel latency is one attempt plus the 100 ms floor.
| SDK | Cancel mechanism | Fetch result on cancel |
|---|---|---|
| Python | stop_event (threading.Event) argument |
raises AppMeshError |
| Go | context.Context (FetchTaskContext) |
returns wrapped ctx.Err() |
| Rust | stop() (watch channel) |
Err(AppMeshError::Cancelled) |
| Java | stop() or thread interrupt |
returns null |
| JavaScript | stop() (per-iteration flag) |
resolves null |
Conformance Scenarios
Each SDK’s test suite should cover these named scenarios; when fixing a bug in one SDK’s demuxer/wait path, add or check the matching scenario in the others.
| # | Scenario | Expected behavior |
|---|---|---|
| S1 | Demuxer-routed request slower than the transport receive timeout (> 60s) | completes normally; no spurious timeout |
| S2 | Transport disconnects mid-wait_for_async_run |
wait unblocks promptly with disconnect signaling (see matrix); no hang |
| S3 | Caller timeout mid-wait | timeout result (see matrix); subscription unsubscribed |
| S4 | Atomic add_app(subscribe_events) on a fast app |
events pushed before callback registration are buffered and flushed in order; none lost |
| S5 | Event flood for a never-registered subscription | memory bounded by 64-sub / 1000-event caps; drop-oldest within a sub |
| S6 | Process killed by signal (negative exit code) | negative code returned as the exit code, not treated as an error/sentinel |
| S7 | Response arrives immediately after send | not dropped (pending waiter registered before send) |
| S8 | App removed while waiting | app-removed signaling; no delete_app attempt |
| S9 | Token renew while other demuxer traffic is in flight | renew reply matched by UUID and applied; unrelated responses not cross-wired |
Coverage status
Covering tests carry a greppable Conformance: S<n> comment (or the scenario
ID in the test name). Update this table in the same change that adds or
removes a covering test. partial = exercises the mechanism but not the
race/edge the scenario names; MISSING = no test at all.
| # | Python | Go | Rust | Java | JavaScript |
|---|---|---|---|---|---|
| S1 | MISSING | MISSING | MISSING | MISSING | MISSING |
| S2 | test/test_appmesh_client.py test_s2_disconnect_unblocks_wait |
subscribe_test.go TestWaitForAsyncRunDisconnectUnblocks |
partial: src/subscribe.rs conformance_s2_disconnect_broadcast_unblocks + src/wait_subscribe.rs conformance_s2_disconnected_event_classified (wait path not driven) |
AsyncRunWaiterTest testDisconnectUnblocksWait |
test/subscribe_test.js wait_for_async_run disconnect unblocks with typed error |
| S3 | MISSING | MISSING | MISSING | MISSING | MISSING |
| S4 | partial: test/test_appmesh_client.py test_66_add_app_with_subscribe_events |
MISSING | MISSING | MISSING | MISSING |
| S5 | MISSING | MISSING | MISSING | MISSING | MISSING |
| S6 | test/test_appmesh_client.py test_s6_negative_exit_code |
subscribe_test.go TestWaitForAsyncRunNegativeExitCode |
partial: src/wait_subscribe.rs conformance_s6_negative_exit_code_is_exit (callback classification only) |
AsyncRunWaiterTest testNegativeExitCodeReturnedAsExitCode |
test/subscribe_test.js wait_for_async_run returns negative exit code as-is |
| S7 | test/test_appmesh_client.py test_s7_response_races_send |
partial: subscribe_test.go TestMessageDemuxerRequestResponse |
partial: src/subscribe.rs conformance_s7_response_routed_to_pre_registered_waiter |
partial: SubscribeTest testDemuxerRoutesResponseToPreRegisteredWaiter |
partial: test/subscribe_test.js MessageDemuxer routes responses by UUID |
| S8 | MISSING | MISSING | MISSING | MISSING | MISSING |
| S9 | MISSING | MISSING | MISSING | MISSING | MISSING |