Module guard 

Worker lifecycle guard, in-band JIT guard wiring, and inference outcomes.

Structs

EmbedRouteContext 🔒

Per-route context passed to finalize_embed_route after inference.

WorkerGuard 🔒

Enums

InferenceOutcome 🔒

Outcome of a single inference call in the worker request loop.

Functions

adaptive_warmup_non_trt_compile_ms 🔒

Adaptive-warmup no-op compile duration for non-TRT execution providers.

build_shape_guard 🔒

Builds the per-request in-band TRT JIT guard from the worker config and the live pool-wide warmed-sequence ceiling.

classify_inference_outcome 🔒

Maps inference error flags to the worker-loop InferenceOutcome.

finalize_embed_route 🔒

Shared post-inference path for dense, sparse, and dual routes.

log_client_abandoned_before_dispatch 🔒

Emits the pre-dispatch abandonment WARN when the client disconnected while the request was still queued.

log_guard_rejection 🔒

Emits a WARN describing an in-band TRT JIT guard refusal.

next_consecutive_failures 🔒

Returns the consecutive-failure counter value after applying an outcome.

should_unload_on_outcome 🔒

Returns true when the worker loop should unload models after inference.