Skip to main content

Module worker

Module worker 

Source
Expand description

Blocking worker thread, request dispatch, and probe wiring.

Submodules:

  • config: per-worker execution policy (WorkerConfig).
  • guard: lifecycle guard, JIT guard wiring, inference outcomes.
  • trt_retry: TRT JIT-OOM detection and workspace-halving retry.
  • propagation: engine propagation broadcast drain.
  • probe: probe inference and arena priming helpers.
  • prewarm_strict: prewarm postcondition readiness gate.
  • startup: model load, cache GC, TRT prewarm, readiness signal.
  • run: request loop and idle lifecycle.
  • dispatch: per-request dense/sparse/dual/probe/adaptive-warmup handling.
  • logging: abandoned-request observability.

Re-exportsยง

pub use config::WorkerConfig;

Modulesยง

config ๐Ÿ”’
Per-worker execution policy shared across the pool.
dispatch ๐Ÿ”’
Per-request dispatch for dense, sparse, dual, probe, and adaptive warmup.
guard ๐Ÿ”’
Worker lifecycle guard, in-band JIT guard wiring, and inference outcomes.
logging ๐Ÿ”’
Observability helpers for abandoned in-flight requests.
prewarm_strict ๐Ÿ”’
Prewarm postcondition readiness gate (BGE_M3_PREWARM_STRICT).
probe ๐Ÿ”’
Probe inference helpers for startup measurement and arena priming.
propagation ๐Ÿ”’
Engine propagation broadcast drain and post-inference cache-miss signaling.
run ๐Ÿ”’
Blocking worker thread and request dispatch loop.
startup ๐Ÿ”’
Worker startup: model load, optional cache GC, TRT prewarm, readiness signal.
trt_retry ๐Ÿ”’
TRT JIT-OOM detection and single retry with halved workspace budget.