Expand description
Blocking worker thread, request dispatch, and probe wiring.
Submodules:
config: per-worker execution policy (WorkerConfig).guard: lifecycle guard, JIT guard wiring, inference outcomes.trt_retry: TRT JIT-OOM detection and workspace-halving retry.propagation: engine propagation broadcast drain.probe: probe inference and arena priming helpers.prewarm_strict: prewarm postcondition readiness gate.startup: model load, cache GC, TRT prewarm, readiness signal.run: request loop and idle lifecycle.dispatch: per-request dense/sparse/dual/probe/adaptive-warmup handling.logging: abandoned-request observability.
Re-exportsยง
pub use config::WorkerConfig;
Modulesยง
- config ๐
- Per-worker execution policy shared across the pool.
- dispatch ๐
- Per-request dispatch for dense, sparse, dual, probe, and adaptive warmup.
- guard ๐
- Worker lifecycle guard, in-band JIT guard wiring, and inference outcomes.
- logging ๐
- Observability helpers for abandoned in-flight requests.
- prewarm_
strict ๐ - Prewarm postcondition readiness gate (
BGE_M3_PREWARM_STRICT). - probe ๐
- Probe inference helpers for startup measurement and arena priming.
- propagation ๐
- Engine propagation broadcast drain and post-inference cache-miss signaling.
- run ๐
- Blocking worker thread and request dispatch loop.
- startup ๐
- Worker startup: model load, optional cache GC, TRT prewarm, readiness signal.
- trt_
retry ๐ - TRT JIT-OOM detection and single retry with halved workspace budget.