Skip to main content

wait_for_workers_to_exit

Function wait_for_workers_to_exit 

Source
pub(crate) async fn wait_for_workers_to_exit(
    live_workers: &Arc<AtomicUsize>,
    timeout: Duration,
)
Expand description

Polls live_workers until it reaches zero or timeout elapses.

Used by the warmup-only path after the EmbedPool handle has been dropped: closing the request channel asks each worker to break out of its receive loop, drop its ort::session::Session, and return — but dropping the pool only signals intent. The actual worker exit happens asynchronously on a spawn_blocking thread, and we want the ORT/TRT destructor to run BEFORE the process exits so any session-shutdown flushes (e.g. the TRT timing cache) make it to disk.

Returns silently on timeout — callers proceed to fsync + exit regardless. Engine plan files are durable independent of this wait (each shape fsyncs before returning from run_warmup_shape), so a slow worker shutdown only loses ancillary state.