Skip to main content

drain_engine_propagation

Function drain_engine_propagation 

Source
pub(super) fn drain_engine_propagation<F>(
    rx: &mut Receiver<(usize, usize)>,
    warmed_local: &mut HashSet<(usize, usize)>,
    worker_id: usize,
    prewarm: F,
)
where F: FnMut((usize, usize)),
Expand description

Drains pending broadcast notifications and runs trt_prewarm for each new shape.

Called at the start of each worker loop iteration (between requests) so peers eagerly warm their in-memory TRT profile before the next real request for a new shape arrives.

warmed_local tracks shapes already warmed by this worker in the current session. The originating worker self-skips on subsequent drains because log_inference_complete inserts the broadcast shape into warmed_local at the call site before returning control to the request loop.