Module trt_retry 

TRT JIT-OOM detection and single retry with halved workspace budget.

Constants

CHUNK_CACHE_HIT_THRESHOLD_MS 🔒

Mirrors trt_warmup::CACHE_HIT_THRESHOLD_MS. Used to classify a per-request inference as a probable TRT engine cache miss so the adaptive warmup task can proactively compile the engine.

Functions

embed_with_trt_retry 🔒

Wraps an embed call with the standard TRT JIT-OOM retry-once-with-halved-budget pattern.

is_trt_engine_build_fatal 🔒

Returns true when an ORT error string indicates a TRT engine build failure severe enough that the worker should exit rather than retry.

is_trt_jit_oom 🔒

Returns true when an ORT error string indicates a TRT JIT workspace overflow — a condition that may resolve with a smaller batch or halved workspace budget.