pub(super) const CACHE_HIT_THRESHOLD_MS: u64 = 5_000;Expand description
Threshold (ms) below which a session.run() is classified as a TRT
engine cache hit (loaded from disk) rather than a fresh compile.
Cold compiles take 30 000–170 000 ms; warm cache loads finish in
≤ 3 000 ms even for 32 × 8192 shapes. A 5 000 ms threshold gives a
comfortable margin above the observed maximum warm-load time while
remaining well below the minimum observed cold-compile time.