Expand description
TensorRT engine pre-warm persistence postconditions.
Two diagnostic predicates the worker calls after a prewarm sweep:
prewarm_persistence_postcondition_failed— fatal ERROR signal. Catches the catastrophic fresh-compiles → 0 engines on disk pattern that produced silent-persistence startup failures in production.prewarm_persistence_suspicious_undercount— non-fatal WARN signal. Retained for future extension; currently silent wheneverengine_count_after > 0(see inline note).
Both are pure functions over (fresh_compiles, engine_count_after) so
they can be unit-tested without spinning up an ORT session or a
filesystem fixture. The companion fixture-backed tests in tests.rs
exercise them through the count_engine_files snapshot mechanism that
the worker uses in production.
§Why engine_count_after, not engine_count_delta
ORT’s TRT EP stores one profile-based .engine file per fused subgraph
that covers all (batch, seq) shapes compiled so far via [min, max]
ranges per input dimension. When a new shape falls inside the existing
range the file is reused (cache hit); when it falls outside the range the
EP rewrites the file in-place with an expanded profile. Either way the
on-disk file count stays at 1 after the first compile — delta == 0 is
the normal steady-state, NOT a persistence failure.
The only actionable signal is engine_count_after == 0: the TRT EP
reported Ok(_) from session.run() yet wrote no engine file at all.
That is the exact failure mode from that incident class.
Constants§
- SUSPICIOUS_
UNDERCOUNT_ 🔒MIN_ FRESH - Minimum
fresh_compilescount below which the suspicious-undercount check is suppressed.
Functions§
- prewarm_
persistence_ 🔒postcondition_ failed - Decides whether a single worker’s prewarm postcondition is violated.
- prewarm_
persistence_ 🔒suspicious_ undercount - Decides whether the on-disk
.enginecount is suspiciously low relative to the number of fresh compiles, in a way not already caught byprewarm_persistence_postcondition_failed.