Skip to main content

Module postcondition

Module postcondition 

Source
Expand description

TensorRT engine pre-warm persistence postconditions.

Two diagnostic predicates the worker calls after a prewarm sweep:

Both are pure functions over (fresh_compiles, engine_count_after) so they can be unit-tested without spinning up an ORT session or a filesystem fixture. The companion fixture-backed tests in tests.rs exercise them through the count_engine_files snapshot mechanism that the worker uses in production.

§Why engine_count_after, not engine_count_delta

ORT’s TRT EP stores one profile-based .engine file per fused subgraph that covers all (batch, seq) shapes compiled so far via [min, max] ranges per input dimension. When a new shape falls inside the existing range the file is reused (cache hit); when it falls outside the range the EP rewrites the file in-place with an expanded profile. Either way the on-disk file count stays at 1 after the first compile — delta == 0 is the normal steady-state, NOT a persistence failure.

The only actionable signal is engine_count_after == 0: the TRT EP reported Ok(_) from session.run() yet wrote no engine file at all. That is the exact failure mode from that incident class.

Constants§

SUSPICIOUS_UNDERCOUNT_MIN_FRESH 🔒
Minimum fresh_compiles count below which the suspicious-undercount check is suppressed.

Functions§

prewarm_persistence_postcondition_failed 🔒
Decides whether a single worker’s prewarm postcondition is violated.
prewarm_persistence_suspicious_undercount 🔒
Decides whether the on-disk .engine count is suspiciously low relative to the number of fresh compiles, in a way not already caught by prewarm_persistence_postcondition_failed.