Skip to main content

Module paths

bge_m3_embedding_server::embedder::trt_cache

Module paths 

Source
Expand description

Cache directory paths and startup snapshot type.

Structs§

TrtCacheInfo 🔒
Summary of the TRT engine cache state, produced at startup so the operator can see in CloudWatch whether the persistent volume is actually being reused between container restarts.

Constants§

TRT_ENGINE_SUBDIR 🔒
Subdirectory under BGE_M3_CACHE_DIR where ORT/TRT writes engine plan files.
TRT_TIMING_SUBDIR 🔒
Subdirectory under BGE_M3_CACHE_DIR where TRT writes the timing cache.

Functions§

engine_cache_path 🔒
Returns the canonical TRT engine-cache directory for a given root cache.
timing_cache_path 🔒
Returns the canonical TRT timing-cache file path for a given root cache.