Module embedder 

Worker-pool–driven BGE-M3 embedding service.

Submodules:

• types: public DTOs and the internal EmbedRequest enum.
• error: small ort::Error → anyhow::Error adapter.
• model_files: hf-hub download / cache layout for the ONNX model files.
• tokenize: tokenizer load + no-pad tokenization + chunk-array build.
• session: ORT execution-provider config and session loading.
• sm_detect: per-device GPU compute-capability detection (smXY) used to filter the TRT engine cache by the worker’s own SM.
• math: pure dense/sparse math helpers (testable without ORT).
• dense: dense embedding pipeline.
• sparse: BGE-M3 SPLADE-style sparse embedding pipeline.
• dual: paired dense + sparse embedding pipeline (one forward pass).
• jit_guard: in-band TensorRT JIT admission guard that refuses chunks whose sequence length is dangerous and uncovered by the warmed engine profile, preventing the process-killing pathological autotuner allocation.
• trt_cache: TensorRT engine-cache path construction, inspection, and durability (fsync after compile). Submodules: paths, inspect, enumerate, prewarm_log, fsync.
• trt_cache_gc (feature cache-gc, off by default): destructive stale-SM engine plan garbage collection — only present in dedicated maintenance / dev binaries.
• trt_warmup: TensorRT engine pre-warming during worker startup.
• worker: blocking worker thread, request dispatch, probe wiring. Submodules: config, guard, trt_retry, propagation, probe, prewarm_strict, startup, run, dispatch, logging.
• pool: EmbedPool async wrapper and test helpers.
• adaptive_warmup: adaptive in-process background warmup loop for TRT engine cache miss recovery.

Modules

adaptive_warmup 🔒

Adaptive in-process background warmup loop.

dense 🔒

Dense embedding pipeline.

dual 🔒

Paired dense + sparse embedding pipeline (one forward pass per chunk).

error 🔒

Error helpers for the embedder.

jit_guard 🔒

In-band TensorRT JIT admission guard.

math 🔒

Pure dense/sparse math helpers (testable without ORT).

model_files 🔒

HuggingFace Hub download + cache-layout helpers for the BGE-M3 model files.

pool 🔒

EmbedPool async wrapper around the worker thread pool.

session 🔒

ORT execution-provider configuration and session loading.

sm_detect 🔒

Per-device GPU compute-capability detection for SM-aware TRT cache filtering.

sparse 🔒

BGE-M3 SPLADE-style sparse embedding pipeline.

tokenize 🔒

Tokenizer load + no-pad tokenization + chunk-array build helpers.

trt_cache 🔒

TensorRT engine cache path construction, inspection, and durability.

trt_warmup 🔒

TensorRT engine pre-warming: compiles and caches engine files during worker startup so the first real request hits a cached engine instead of triggering an on-demand 30–170 s compile.

types 🔒

Public DTOs and the internal EmbedRequest enum exchanged between the pool and the worker threads.

worker 🔒

Blocking worker thread, request dispatch, and probe wiring.

Structs

EmbedPool

Async handle to the embedding worker thread pool.