Module config 

Server configuration loaded from environment variables at startup.

All settings are read once via Config::from_env and then immutable for the server’s lifetime. See each field’s doc comment for the corresponding environment variable name and default value.

Structs

Config

Runtime configuration loaded from environment variables.

Enums

EpSelection

ONNX Runtime execution provider selection.

ModelVariant

ONNX model variant to load.

Constants

DEFAULT_GPU_VRAM_BUDGET_BYTES 🔒

VRAM workspace budget used for GPU EPs when BGE_M3_GPU_VRAM_BUDGET_BYTES is unset.

DEFAULT_TRT_WARMUP_SHAPES 🔒

Default TRT warmup shapes: a 2D {1, 2, 4, 8, 16, 32} × {128, 512, 2048, 8192} grid composed in batch-major order.

MODEL_MAX_SEQ

Maximum sequence length supported by the model architecture. BGE-M3’s positional embedding table extends to 8192; this is the hard upper bound used to validate BGE_M3_MAX_SEQ_LENGTH.

VRAM_WARN_THRESHOLD_BYTES 🔒

Advisory upper-bound for VRAM byte values (128 GiB).

Functions

parse_trt_warmup_shapes 🔒

Parses BGE_M3_TRT_WARMUP_SHAPES from its raw env-var value.

resolve_cost_model_override 🔒

Resolves an optional CostModel from env vars that explicitly override auto-tuning.

warn_if_small_batch_coverage_missing 🔒

Emits a startup WARN when the resolved warmup shape grid does not cover the small-batch shapes that real router traffic routinely bin-packs to.