Skip to main content

embed_dense

bge_m3_embedding_server::embedder::dense

Function embed_dense 

Source 
pub(super) fn embed_dense(
    session: &mut Session,
    tokenizer: &Tokenizer,
    texts: &[String],
    cost_model: &CostModel,
    model_variant: ModelVariant,
    guard: Option<&TrtJitGuard>,
) -> Result<(Vec<Vec<f32>>, EmbedStats)>
Expand description

Produces L2-normalized dense embeddings.

Tokenizes once, then uses the cost model to bin-pack into chunks that fit within the workspace budget. Results are scattered back to the original input order.

guard is the in-band TRT JIT admission guard (see super::jit_guard). When Some, any chunk whose sequence length is in the dangerous range and uncovered by the warmed engine profile is refused (returning super::jit_guard::TrtJitRejection) before any session.run() executes, so a single request fails with HTTP 503 instead of risking a process-killing pathological autotuner allocation.