Skip to main content

embed_sparse

bge_m3_embedding_server::embedder::sparse

Function embed_sparse 

Source 
pub(super) fn embed_sparse(
    session: &mut Session,
    tokenizer: &Tokenizer,
    texts: &[String],
    cost_model: &CostModel,
    model_variant: ModelVariant,
) -> Result<(Vec<SparseEmbedding>, EmbedStats)>
Expand description

Produces sparse embeddings via the BGE-M3 sparse-linear projection.

Tokenizes once, then uses the cost model to bin-pack into chunks. Results are scattered back to the original input order.