Skip to main content

guard_chunks

Function guard_chunks 

Source
pub(crate) fn guard_chunks(
    guard: Option<&TrtJitGuard>,
    chunks: &[Vec<usize>],
    seq_lens: &[usize],
) -> Result<(), TrtJitRejection>
Expand description

Validates every chunk produced by bin_pack against an optional guard.

Returns Err(TrtJitRejection) for the first chunk that would trigger a dangerous in-band JIT, so the whole request is refused atomically before any session.run() executes (no partially-computed output). A None guard admits everything.

chunks are the original-index groups returned by crate::binpack::bin_pack; seq_lens is the per-text tokenized length (indexed by original position). The per-chunk shape is (chunk.len(), max(seq_lens[i] for i in chunk)).