Struct WorkerConfig 

pub struct WorkerConfig {
    pub cost_model: Arc<ArcSwap<CostModel>>,
    pub idle_timeout: Option<Duration>,
    pub model_variant: ModelVariant,
    pub max_seq_length: usize,
    pub intra_threads: usize,
}

Execution-policy configuration shared by all workers.

cost_model is an Arc<ArcSwap<CostModel>> so all workers share a single handle and the background probe can update the cost model atomically after fitting. Each worker loads the current value lock-free at the start of every session.run() call via config.cost_model.load().

Fields

cost_model: Arc<ArcSwap<CostModel>>

Quadratic-aware workspace cost model and per-worker budget.

Shared across all workers via ArcSwap. The background probe updates this handle once fitted coefficients are available; workers observe the new model on their next request without any coordination or restart.

idle_timeout: Option<Duration>

Duration of inactivity before workers unload their model instances.

model_variant: ModelVariant

ONNX model variant to load (FP32, FP16, or INT8).

max_seq_length: usize

Maximum tokenized sequence length.

intra_threads: usize

Number of intra-op threads each ORT session may use for a single session.run() call. Plumbed through to load_session at model load time. See crate::config::Config::intra_threads for sizing guidance.

Trait Implementations

impl Clone for WorkerConfig

fn clone(&self) -> WorkerConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.