Struct AppState 

pub struct AppState {
    pub pool: EmbedPool,
    pub ready: AtomicBool,
    pub max_batch: usize,
    pub total_workers: usize,
    pub max_seq_length: usize,
    pub tuning: OnceLock<TuningInfo>,
    pub cost_model: Arc<ArcSwap<CostModel>>,
    pub probe_status: AtomicU8,
    pub request_permits: Arc<Semaphore>,
}

Shared application state injected into every request handler via [axum::extract::State].

Fields

pool: EmbedPool

The embedding worker pool. Handles dense and sparse embedding requests.

ready: AtomicBool

Atomic flag set to true once model warm-up and readiness probes complete.

Handlers check this before dispatching to the pool to return 503 while models are still loading.

max_batch: usize

Maximum batch size enforced by the handler layer.

total_workers: usize

Total number of workers configured at startup.

Used by the /health endpoint to report degraded state when live_workers < total_workers.

max_seq_length: usize

Maximum tokenized sequence length in use.

tuning: OnceLock<TuningInfo>

Static memory-detection info written once before the probe starts.

Written to OnceLock as soon as memory detection completes (before the background probe finishes), so /health can show memory_source, available_bytes, and model_rss_bytes_per_worker even while the probe is still running.

cost_model: Arc<ArcSwap<CostModel>>

Live cost-model coefficients.

Initialized to conservative defaults at startup. Updated atomically by the background probe (or cache-hit path) once fitted coefficients are available. All workers share this same handle and observe the update lock-free on their next session.run() call.

probe_status: AtomicU8

Current state of the background memory probe.

Updated atomically from the background probe task. Read by /health to expose probe_status in the tuning block.

request_permits: Arc<Semaphore>

Concurrency gate for in-flight embedding requests.

Initialized to max(cfg_workers - 1, 1) permits, reserving one worker slot for the background auto-budget probe. Raised to cfg_workers atomically on every terminal probe-status transition (Disabled, CacheHit, Complete, Failed) so full concurrency is available once the probe no longer needs a reserved worker.

Test helpers set this to usize::MAX (effectively uncapped) so that existing tests do not need to acquire a permit.