Enum EpSelection 

pub enum EpSelection {
    Cpu,
    Cuda,
    TensorRt,
}

ONNX Runtime execution provider selection.

Controlled by BGE_M3_EP. Defaults to EpSelection::Cpu.

On macOS the CoreML EP is always used regardless of this setting.

Variants

Cpu

CPU inference via MLAS (default). Works everywhere, no GPU required.

Cuda

NVIDIA CUDA execution provider (requires cuda feature and a CUDA ORT build).

Set BGE_M3_EP=cuda to enable. BGE_M3_WORKERS is clamped to BGE_M3_GPU_COUNT so each worker is pinned to a distinct CUDA device. Set BGE_M3_GPU_COUNT to match the number of GPUs on the instance for maximum parallel inference throughput.

TensorRt

NVIDIA TensorRT execution provider (requires tensorrt feature and a TRT ORT build).

Set BGE_M3_EP=tensorrt to enable. Falls back to CUDA for ops TRT cannot handle. BGE_M3_WORKERS is clamped to BGE_M3_GPU_COUNT; each worker compiles its own per-GPU TRT shard of the warmup shapes during startup, enabling parallel engine compilation across GPUs.

Trait Implementations

impl Clone for EpSelection

fn clone(&self) -> EpSelection

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EpSelection

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for EpSelection

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for EpSelection

fn eq(&self, other: &EpSelection) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for EpSelection

impl Eq for EpSelection

impl StructuralPartialEq for EpSelection