Skip to content

Getting Started

rustnn is a Rust crate. This page goes from an empty project to a graph that runs on a backend, then shows how to load graphs from files and how to use the command line tool.

Requirements

Requirement Needed for
Rust 1.97 or newer (edition 2024); rust-toolchain.toml pins it for the repository itself the crate
protoc, the Protocol Buffers compiler, on PATH build.rs compiles the ONNX and CoreML schemas
flatc the litert-runtime feature only (TFLite schema)
Node.js the WPT conformance tests only
A backend library execution; see Backends. The ONNX Runtime shared library works on every platform and is the usual starting point

On Windows run git config --system core.longpaths true before cloning; the repository contains paths longer than 260 characters.

Add the crate

The WebNN MLContext API is published on crates.io from version 0.6.0 (the 0.5.x releases were the earlier converter and loader crate). Add the crate with a backend feature:

[dependencies]
rustnn = { version = "0.6", features = ["onnx-runtime"] }

To follow unreleased changes on main, use the git dependency instead: rustnn = { git = "https://github.com/rustnn/rustnn", features = ["onnx-runtime"] }.

Features select backends and go on this dependency line. Without a backend feature the crate validates and converts graphs but cannot execute them. The full feature list is in the crate documentation (make docs-api writes it to target/doc/rustnn/index.html; the site publishes it under /api/) and in Backends.

Provide ONNX Runtime

The onnx-runtime feature loads the ONNX Runtime shared library at run time from the path in ORT_DYLIB_PATH. The ort crate rustnn is built against requires ONNX Runtime 1.29; the library in Windows System32 is older (1.17) and the process aborts with a BadVersion panic from ort when it is picked up, so set the variable before every run.

Download the pinned release. From the repository root with make installed:

make onnxruntime-download          # into target/onnxruntime/

Without make, fetch the same archive from the ONNX Runtime GitHub release v1.29.0 and unpack it into target/onnxruntime/; the archive names are onnxruntime-linux-x64-1.29.0.tgz, onnxruntime-osx-arm64-1.29.0.tgz and onnxruntime-win-x64-1.29.0.zip:

# Windows PowerShell
New-Item -ItemType Directory -Force target\onnxruntime | Out-Null
Invoke-WebRequest https://github.com/microsoft/onnxruntime/releases/download/v1.29.0/onnxruntime-win-x64-1.29.0.zip -OutFile target\onnxruntime\ort.zip
Expand-Archive target\onnxruntime\ort.zip -DestinationPath target\onnxruntime

Then point the variable at the library:

# Linux
export ORT_DYLIB_PATH=$PWD/target/onnxruntime/onnxruntime-linux-x64-1.29.0/lib/libonnxruntime.so.1.29.0
# macOS (Apple Silicon)
export ORT_DYLIB_PATH=$PWD/target/onnxruntime/onnxruntime-osx-arm64-1.29.0/lib/libonnxruntime.1.29.0.dylib
# Windows (Git Bash)
export ORT_DYLIB_PATH=$PWD/target/onnxruntime/onnxruntime-win-x64-1.29.0/lib/onnxruntime.dll

# Windows PowerShell
$env:ORT_DYLIB_PATH = "$PWD\target\onnxruntime\onnxruntime-win-x64-1.29.0\lib\onnxruntime.dll"

First graph

Follow the crate example, which creates a context, records relu(x + 1), compiles a graph and dispatches tensors. The example lives in src/lib.rs; API examples are maintained with the code.

Load a graph from a file

See Graph files and weights for the shared save/load/compile workflow and external-weight format links.

Command line tool

The rustnn binary validates a graph file, prints its inputs, outputs and dependency fan-out, and optionally exports or executes it. The commands below run from the repository root, where the sample graphs live in examples/ and --features selects the backend for this package. It needs a runtime feature at build time (without one it exits with "rustnn CLI requires a runtime feature"). Validation and conversion do not need ORT_DYLIB_PATH; --run-onnx does. Each --run-* flag exists only when its feature is compiled in (--run-onnx with onnx-runtime, --run-trtx with trtx-runtime, --run-coreml with coreml-runtime on macOS) and --help lists the flags of the current build.

# Validate and describe
cargo run --features onnx-runtime -- examples/sample_graph.webnn

# Graphviz export
cargo run --features onnx-runtime -- examples/sample_graph.webnn --export-dot target/graph.dot
dot -Tpng target/graph.dot -o target/graph.png

# Convert; formats are onnx, coreml and, with their features, trtx, litert and cann
cargo run --features onnx-runtime -- examples/sample_graph.webnn --convert onnx --convert-output target/graph.onnx

# Convert and execute once with zeroed inputs (ORT_DYLIB_PATH must be set for --run-onnx)
cargo run --features onnx-runtime -- examples/sample_graph.webnn --convert onnx --run-onnx
cargo run --features onnx-runtime,trtx-runtime -- examples/sample_graph.webnn --convert onnx --run-trtx
cargo run --features coreml-runtime -- examples/sample_graph.webnn --convert coreml --run-coreml   # macOS

--tensor-limit <bytes> raises the validator's tensor byte limit for very large models. The make run, make viz, make onnx and make coreml targets wrap these commands.

Next steps