ContextQMD
Back to Wasmtime

Image Classification Example

crates/wasi-nn/examples/classification-example/README.md

44.0.12.9 KB
Original Source

Image Classification Example

This example project demonstrates using the wasi-nn API to perform machine learning (ML) inference. It shows how to collect and use the various parts of a wasi-nn program:

  • an ML framework ("backend" in wasi-nn terms)
  • an ML model ("graph" in wasi-nn terms)
  • a WebAssembly program
  • a wasi-nn-compatible engine

Pre-requisite: Framework

This example uses the OpenVINO framework: installation instructions. If you're interested in how the engine forwards calls from the WebAssembly program to this framework, see the backend source code.

Pre-requisite: Model

MobileNet is a small, common model for classifying images; it returns the probabilities for words that best describe the image. To retrieve the files needed to use this model on OpenVINO, download:

wget https://download.01.org/openvinotoolkit/fixtures/mobilenet/mobilenet.bin -O fixture/model.bin
wget https://download.01.org/openvinotoolkit/fixtures/mobilenet/mobilenet.xml -O fixture/model.xml
wget https://download.01.org/openvinotoolkit/fixtures/mobilenet/tensor-1x224x224x3-f32.bgr -O fixture/tensor.bgr

The .bgr file is a tensor representation of an image file (more details here) which is the input to this model, i.e., the image to be classified.

Pre-requisite: Program

Compile this Rust example to a WebAssembly program using the wasm32-wasip1 target. This requires a Rust toolchain (e.g., rustup) and the appropriate compilation target (e.g., rustup target add wasm32-wasip1). To compile the program to a *.wasm file in the target directory:

cargo build --target=wasm32-wasip1

Pre-requisites: Engine

This example uses Wasmtime, which contains a wasi-nn implementation. To use Wasmtime, follow the instructions to either build or install it.

Run

With the pre-requisites in place, run the example:

<path>/<to>/wasmtime run --wasi=nn --dir=fixture target/wasm32-wasip1/debug/wasi-nn-example.wasm

Some words of explanation: the --wasi flag enables the wasi-nn proposal (see -S help), the --dir maps our host-side fixture directory to a directory of the same name in the guest, and we pass the *.wasm module as the sole argument. For this model (see the source), we expect to see the list of tags that mostly likely describe the image:

...
Found results, sorted top 5: [InferenceResult(885, 0.3958254), InferenceResult(904, 0.36464655), InferenceResult(84, 0.010480323), InferenceResult(911, 0.0082290955), InferenceResult(741, 0.007244849)]