Back to Candle

candle-quantized-qwen3

candle-examples/examples/quantized-qwen3/README.md

0.10.1547 B
Original Source

candle-quantized-qwen3

Qwen3 is an upgraded version of Qwen2.5, released by Alibaba Cloud.

Running the example

bash
cargo run --example quantized-qwen3 --release -- --prompt "Write a function to count prime numbers up to N."

0.6b is used by default, 1.7b, 4b, 8b, 14b, and 32b models are available via --which argument.

bash
cargo run --example quantized-qwen3 --release -- --which 4b   --prompt "A train is travelling at 120mph, how far does it travel in 3 minutes 30 seconds?"