Score vs. cost per question
Qwen models in color, every other model in grey
Provider
9 models tested, compared with 69 models from other labs.
Qwen models in color, every other model in grey
Qwen models in color, every other model in grey
Qwen models highlighted against every model in this run · higher is better
Qwen models highlighted against every model in this run · higher is better
Qwen models highlighted against every model in this run · lower is better
Qwen models highlighted against every model in this run · lower is better
Qwen models highlighted against every model in this run · lower is better
Qwen models highlighted against every model in this run · higher is better
| # | ||||||||
|---|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8 Max (0902) Qwen · served by Alibaba | 75.0 | 60 / 80 | 46 | 1.2 s | $0.011 | $2 / $6 | 1M |
| 2 | Qwen3.8 27B Qwen · served by Modal | 63.8 | 51 / 80 | 49 | 0.5 s | $0.0043 | $0.2 / $2.5 | 1M |
| 3 | Qwen3 Coder Next Qwen · served by Parasail | 62.5 | 50 / 80 | 106 | 0.4 s | $0.0017 | $0.12 / $0.8 | 262k |
| 4 | Qwen3.7 Plus Qwen · served by Alibaba | 60.0 | 48 / 80 | 62 | 1.1 s | $0.0033 | $0.32 / $1.28 | 1M |
| 5 | Qwen3.8 2.4T A95B Qwen · served by Venice | 57.5 | 46 / 80 | 44 | 3.1 s | $0.015 | $2 / $6 | 1M |
| 6 | Qwen3.6 35B A3B Qwen · served by DeepInfra | 37.5 | 30 / 80 | 135 | 0.4 s | $0.0037 | $0.1 / $0.95 | 262k |
| 7 | Qwen3.5 397B A17B Qwen · served by Phala | 32.5 | 26 / 80 | 240 | 0.6 s | $0.013 | $0.5 / $3.6 | 262k |
| 8 | Qwen3.5-122B-A10B Qwen · served by SiliconFlow | — | 0 / 0 | 108 | 1.2 s | — | $0.4 / $3.2 | 262k |
| 9 | Qwen3 Next 80B A3B Thinking | — | 0 / 0 | — | — | — | $0.15 / $1.2 | 262k |