Verdict
Built from the measurements below
- Accuracy is too close to call: Qwen3.7 Plus scored 60.0% and Qwen3.8 2.4T A95B 57.5% on the same 80 questions, a gap within what chance alone could produce.
- Qwen3.7 Plus is 4.7× cheaper per question as billed ($0.0033 vs $0.015), reasoning tokens included.
- Qwen3.7 Plus generates text 1.4× faster: 62 tokens per second against 44.
- Qwen3.7 Plus starts answering sooner: 1.1 s to the first token, against 3.1 s for Qwen3.8 2.4T A95B.
Which should you choose?
Choose Qwen3.7 Plus for the lower cost per correct answer ($0.0054 against $0.026), faster streaming and a quicker first token. Choose Qwen3.8 2.4T A95B for open weights you can run yourself and more hosts to choose from on OpenRouter (5 against 1). The widest subject gap is in humanities & social science: Qwen3.7 Plus answered 9 of 9 correctly, Qwen3.8 2.4T A95B 7 of 9. With so few questions per subject, treat this as a lead to test on your own prompts, not a result.
Accuracy gap +2.5 points (z = 0.32). We call a winner only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.