Verdict
Built from the measurements below
- Accuracy is too close to call: Nemotron 3 Ultra scored 60.0% and Qwen3.8 2.4T A95B 57.5% on the same 80 questions, a gap within what chance alone could produce.
- Nemotron 3 Ultra is 2.7× cheaper per question as billed ($0.0057 vs $0.015), reasoning tokens included.
- Nemotron 3 Ultra generates text 3.1× faster: 136 tokens per second against 44.
- Nemotron 3 Ultra starts answering sooner: 0.8 s to the first token, against 3.1 s for Qwen3.8 2.4T A95B.
- Qwen3.8 2.4T A95B accepts more context: 1M tokens against 262k.
Which should you choose?
Choose Nemotron 3 Ultra for the lower cost per correct answer ($0.0094 against $0.026), faster streaming and a quicker first token. Choose Qwen3.8 2.4T A95B for a 1M-token context window and more hosts to choose from on OpenRouter (5 against 3). The widest subject gap is in humanities & social science: Nemotron 3 Ultra answered 9 of 9 correctly, Qwen3.8 2.4T A95B 7 of 9. With so few questions per subject, treat this as a lead to test on your own prompts, not a result.
Accuracy gap +2.5 points (z = 0.32). We call a winner only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.