Verdict
Built from the measurements below
- Accuracy is too close to call: gpt-oss-20b scored 51.2% and Mercury 2.5 58.2% on the same 80 questions, a gap within what chance alone could produce.
- gpt-oss-20b is 1.7× cheaper per question as billed ($0.000066 vs $0.00011), reasoning tokens included.
- Mercury 2.5 generates text 1.5× faster: 157 tokens per second against 103.
- gpt-oss-20b starts answering sooner: 0.7 s to the first token, against 2.3 s for Mercury 2.5.
- Mercury 2.5 accepts more context: 260k tokens against 131k.
Which should you choose?
Choose gpt-oss-20b for the lower cost per correct answer ($0.00013 against $0.00019), a quicker first token, open weights you can run yourself and more hosts to choose from on OpenRouter (10 against 1). Choose Mercury 2.5 for faster streaming and a 260k-token context window. The widest subject gap is in economics: Mercury 2.5 answered 8 of 9 correctly, gpt-oss-20b 6 of 9. With so few questions per subject, treat this as a lead to test on your own prompts, not a result.
Accuracy gap -7.0 points (z = -0.88). We call a winner only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.