Score vs. cost per question
Anthropic models in color, every other model in grey
Provider
4 models tested, compared with 74 models from other labs.
Anthropic#1Claude Opus 5.585.0% verification score129 tok/s2.3 s first token$0.015 / questionAnthropic#5Claude Fable 5.181.2% verification score70 tok/s2.1 s first token$0.016 / questionAnthropic#7Claude Sonnet 5.578.8% verification score126 tok/s0.7 s first token$0.0054 / questionAnthropic#26Claude Haiku 5.568.8% verification score192 tok/s0.7 s first token$0.00048 / questionAnthropic models in color, every other model in grey
Anthropic models in color, every other model in grey
Anthropic models highlighted against every model in this run · higher is better
Anthropic models highlighted against every model in this run · higher is better
Anthropic models highlighted against every model in this run · lower is better
Anthropic models highlighted against every model in this run · lower is better
Anthropic models highlighted against every model in this run · lower is better
Anthropic models highlighted against every model in this run · higher is better
| # | ||||||||
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 Anthropic · served by Claude Platform on AWS | 85.0 | 68 / 80 | 129 | 2.3 s | $0.015 | $4 / $20 | 1M |
| 2 | Claude Fable 5.1 Anthropic · served by Anthropic | 81.2 | 65 / 80 | 70 | 2.1 s | $0.016 | $10 / $50 | 1M |
| 3 | Claude Sonnet 5.5 Anthropic · served by Claude Platform on AWS | 78.8 | 63 / 80 | 126 | 0.7 s | $0.0054 | $2 / $10 | 1M |
| 4 | Claude Haiku 5.5 Anthropic · served by Claude Platform on AWS | 68.8 | 55 / 80 | 192 | 0.7 s | $0.00048 | $0.1 / $0.5 | 1M |