Score vs. cost per question
DeepSeek models in color, every other model in grey
Provider
3 models tested, compared with 75 models from other labs.
DeepSeek#18DeepSeek V4.1 Flash 101773.8% verification score57 tok/s0.9 s first token$0.00091 / questionDeepSeek#24DeepSeek V4 Flash Vision Exp71.2% verification score170 tok/s0.7 s first token$0.0011 / questionDeepSeek#24DeepSeek V4 Pro 081371.2% verification score56 tok/s0.8 s first token$0.0036 / questionDeepSeek models in color, every other model in grey
DeepSeek models in color, every other model in grey
DeepSeek models highlighted against every model in this run · higher is better
DeepSeek models highlighted against every model in this run · higher is better
DeepSeek models highlighted against every model in this run · lower is better
DeepSeek models highlighted against every model in this run · lower is better
DeepSeek models highlighted against every model in this run · lower is better
DeepSeek models highlighted against every model in this run · higher is better
| # | ||||||||
|---|---|---|---|---|---|---|---|---|
| 1 | DeepSeek V4.1 Flash 1017 DeepSeek · served by Wafer | 73.8 | 59 / 80 | 57 | 0.9 s | $0.00091 | $0.15 / $0.6 | 1M |
| 2 | DeepSeek V4 Flash Vision Exp DeepSeek · served by DeepInfra | 71.2 | 57 / 80 | 170 | 0.7 s | $0.0011 | $0.2156 / $0.6468 | 1M |
| 3 | DeepSeek V4 Pro 0813 DeepSeek · served by Alibaba | 71.2 | 57 / 80 | 56 | 0.8 s | $0.0036 | $1.32 / $3.96 | 1M |