Score vs. cost per question
Meta models in color, every other model in grey
Provider
5 models tested, compared with 73 models from other labs.
Meta#7Muse Wonder78.8% verification score175 tok/s16.1 s first token$0.021 / questionMeta#9Muse Spark Open77.5% verification score220 tok/s0.5 s first token$0.0045 / questionMeta#40Llama 4 Maverick59.0% verification score13 tok/s0.7 s first token$0.00059 / questionMeta#52Llama 3.3 70B Instruct42.5% verification score42 tok/s0.7 s first token$0.00053 / questionMeta#54Llama 4 Scout41.2% verification score15 tok/s1.2 s first token$0.00037 / questionMeta models in color, every other model in grey
Meta models in color, every other model in grey
Meta models highlighted against every model in this run · higher is better
Meta models highlighted against every model in this run · higher is better
Meta models highlighted against every model in this run · lower is better
Meta models highlighted against every model in this run · lower is better
Meta models highlighted against every model in this run · lower is better
Meta models highlighted against every model in this run · higher is better
| # | ||||||||
|---|---|---|---|---|---|---|---|---|
| 1 | Muse Wonder Meta · served by Meta | 78.8 | 63 / 80 | 175 | 16.1 s | $0.021 | $4 / $20 | 1M |
| 2 | Muse Spark Open Meta · served by Together | 77.5 | 62 / 80 | 220 | 0.5 s | $0.0045 | $1.25 / $4.25 | 1M |
| 3 | Llama 4 Maverick Meta · served by DigitalOcean | 59.0 | 46 / 78 | 13 | 0.7 s | $0.00059 | $0.1875 / $0.6525 | 524k |
| 4 | Llama 3.3 70B Instruct Meta · served by DeepInfra | 42.5 | 34 / 80 | 42 | 0.7 s | $0.00053 | $0.1 / $0.32 | 131k |
| 5 | Llama 4 Scout Meta · served by DeepInfra | 41.2 | 33 / 80 | 15 | 1.2 s | $0.00037 | $0.1 / $0.3 | 1.3M |