Changelog
Newly benchmarked models
Each model appears on the date its latest results were recorded. Scored models were tested on the 80-question verification suite; the others have speed and price measurements so far.
GPT-5.3-CodexOpenAIScored 82.5%
Claude Fable 5.1AnthropicScored 81.2%
Grok 4.7SpaceXAIScored 80.0%
Muse WonderMetaScored 78.8%- GPT-6 AstraOpenAIScored 76.2%
- GPT-6.1 Terra ProOpenAIScored 76.2%
- GPT-5.6 TerraOpenAIScored 75.0%
- GPT-6.1 TerraOpenAIScored 73.8%
- o3OpenAIScored 72.5%
InklingThinking MachinesScored 66.2%
Kimi K2.7 CodeMoonshotAIScored 65.0%
Step 5 PreviewStepFunScored 63.8%
Qwen3.7 PlusQwenScored 60.0%- Qwen3.8 2.4T A95BQwenScored 57.5%
Command ACohereScored 45.0%- Llama 4 ScoutMetaScored 41.2%
Ling 3.1 FlashinclusionAIScored 41.2%
Nemotron 3.5 Super VLNVIDIAScored 40.0%- Qwen3.5 397B A17BQwenScored 32.5%
Gemma 4 31BGoogleScored 30.0%
Mistral Medium 3.5MistralScored 28.8%
Hermes 4 405BNousScored 26.2%- Gemma 4 26B A4BGoogleScored 15.0%
Reka Flash 3rekaaiScored 1.2%
- Gemini 3.1 Pro PreviewGoogleScored 77.5%
- Muse Spark OpenMetaScored 77.5%
- Gemini 3.5 Flash LiteGoogleScored 75.0%
DeepSeek V4 Flash Vision ExpDeepSeekScored 71.2%- DeepSeek V4 Pro 0813DeepSeekScored 71.2%
GLM 5.3 FlashZ.aiScored 67.5%- Inkling SmallThinking MachinesScored 67.5%
- Qwen3.8 27BQwenScored 63.8%
MiMo-V2.6-FlashXiaomiScored 62.5%- Qwen3 Coder NextQwenScored 62.5%
- Nemotron 3 UltraNVIDIAScored 60.0%
Hy3TencentScored 58.8%
Mercury 2.5InceptionScored 58.2%
MiniMax M3MiniMaxScored 57.5%- gpt-oss-20bOpenAIScored 51.2%
- Mistral Large 4MistralScored 50.0%
LongCat 2.0MeituanScored 48.8%
Solar Pro 4UpstageScored 45.0%- Nemotron 3.5 LightningNVIDIAScored 43.8%
- Ministral 3 8B 2512MistralScored 42.5%
- Llama 3.3 70B InstructMetaScored 42.5%
Phi 4MicrosoftScored 41.2%- Solar Mini 4UpstageScored 41.2%
- Qwen3.6 35B A3BQwenScored 37.5%
Granite 4.2 8BIBMScored 36.2%- Ministral 3 14B 2512MistralScored 33.8%
- Mistral Small 4MistralScored 32.5%
Trinity Large ThinkingArcee AIScored 28.8%- Ministral 3 3B 2512MistralScored 26.2%
- Granite 4.0 MicroIBMScored 25.0%
Nova Micro 1.0AmazonScored 22.5%
LFM2.5-2.6BLiquidAISpeed and price measured
- Claude Opus 5.5AnthropicScored 85.0%
- Gemini 4 ArgonGoogleScored 84.8%
- Gemini 3.8 FlashGoogleScored 82.5%
- Claude Sonnet 5.5AnthropicScored 78.8%
- Kimi K3.1 ProMoonshotAIScored 77.5%
- GPT-6.1 SolOpenAIScored 76.2%
- Qwen3.8 Max (0902)QwenScored 75.0%
- DeepSeek V4.1 Flash 1017DeepSeekScored 73.8%
- GPT-6 SolOpenAIScored 73.8%
- GPT-5.4OpenAIScored 72.5%
- GPT-6 LunaOpenAIScored 72.5%
- Claude Haiku 5.5AnthropicScored 68.8%
- MiMo-V2.6-ProXiaomiScored 68.8%
- GLM 5.3Z.aiScored 67.5%
- MiniMax M3.1 FlashMiniMaxScored 66.2%
- Llama 4 MaverickMetaScored 59.0%
- gpt-oss-120bOpenAIScored 58.8%
- Ling 3.0 Flash VLinclusionAISpeed and price measured
- Qwen3.5-122B-A10BQwenSpeed and price measured
- Qwen3 Next 80B A3B ThinkingQwenSpeed and price measured
- Nova Premier 1.0AmazonSpeed and price measured
- GPT-5.5 ProOpenAISpeed and price measured