Medicine: score by model
11 questions · the bar shows each score's 95% interval
| # | Model | Correct | Score | 95% interval |
|---|---|---|---|---|
| 1 |  Claude Fable 5.1Anthropic | 7 / 11 | 63.6% | 35–85% |
| 1 |  o3OpenAI · statistically tied with #1 | 7 / 11 | 63.6% | 35–85% |
| 1 | GPT-5.3-CodexOpenAI · statistically tied with #1 | 7 / 11 | 63.6% | 35–85% |
| 1 |  Gemini 3.5 Flash LiteGoogle · statistically tied with #1 | 7 / 11 | 63.6% | 35–85% |
| 1 | Gemini 3.1 Pro PreviewGoogle · statistically tied with #1 | 7 / 11 | 63.6% | 35–85% |
| 1 | Claude Opus 5.5Anthropic · statistically tied with #1 | 7 / 11 | 63.6% | 35–85% |
| 1 | Gemini 4 ArgonGoogle · statistically tied with #1 | 7 / 11 | 63.6% | 35–85% |
| 1 | Claude Sonnet 5.5Anthropic · statistically tied with #1 | 7 / 11 | 63.6% | 35–85% |
| 9 |  InklingThinking Machines · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 |  Command ACohere · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 |  Llama 4 ScoutMeta · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 |  Qwen3.7 PlusQwen · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 | GPT-6 AstraOpenAI · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 |  Grok 4.7SpaceXAI · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 | GPT-6.1 TerraOpenAI · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 | GPT-6.1 Terra ProOpenAI · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 |  Ministral 3 14B 2512Mistral · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 |  MiMo-V2.6-FlashXiaomi · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 | Llama 3.3 70B InstructMeta · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 | Qwen3.6 35B A3BQwen · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 | Inkling SmallThinking Machines · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 | Muse Spark OpenMeta · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 | Gemini 3.8 FlashGoogle · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 | GPT-6 LunaOpenAI · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 | MiMo-V2.6-ProXiaomi · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 |  Kimi K3.1 ProMoonshotAI · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 9 | Qwen3.8 Max (0902)Qwen · statistically tied with #1 | 6 / 11 | 54.5% | 28–79% |
| 28 | Llama 4 MaverickMeta · statistically tied with #1 | 5 / 10 | 50.0% | 24–76% |
| 29 | Kimi K2.7 CodeMoonshotAI · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 | Muse WonderMeta · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 | Ministral 3 8B 2512Mistral · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 |  GLM 5.3 FlashZ.ai · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 |  Solar Pro 4Upstage · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 | Mistral Small 4Mistral · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 | Qwen3 Coder NextQwen · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 | Qwen3.8 27BQwen · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 | GLM 5.3Z.ai · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 | GPT-6.1 SolOpenAI · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 | Claude Haiku 5.5Anthropic · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 | GPT-6 SolOpenAI · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 29 |  MiniMax M3.1 FlashMiniMax · statistically tied with #1 | 5 / 11 | 45.5% | 21–72% |
| 42 |  Step 5 PreviewStepFun · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 | Qwen3.8 2.4T A95BQwen · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 | GPT-5.6 TerraOpenAI · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 | gpt-oss-20bOpenAI · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 |  Granite 4.0 MicroIBM · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 | Ministral 3 3B 2512Mistral · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 |  Mercury 2.5Inception · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 |  Phi 4Microsoft · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 |  Nemotron 3.5 LightningNVIDIA · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 |  Hy3Tencent · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 |  DeepSeek V4 Flash Vision ExpDeepSeek · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 |  LongCat 2.0Meituan · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 | Nemotron 3 UltraNVIDIA · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 | DeepSeek V4 Pro 0813DeepSeek · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 | DeepSeek V4.1 Flash 1017DeepSeek · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 | GPT-5.4OpenAI · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 42 | gpt-oss-120bOpenAI · statistically tied with #1 | 4 / 11 | 36.4% | 15–65% |
| 59 | Mistral Medium 3.5Mistral · statistically tied with #1 | 3 / 11 | 27.3% | 10–57% |
| 59 |  Hermes 4 405BNous · statistically tied with #1 | 3 / 11 | 27.3% | 10–57% |
| 59 |  Nova Micro 1.0Amazon · statistically tied with #1 | 3 / 11 | 27.3% | 10–57% |
| 59 |  Trinity Large ThinkingArcee AI · statistically tied with #1 | 3 / 11 | 27.3% | 10–57% |
| 59 | MiniMax M3MiniMax · statistically tied with #1 | 3 / 11 | 27.3% | 10–57% |
| 64 | Nemotron 3.5 Super VLNVIDIA · statistically tied with #1 | 2 / 11 | 18.2% | 5–48% |
| 64 | Gemma 4 31BGoogle · statistically tied with #1 | 2 / 11 | 18.2% | 5–48% |
| 64 |  Ling 3.1 FlashinclusionAI · statistically tied with #1 | 2 / 11 | 18.2% | 5–48% |
| 64 | Qwen3.5 397B A17BQwen · statistically tied with #1 | 2 / 11 | 18.2% | 5–48% |
| 64 | Solar Mini 4Upstage · statistically tied with #1 | 2 / 11 | 18.2% | 5–48% |
| 64 | Mistral Large 4Mistral · statistically tied with #1 | 2 / 11 | 18.2% | 5–48% |
| 70 |  Reka Flash 3rekaai · statistically tied with #1 | 1 / 11 | 9.1% | 2–38% |
| 70 | Gemma 4 26B A4BGoogle · statistically tied with #1 | 1 / 11 | 9.1% | 2–38% |
| 70 | Granite 4.2 8BIBM · statistically tied with #1 | 1 / 11 | 9.1% | 2–38% |