Data as of · 78 models, 80 questions · reviewed by LLM Verifications team · methodology

FAQ · results from 27 Oct 2026

Frequently asked questions

Answers come straight from our latest measurements and update with every run. Details are on the methodology page.

Results

Which AI model is the most accurate?
Claude Opus 5.5 by Anthropic scored highest in our 2026-10-27 run, answering 68 of 80 questions correctly (85.0%), out of 78 models tested.
Which AI model is the fastest?
Nemotron 3.5 Super VL by NVIDIA had the highest median output speed in our 2026-10-27 run: 715 tokens per second.
Which AI model is the cheapest to run?
Granite 4.0 Micro by IBM had the lowest billed cost per question in our 2026-10-27 run: $0.000051, reasoning tokens included.
Which AI model responds the quickest?
Ministral 3 3B 2512 by Mistral had the shortest median time to first token in our 2026-10-27 run: 0.3 s.
Which AI model gives a complete answer the quickest?
Nemotron 3.5 Super VL by NVIDIA had the shortest median end-to-end time in our 2026-10-27 run: 1.5 seconds to stream a full answer of about 300 words.
Which AI model has the largest context window?
Llama 4 Scout by Meta accepts the most context of the models we test: 1,310,720 tokens, according to OpenRouter's catalogue.
What is the best open-weights AI model?
Muse Spark Open by Meta has the highest verification score among open-weights models we test: 77.5%.
How many AI models do you compare?
78 models from 28 labs, each tested through OpenRouter with the same questions and speed runs.
Which AI lab makes the most accurate model?
Anthropic: Claude Opus 5.5 has the highest verification score we measured in our 2026-10-27 run, at 85.0%.
Which AI lab has the fastest model?
NVIDIA: Nemotron 3.5 Super VL had the highest median output speed in our 2026-10-27 run, at 715 tokens per second.
Which AI lab has the cheapest model?
IBM: Granite 4.0 Micro had the lowest billed cost per question in our 2026-10-27 run, at $0.000051.
Which AI lab has the most models on this site?
OpenAI, with 13 models tested: GPT-5.3-Codex, GPT-6 Astra, GPT-6.1 Terra Pro, GPT-6.1 Sol, GPT-5.6 Terra, GPT-6.1 Terra, GPT-6 Sol, o3, GPT-5.4, GPT-6 Luna, gpt-oss-120b, gpt-oss-20b, GPT-5.5 Pro.
How many AI labs do you compare?
28 labs, 78 models in total.

How we measure

How are these benchmarks measured?
Every model is called through OpenRouter's public API with the same 80 questions, each with one correct answer, plus 3 streamed speed runs. Every request and response is stored and hashed.
Which datasets do the questions come from?
SuperGPQA (50 questions, ODC-BY 1.0) and MMLU-Pro (30 questions, MIT). All are openly licensed and publicly available.
How certain are the scores?
Each model answers the same 80 questions, so a score has a 95% margin of roughly ±10 points. Comparisons only name a more accurate model when a two-proportion test puts the gap beyond chance at the 95% level.
Why is a model listed without a score?
Either it hasn't been run on the current question suite yet, or more than 20% of its requests failed on the host's side during testing, which is too many for a fair score. Its speed, latency and price are still measured.
Is cost per question the list price?
No. It is what the API billed for our questions, divided by the number of questions, so reasoning tokens a model spends thinking are included. List prices are shown separately.