Data as of · 78 models, 80 questions · reviewed by LLM Verifications team · methodology

About

About LLM Verifications

Independent LLM leaderboard: accuracy, output speed (tokens/s), latency and billed API cost for GPT, Claude, Gemini and more, from requests we record.

Written and maintained by LLM Verifications team · Data as of

What this site is

LLM Verifications is an independent benchmark of large language models. We send the same questions and the same speed test to every model through a public API, record every request and response, and publish what we measured: how often each model is right, how fast it writes, how long it takes to start, and what it actually costs. Every ranking, chart and comparison on the site is computed from those recordings. Nothing is copied from a vendor's announcement or model card.

The site exists because the numbers that usually decide which model a team picks are hard to compare. Labs report results on different benchmarks, with different prompts and settings, often picked to show their own model at its best. List prices say little about what a question really costs once a model spends hundreds of tokens thinking. Speed claims depend on who is serving the model and where. We wanted one place where every model is measured the same way, with the uncertainty stated, and the measurements kept as evidence.

What we measure

The latest data covers 78 AI language models from 28 labs, served by 70 API hosts, with the most recent run on 27 Oct 2026. For each model we record:

  • Verification score: the share of 80 ten-option multiple-choice questions it answers correctly, drawn from SuperGPQA and MMLU-Pro. These are expert-level questions in subjects such as science, engineering, medicine, economics.
  • Output speed: tokens generated per second once the answer starts streaming, the median of repeated runs of the same prompt.
  • Latency: time to the first token, including any time the model spends thinking first.
  • Cost: what the API actually billed per question, reasoning tokens included, shown next to the list price.
  • Specifications: context window, release date, open or closed weights, and the hosts that serve the model, from OpenRouter's catalogue.

Who we are

The site is run by the LLM Verifications team, a small editorial team that builds and operates the benchmark. The team is credited as the author and reviewer of every page, and is responsible for everything the site publishes. In practice, the team:

  • Runs the benchmark harness. The team chooses which models to test, starts each run, and keeps the proof files, run manifests and harness versions that back every number.
  • Reviews every run before it is published. Each run is checked for host failures, incomplete suites, unusual speed results such as burst delivery, and billed costs that don't match list prices. A run with problems is fixed and re-run rather than edited by hand.
  • Maintains the question suite. The team selects the open datasets, samples the questions with fixed seeds, and records the suite's hash so a change of questions is always visible.
  • Maintains the methodology. When the way we measure something changes, the methodology page and the changelog are updated with it.
  • Handles corrections. Reports of errors go to the team, which checks the underlying records and publishes a correction where one is needed.

The rules the team works by, on independence, sourcing, review and corrections, are set out in our editorial standards.

How we test

Every model is called through OpenRouter's public API with the same questions and settings: temperature 0, reasoning effort "low" for models that support it. Answers are graded automatically against each dataset's answer key, with no human or AI judge, so a verdict can be re-checked from the response text. Speed and latency come from 3 streamed runs of a fixed prompt. Cost comes from the amount OpenRouter billed for each request.

With 80 questions, scores carry real statistical uncertainty, so comparison pages only call a winner when the gap is statistically significant. Requests that fail on the host's side are retried and, if they still fail, left out of the score instead of being counted as wrong. Every request and response is written to its own file and hashed. The full procedure, with a sample question and the live hashes, is on the methodology page; the source datasets are described on the benchmarks page.

Independence and funding

Rankings come only from our measurements. The site does not accept payment from model makers or API hosts for placement, ranking, inclusion or coverage, and there are no sponsored results. We don't accept model submissions: the team decides which models to test, based on what is available on OpenRouter and what readers are likely to compare. The API calls are paid for at normal public prices, the same as any other customer. No lab or host has early access to results or a chance to review them before they are published.

The site uses Google Analytics only if you accept cookies, to see which pages are read. See the privacy policy for details.

What the numbers are good for, and what they aren't

Use them to shortlist. The verification score is a fair, like-for-like measure of how well models handle hard expert-level questions, and the billed cost and measured speed show the practical trade-offs between models in a way list prices and vendor claims can't. The score-versus-cost chart is a quick way to find models that give the most accuracy for the money.

Be careful with them in a few cases:

  • Small gaps. Two models a few points apart are statistically tied on this many questions. Treat them as equals and decide on price, speed or your own tests.
  • Your own task. Multiple-choice knowledge questions don't measure coding agents, long documents, writing quality, tool use or safety behaviour. Always test the shortlisted models on your own prompts.
  • Your own setup. Speed and latency depend on the host and on your distance to it, and prices change. Our figures are a snapshot from a single location on the test date shown on each page.

Where to start

  • The leaderboard lists every model with every measure in one sortable table. Start here if you want the full picture, or sort by cost or speed to find the cheapest or fastest model that clears the accuracy you need.
  • Comparisons put two models side by side, with a written verdict that only names a winner when the difference is larger than chance, and a breakdown by subject.
  • Model pages show one model in depth: its score and rank, speed runs, billed cost against list price, the hosts that serve it, and the date it was last tested.
  • Provider pages group the models by the lab that makes them, and host pages show which API hosts serve which models and at what price.
  • Subject pages break the score down by field, such as science, engineering, medicine or law, with the caveat that each subject has only a few questions.
  • The price calculator turns per-token prices into a monthly bill for your own traffic, and the FAQ answers the most common questions in a sentence or two.

Every data page shows the date its numbers were measured. Results change as models are updated and re-tested, so when you make a decision, check the date and, for close calls, re-check the page before you commit.

How to cite us

You are welcome to quote and cite our results. Please link to the page you took the number from and give the test date shown there. A suggested citation:

LLM Verifications (2026). AI model benchmark results. https://www.llmverifications.com/leaderboard (data as of 27 Oct 2026).

Journalists and researchers will find facts, a short description of the site and our logos on the media page and the brand assets page. Reuse of the data is covered by our terms of use; the source questions remain under their own licences.

Corrections and contact

If you think a number is wrong, tell us which page and which model, and what you expected. The team checks the recorded requests and responses behind it. If we got something wrong, we fix the cause, re-run the affected tests and note the change in the changelog.

Email support@globesword.com, or see the contact page.