Score vs. cost per question
Thinking Machines models in color, every other model in grey
Provider
2 models tested, compared with 76 models from other labs.
Thinking Machines models in color, every other model in grey
Thinking Machines models in color, every other model in grey
Thinking Machines models highlighted against every model in this run · higher is better
Thinking Machines models highlighted against every model in this run · higher is better
Thinking Machines models highlighted against every model in this run · lower is better
Thinking Machines models highlighted against every model in this run · lower is better
Thinking Machines models highlighted against every model in this run · lower is better
Thinking Machines models highlighted against every model in this run · higher is better
| # | ||||||||
|---|---|---|---|---|---|---|---|---|
| 1 | Inkling Small Thinking Machines · served by DeepInfra | 67.5 | 54 / 80 | 233 | 0.5 s | $0.00047 | $0.45 / $1.2 | 524k |
| 2 | Inkling Thinking Machines · served by Together | 66.2 | 53 / 80 | 118 | 0.6 s | $0.0012 | $1 / $4.05 | 524k |