Summary
- Answers 36 of 80 verification questions correctly (45.0%), placing #49 of 72 models tested on the current suite.
- Generates 7 output tokens per second, faster than 1% of the models we measure.
- Starts responding after 1.8 s (median time to first token).
- Costs $0.007 per question as billed, #15 cheapest of 31 models priced over $1 per 1M tokens.