Verdict
Built from the measurements below
- Accuracy is too close to call: Nemotron 3.5 Lightning scored 43.8% and Phi 4 41.2% on the same 80 questions, a gap within what chance alone could produce.
- Phi 4 is 4.1× cheaper per question as billed ($0.00012 vs $0.00049), reasoning tokens included.
- Nemotron 3.5 Lightning generates text 1.1× faster: 116 tokens per second against 103.
- Nemotron 3.5 Lightning accepts more context: 262k tokens against 16k.
Which should you choose?
Choose Nemotron 3.5 Lightning for faster streaming, a 262k-token context window and more hosts to choose from on OpenRouter (4 against 1). Choose Phi 4 for the lower cost per correct answer ($0.00029 against $0.0011) and shorter outputs (about 699 tokens per question against 2,846). The widest subject gap is in humanities & social science: Nemotron 3.5 Lightning answered 6 of 9 correctly, Phi 4 4 of 9. With so few questions per subject, treat this as a lead to test on your own prompts, not a result.
Accuracy gap +2.5 points (z = 0.32). We call a winner only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.