Chapter 3

Results

Your predictions were: Kimi K3 is the brightest, the others will be on par, and multi-step questions will cause the problems.

Model	Right	$ per 1k questions	Time	Output tokens
GLM 5.3 Flash	23/23	$0.46	2.3s	163
Nemotron 3.5 Lightning thinks out loud	23/23	$0.44	3.8s	1,471
GPT-OSS 120B made up a margin	22/23	$0.51	1.4s	234
MiniMax M3	23/23	$0.59	2.4s	205
DeepSeek V4.1 Flash marker rounding glitch	23/23	$0.89	2.2s	129
GLM 5.3	23/23	$2.09	4.4s	249
Nemotron 3 Ultra	23/23	$2.29	1.8s	211
Qwen 3.8 Max	23/23	$9.67	4.5s	569
Kimi K3 most expensive	23/23	$12.06	8.1s	241

DeepSeek is shown as 23/23 because its one “miss” was a flaw in the marker (see the next chapter). Prices are Fireworks serverless standard tier, checked 14 Oct 2026.

How the predictions did

Prediction	Verdict
Kimi K3 is the brightest	It tied for first, alongside models costing a small fraction of its price
The others will be on par	Right. 8 of 9 are effectively perfect
Multi-step questions cause the issues	Not here. All 7 hard questions were answered correctly by every model

What to take from it

Kimi K3 costs 26× more than GLM 5.3 Flash for the same score, and is 3.5× slower On this kind of work, paying for the biggest model buys nothing. That is the heart of the Fireworks pitch: pick the right-sized model for the job.

Price per token isn’t price per question

Nemotron Lightning is the cheapest per token ($0.05 in, $0.20 out per million). But it writes about 1,470 tokens of thinking per question, against about 160 for GLM Flash. You pay for that thinking, so it ends up the same price per question and slower.

Most of the cost is the schema, not the answer

Each question sends about 2,800 tokens of table descriptions and gets back about 200. The schema is identical every time, and Fireworks charges much less for input it has already seen (“cached input”, around a fifth of the price for GLM Flash). Caching that repeated prefix is the next big saving, and a good thing to bring up in a technical sales conversation.

Recommendation, for now

GLM 5.3 Flash: perfect score, second-cheapest, quick. But see the honest caveat in the next chapter.