Gemini 3.8 Flash leads 14 models; GLM-5.3 Flash is 6x cheaper
Thirteen new models, tested on six document tasks in five languages: Gemini 3.8 Flash ranks first, GLM-5.3 Flash is the cheap runner-up, and Sonnet 5.5 replaces Sonnet 5.
Published October 12, 2026 · numbers from run 20261010T220935Z-c30495
The latest run adds 13 models to the benchmark, so 14 are now ranked. A document counts as correct only if every field is right. Costs are what the API charged per 1,000 documents. All documents are synthetic, temperature is 0, and languages are cs, de, en, es and fr.
The ranking: two Flash models on top
Gemini 3.8 Flash takes first place with 97.40 % of documents fully right at $4.03. GLM-5.3 Flash, an open-weights model, is second with 96.55 % at $0.68. That is about 6x cheaper than the leader for a gap of less than one point. GPT-6 Sol is third and Claude Sonnet 5.5 fourth.
| Rank | Model | Fully right | USD per 1,000 docs |
|---|---|---|---|
| 1 | Gemini 3.8 Flash | 97.40 % | $4.03 |
| 2 | GLM-5.3 Flash | 96.55 % | $0.68 |
| 3 | GPT-6 Sol | 95.97 % | $4.56 |
| 4 | Claude Sonnet 5.5 | 95.48 % | $5.55 |
| 5 | DeepSeek V4.1 Flash | 93.00 % | $0.45 |
| 6 | GPT-6 Luna | 92.44 % | $0.24 |
Below that: Claude Sonnet 5 (8th, 91.28 %), Mistral Large 4 (9th, 88.99 %), Qwen3.8 Flash (10th, 88.60 %), Gemini 3.5 Flash Lite (11th, 86.99 %), Mistral Medium 3.5 (12th, 86.37 %), Gemma 4 31B (13th, 81.15 %) and Mistral Small 4 (14th, 73.66 %).
List price is a poor guide to cost. Mistral Large 4 lists at $0.68 / $2.09 per million input / output tokens, yet it costs $5.50 per 1,000 documents. Reasoning models write long hidden output that is billed.
Sonnet 5.5 against the model it replaces
Claude Sonnet 5.5 replaces Claude Sonnet 5 and is better on almost every line. Overall it scores 95.48 % against 91.28 %, and it is cheaper at $5.55 against $5.96. On invoices it gets 99.59 % fully right against 92.66 %. On scans and photos it reaches 100.00 %, the best result of all 14 models; Sonnet 5 gets 91.33 %.
Its weak spot is meeting summaries: 89.46 %, while Sonnet 5 gets 77.86 %. Gemini 3.8 Flash and DeepSeek V4.1 Flash both reach 100.00 % there. Email triage is also modest at 84.80 %. See the head-to-head or the Sonnet 5.5 page.
What they get right and wrong, per task
- Contract clauses: Gemini 3.8 Flash 99.73 %, Sonnet 5.5 99.46 %. Gemma 4 31B falls to 56.39 %.
- Email triage: the hardest task. GLM-5.3 Flash leads with 89.06 % at $0.47, just ahead of Gemini 3.8 Flash at 88.93 %. Mistral Medium 3.5 gets 53.73 % and Mistral Small 4 only 35.59 %.
- PII redaction: close to solved for most. Sonnet 5.5 gets 99.60 %, GLM-5.3 Flash and Gemma 4 31B 99.06 %. Gemini 3.5 Flash Lite is the outlier at 84.66 %.
- Questions over a document: GPT-6 Sol, Sonnet 5.5, Mistral Medium 3.5, Gemma 4 31B and Mistral Small 4 all reach 100.00 %. Mistral Large 4 gets 88.53 %, with 4.66 % of documents without a valid answer.
- Meeting summaries: Mistral Medium 3.5 reaches 100.00 % at $1.92. Mistral Large 4 gets 89.33 % at $8.56, and 10.66 % of its documents have no valid answer.
Qwen3.8 Flash shows a similar problem on invoices: 6.66 % of documents without a valid answer, and 75.86 % fully right. Our Mistral Large 4 review covers that model in more detail.
Invoices, scans and the real price
Invoices separate the field well. Each has 13 fields, so one wrong field makes the whole invoice wrong; the article on field versus document accuracy explains why that matters. With a fix cost of 3 minutes per wrong invoice at $40/hour, the true cost per 1,000 invoices looks like this:
- Sonnet 5.5: API $10.65, $19 including fixing.
- Gemini 3.8 Flash: API $5.83, $49 including fixing.
- GLM-5.3 Flash: API $1.00, $110 including fixing.
- GPT-6 Luna: API $0.53, $169 including fixing.
- Mistral Small 4: API $0.43, $998 including fixing.
The cheapest API bill is not the cheapest outcome. On scans and photos the gap widens: Gemini 3.8 Flash gets 98.44 %, GLM-5.3 Flash 93.99 %, GPT-6 Luna 86.00 %, Gemma 4 31B 43.33 % and Mistral Small 4 40.89 %.
Languages
Gemini 3.8 Flash is the most even: its range is 96.44 % (de) to 98.11 % (cs). GPT-6 Sol drops to 93.66 % in both es and fr. GLM-5.3 Flash is weakest in fr at 95.00 % and de at 95.11 %. Gemma 4 31B shows the largest swing, from 91.66 % in en to 73.89 % in fr and 75.44 % in de. DeepSeek V4.1 Flash is weakest in de at 89.44 %.
Which to pick
For the best accuracy, pick Gemini 3.8 Flash. For the best price-to-accuracy ratio, pick GLM-5.3 Flash. For invoices and scans where errors are costly, Sonnet 5.5 is the strongest. GPT-6 Luna at $0.24 is the cheapest model that still scores above 92 %. Avoid Mistral Small 4 and Gemma 4 31B for invoices.
See the full table on the leaderboard and how we test in the methodology.