94.25 % accuracy, 49.87 % of invoices wrong
Why field accuracy hides most extraction errors, and what that means for the real cost of AI invoice processing. 12 models, 5 languages.
Published October 6, 2026 · numbers from run 20260930T183434Z-453f94
When a vendor says its AI reads invoices with 94.25 % accuracy, it almost always means fields: of all the supplier names, dates, VAT numbers and totals, 94.25 % came out right. That sounds close to done. It isn’t. What your accounts team cares about is how many invoices go through without anyone touching them, and that number can be as low as 50.13 %.
The gap, measured
We had 12 models extract 13 fields from 150 synthetic invoices in each of five languages (English, German, French, Spanish, Czech), from plain text to scans and phone photos. Here is the share of fields they got right next to the share of invoices where every field was right:
| Model | Fields right | Invoices fully right | API cost / 1,000 | Incl. fixing / 1,000 |
|---|---|---|---|---|
| Claude Sonnet 5.5 | 99.97 % | 99.59 % | $10.65 | $18.69 |
| Gemini 3.8 Flash | 98.84 % | 97.86 % | $5.83 | $48.51 |
| GPT-6 Sol | 99.78 % | 97.20 % | $10.44 | $66.44 |
| GLM-5.3 Flash | 99.52 % | 94.53 % | $1.00 | $110.32 |
| GPT-6 Luna | 99.28 % | 91.60 % | $0.53 | $168.53 |
| DeepSeek V4.1 Flash | 99.31 % | 91.46 % | $0.57 | $171.25 |
| Mistral Medium 3.5 | 96.81 % | 73.73 % | $4.67 | $529.99 |
| Gemma 4 31B | 95.35 % | 62.66 % | $0.20 | $746.84 |
| Mistral Small 4 | 94.25 % | 50.13 % | $0.43 | $997.75 |
(Selection of the 12; averages over the five languages. All models and per-language results are on the leaderboard.)
Mistral Small 4 gets 94.25 % of fields right, and still 49.87 % of invoices have at least one wrong field. Gemma 4: 95.35 % of fields, but more than a third of invoices need a fix.
Why the two numbers drift apart
An invoice has 13 fields. A model that is wrong on 5.75 % of fields doesn’t put all its mistakes on the same few invoices; they spread out. One wrong field is enough for an invoice to need a human. So a small error rate per field becomes a large error rate per document.
It also matters which field is wrong. Mistral Small’s weakest fields are the tax point date (82.33 %) and the VAT ID (84.33 %). Gemma 4’s is the bank account (80.93 %). Even GLM-5.3 Flash is least reliable on the bank account (97.86 %), the one field you least want to get wrong.
What it does to the cost
API prices are the small part of the bill. Assume it takes a person 3 minutes to fix an invoice at $40 an hour, so $2 per wrong invoice. Then per 1,000 invoices:
- Claude Sonnet 5.5 costs $10.65 in API calls and $8.04 in fixes: $18.69.
- Mistral Small 4 costs $0.43 in API calls and $997.32 in fixes: $997.75.
The cheapest API model is the most expensive one to run. GLM-5.3 Flash sits in between: ten times cheaper than Sonnet on the API, but its 5.46 % of imperfect invoices make it $110.32 per 1,000 in total. And all of this assumes you know which invoices are wrong. Without checks such as totals that must add up, VAT ID and IBAN checksums, the errors go straight into your books.
You can change the minutes and the hourly rate for your own case in the recommender.
What to ask before you choose a model
- Accuracy per document, not per field. “Every field right” is the number that maps to touch-free processing.
- Per field. Which fields fail matters as much as how often: a wrong bank account is not a wrong delivery address.
- In your language and on your input quality. Scans and phone photos are harder; GLM-5.3 Flash drops to 93.99 % there while Sonnet 5.5 stays at 100 %.
- On your own documents, if you can. Synthetic invoices are a fair comparison, not a guarantee.
Every number here links back to the actual model outputs: open any model on the leaderboard or browse the examples. How we generate the invoices and score the answers is in the methodology.