Language gap
Language gap: how much worse is a model outside English
Same kinds of invoices, same instructions, different language. The gap shows how much accuracy a model loses on documents in another language compared with English ones.
| Model | Correct fields 82% – 100% | English | German | Gap | Error-free documents (German / English) |
|---|---|---|---|---|---|
| Qwen3.8 Flash Qwen | 82.4% | 93.4% | +11.0 pp | 87% / 57% | |
| Qwen3.8 Flash Qwen | 82.4% | 91.3% | +9.0 pp | 77% / 57% | |
| Qwen3.8 Flash Qwen | 82.4% | 90.5% | +8.1 pp | 82% / 57% | |
| Qwen3.8 Flash Qwen | 82.4% | 88.2% | +5.8 pp | 78% / 57% | |
| Mistral Small 4 Mistral | 95.6% | 97.5% | +1.9 pp | 77% / 58% | |
| Gemini 3.5 Flash Lite | 99.2% | 100.0% | +0.8 pp | 100% / 92% | |
| DeepSeek V4.1 Flash DeepSeek | 99.4% | 99.9% | +0.5 pp | 99% / 93% | |
| Gemini 3.5 Flash Lite | 99.2% | 99.6% | +0.4 pp | 96% / 92% | |
| Mistral Medium 3.5 Mistral | 97.8% | 98.1% | +0.4 pp | 81% / 79% | |
| Mistral Medium 3.5 Mistral | 97.8% | 98.0% | +0.2 pp | 83% / 79% | |
| DeepSeek V4.1 Flash DeepSeek | 99.4% | 99.6% | +0.2 pp | 95% / 93% | |
| Claude Sonnet 5 Anthropic | 99.8% | 100.0% | +0.2 pp | 100% / 99% | |
| Gemini 3.5 Flash Lite | 99.2% | 99.4% | +0.2 pp | 93% / 92% | |
| Gemini 3.8 Flash | 99.8% | 99.9% | +0.1 pp | 99% / 98% | |
| Gemini 3.8 Flash | 99.8% | 99.9% | +0.1 pp | 99% / 98% | |
| DeepSeek V4.1 Flash DeepSeek | 99.4% | 99.4% | −0.0 pp | 93% / 93% | |
| GPT-6 Sol OpenAI | 99.9% | 99.9% | −0.0 pp | 99% / 99% | |
| GPT-6 Sol OpenAI | 99.9% | 99.9% | −0.0 pp | 99% / 99% | |
| GPT-6 Sol OpenAI | 99.9% | 99.9% | −0.1 pp | 99% / 99% | |
| Mistral Medium 3.5 Mistral | 97.8% | 97.6% | −0.1 pp | 75% / 79% | |
| Mistral Small 4 Mistral | 95.6% | 95.4% | −0.2 pp | 56% / 58% | |
| Mistral Small 4 Mistral | 95.6% | 95.2% | −0.4 pp | 65% / 58% | |
| GPT-6 Luna OpenAI | 99.7% | 99.3% | −0.4 pp | 94% / 97% | |
| GPT-6 Luna OpenAI | 99.7% | 99.2% | −0.5 pp | 91% / 97% | |
| Gemini 3.5 Flash Lite | 99.2% | 98.6% | −0.6 pp | 85% / 92% | |
| GPT-6 Luna OpenAI | 99.7% | 98.9% | −0.8 pp | 88% / 97% | |
| GPT-6 Luna OpenAI | 99.7% | 98.9% | −0.8 pp | 88% / 97% | |
| GPT-6 Sol OpenAI | 99.9% | 99.0% | −0.9 pp | 89% / 99% | |
| Claude Sonnet 5 Anthropic | 99.8% | 98.8% | −1.0 pp | 87% / 99% | |
| Mistral Small 4 Mistral | 95.6% | 94.5% | −1.1 pp | 53% / 58% | |
| Claude Sonnet 5 Anthropic | 99.8% | 98.6% | −1.2 pp | 96% / 99% | |
| DeepSeek V4.1 Flash DeepSeek | 99.4% | 98.0% | −1.3 pp | 78% / 93% | |
| Gemma 4 31B | 97.6% | 96.1% | −1.5 pp | 67% / 81% | |
| Gemma 4 31B | 97.6% | 95.2% | −2.4 pp | 67% / 81% | |
| Gemini 3.8 Flash | 99.8% | 97.3% | −2.5 pp | 97% / 98% | |
| Mistral Medium 3.5 Mistral | 97.8% | 94.9% | −2.9 pp | 67% / 79% | |
| Gemma 4 31B | 97.6% | 94.7% | −3.0 pp | 54% / 81% | |
| Gemini 3.8 Flash | 99.8% | 95.9% | −3.9 pp | 95% / 98% | |
| Gemma 4 31B | 97.6% | 93.3% | −4.3 pp | 52% / 81% | |
| Claude Sonnet 5 Anthropic | 99.8% | 93.4% | −6.5 pp | 77% / 99% |
Only fields present in both versions of the document are compared (Supplier, Supplier ID (EIN, IČO…), Invoice number, Payment reference, Invoice date, Due date, Net by tax rate, Tax by rate, Total, Currency, Bank account). US invoices have no VAT ID or tax point, so those fields are left out.